[WIP] add camera spline widget

[kevinyamauchi]

This PR adds a widget for annotating a spline path through the data and then using it to set the camera path.

[kevinyamauchi]

I need to cut a new release of napari-animation napari-threedee for this PR to work. I will do so this week.

[alisterburt]

sorry, do you mean a new release of napari-threedee? 🙂

[kevinyamauchi]

Sorry - yes!

[psobolewskiPhD]

Test fails are due to hard-coded PyQt5 import, I have made a PR upstream to fix:
napari-threedee/napari-threedee#147

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (521d0b3) 86.23% compared to head (3d27ee4) 86.36%.

Files	Patch %	Lines
napari_animation/_qt/camera_spline_widget.py	93.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #159      +/-   ##
==========================================
+ Coverage   86.23%   86.36%   +0.13%     
==========================================
  Files          26       27       +1     
  Lines        1075     1093      +18     
==========================================
+ Hits          927      944      +17     
- Misses        148      149       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[psobolewskiPhD]

Maybe I'm not using it properly, but I used the camera spline path to make a spline path and that works (modulo the broken set from current view button). I then tried to add keyframes and I get a Traceback:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

ValueError                                Traceback (most recent call last)
File ~/Documents/dev/napari-animation/napari_animation/_qt/animation_widget.py:118, in AnimationWidget._capture_keyframe_callback(self=<napari_animation._qt.animation_widget.AnimationWidget object>, event=False)
    116 def _capture_keyframe_callback(self, event=None):
    117     """Record current key-frame"""
--> 118     self.animation.capture_keyframe(**self._input_state())
        self.animation = <napari_animation.animation.Animation object at 0x2a5ec0370>
        self = <napari_animation._qt.animation_widget.AnimationWidget object at 0x2a19365f0>

File ~/Documents/dev/napari-animation/napari_animation/animation.py:98, in Animation.capture_keyframe(self=<napari_animation.animation.Animation object>, steps=15, ease=<Easing.LINEAR: functools.partial(<function linear_interpolation at 0x168cd9ab0>)>, insert=True, position=-1)
     95 new_frame.name = f"Key Frame {next(self._keyframe_counter)}"
     97 if insert:
---> 98     self.key_frames.insert(position + 1, new_frame)
        new_frame = <KeyFrame: Key Frame 0>
        position = -1
        self = <napari_animation.animation.Animation object at 0x2a5ec0370>
        position + 1 = 0
        self.key_frames = [<KeyFrame: Key Frame 0>]
     99 else:
    100     self.key_frames[position] = new_frame

File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/containers/_selectable_list.py:69, in SelectableEventedList.insert(self=[<KeyFrame: Key Frame 0>], index=0, value=<KeyFrame: Key Frame 0>)
     66 super().insert(index, value)
     67 if self._activate_on_insert:
     68     # Make layer selected and unselect all others
---> 69     self.selection.active = value
        value = <KeyFrame: Key Frame 0>
        self = [<KeyFrame: Key Frame 0>]

File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/containers/_selection.py:108, in Selection.active(self=Selection({<KeyFrame: Key Frame 0>}), value=<KeyFrame: Key Frame 0>)
    106 self.clear() if value is None else self.select_only(value)
    107 self._current = value
--> 108 self.events.active(value=value)
        value = <KeyFrame: Key Frame 0>
        self = Selection({<KeyFrame: Key Frame 0>})
        self.events.active = <napari.utils.events.event.EventEmitter object at 0x2a1d3dc00>
        self.events = <napari.utils.events.event.EmitterGroup object at 0x2a1afbe50>

File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/event.py:768, in EventEmitter.__call__(self=<napari.utils.events.event.EventEmitter object>, *args=(), **kwargs={'value': <KeyFrame: Key Frame 0>})
    765     self._block_counter.update([cb])
    766     continue
--> 768 self._invoke_callback(cb, event if pass_event else None)
        event = <Event blocked=False handled=False native=None source=None sources=[] type='active'>
        self = <napari.utils.events.event.EventEmitter object at 0x2a1d3dc00>
        cb = <bound method Animation._on_active_keyframe_changed of <napari_animation.animation.Animation object at 0x2a5ec0370>>
        pass_event = True
    769 if event.blocked:
    770     break

File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/event.py:806, in EventEmitter._invoke_callback(self=<napari.utils.events.event.EventEmitter object>, cb=<bound method Animation._on_active_keyframe_chan...of <napari_animation.animation.Animation object>>, event=<Event blocked=False handled=False native=None source=None sources=[] type='active'>)
    804     self.disconnect(cb)
    805     return
--> 806 _handle_exception(
        self = <napari.utils.events.event.EventEmitter object at 0x2a1d3dc00>
        event = <Event blocked=False handled=False native=None source=None sources=[] type='active'>
        cb = <bound method Animation._on_active_keyframe_changed of <napari_animation.animation.Animation object at 0x2a5ec0370>>
        (cb, event) = (<bound method Animation._on_active_keyframe_changed of <napari_animation.animation.Animation object at 0x2a5ec0370>>, <Event blocked=False handled=False native=None source=None sources=[] type='active'>)
    807     self.ignore_callback_errors,
    808     self.print_callback_errors,
    809     self,
    810     cb_event=(cb, event),
    811 )

File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/event.py:793, in EventEmitter._invoke_callback(self=<napari.utils.events.event.EventEmitter object>, cb=<bound method Animation._on_active_keyframe_chan...of <napari_animation.animation.Animation object>>, event=<Event blocked=False handled=False native=None source=None sources=[] type='active'>)
    791 try:
    792     if event is not None:
--> 793         cb(event)
        event = <Event blocked=False handled=False native=None source=None sources=[] type='active'>
        cb = <bound method Animation._on_active_keyframe_changed of <napari_animation.animation.Animation object at 0x2a5ec0370>>
    794     else:
    795         cb()

File ~/Documents/dev/napari-animation/napari_animation/animation.py:268, in Animation._on_active_keyframe_changed(self=<napari_animation.animation.Animation object>, event=<Event blocked=False handled=False native=None source=None sources=[] type='active'>)
    266 if active_keyframe:
    267     keyframe_index = self.key_frames.index(active_keyframe)
--> 268     self.set_key_frame_index(keyframe_index)
        keyframe_index = 0
        self = <napari_animation.animation.Animation object at 0x2a5ec0370>

File ~/Documents/dev/napari-animation/napari_animation/animation.py:120, in Animation.set_key_frame_index(self=<napari_animation.animation.Animation object>, index=0)
    118 def set_key_frame_index(self, index: int):
    119     frame_index = self._keyframe_frame_index(index)
--> 120     self.set_movie_frame_index(frame_index)
        frame_index = 0
        self = <napari_animation.animation.Animation object at 0x2a5ec0370>

File ~/Documents/dev/napari-animation/napari_animation/animation.py:134, in Animation.set_movie_frame_index(self=<napari_animation.animation.Animation object>, index=0)
    131     if self.key_frames.selection.active != key_frame:
    132         self.key_frames.selection.active = key_frame
--> 134     self._frames.set_movie_frame_index(self.viewer, index)
        index = 0
        self.viewer = Viewer(camera=Camera(center=(62.99999999999997, 77.71788787841795, 64.83293151855469), zoom=3.5162095392142843, angles=(0.6395245182826729, 33.01808292794204, 1.3184949621363917), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(143.90210915518512, 135.01486633648213, 155.84283798168246), scaled=True, size=1, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=3, ndisplay=3, last_used=0, range=((0.0, 128.0, 1.0), (0.0, 128.0, 1.0), (0.0, 128.0, 1.0)), current_step=(76, 63, 63), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'binary_blobs' at 0x2a1afb970>, <Points layer 'n3d paths' at 0x2a5e9cca0>, <Shapes layer 'n3d paths (smooth fit)' at 0x2a5dd6620>], help='use <2> for transform', status='', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x12dfa8790>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={'Alt-F': <bound method AnimationWidget._capture_keyframe_callback of <napari_animation._qt.animation_widget.AnimationWidget object at 0x2a19365f0>>, 'Alt-R': <bound method AnimationWidget._replace_keyframe_callback of <napari_animation._qt.animation_widget.AnimationWidget object at 0x2a19365f0>>, 'Alt-D': <bound method AnimationWidget._delete_keyframe_callback of <napari_animation._qt.animation_widget.AnimationWidget object at 0x2a19365f0>>, 'Alt-A': <function AnimationWidget._add_keybind_callbacks.<locals>.<lambda> at 0x287bc5d80>, 'Alt-B': <function AnimationWidget._add_keybind_callbacks.<locals>.<lambda> at 0x287bc7400>})
        self = <napari_animation.animation.Animation object at 0x2a5ec0370>
        self._frames = <napari_animation.frame_sequence.FrameSequence object at 0x287a2ef20>
    135     self._current_frame = index
    137 except KeyError:

File ~/Documents/dev/napari-animation/napari_animation/frame_sequence.py:162, in FrameSequence.set_movie_frame_index(self=<napari_animation.frame_sequence.FrameSequence object>, viewer=Viewer(camera=Camera(center=(62.99999999999997, ...ind_callbacks.<locals>.<lambda> at 0x287bc7400>}), index=0)
    161 def set_movie_frame_index(self, viewer: napari.viewer.Viewer, index: int):
--> 162     self[index].apply(viewer)
        index = 0
        viewer = Viewer(camera=Camera(center=(62.99999999999997, 77.71788787841795, 64.83293151855469), zoom=3.5162095392142843, angles=(0.6395245182826729, 33.01808292794204, 1.3184949621363917), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(143.90210915518512, 135.01486633648213, 155.84283798168246), scaled=True, size=1, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=3, ndisplay=3, last_used=0, range=((0.0, 128.0, 1.0), (0.0, 128.0, 1.0), (0.0, 128.0, 1.0)), current_step=(76, 63, 63), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'binary_blobs' at 0x2a1afb970>, <Points layer 'n3d paths' at 0x2a5e9cca0>, <Shapes layer 'n3d paths (smooth fit)' at 0x2a5dd6620>], help='use <2> for transform', status='', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x12dfa8790>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={'Alt-F': <bound method AnimationWidget._capture_keyframe_callback of <napari_animation._qt.animation_widget.AnimationWidget object at 0x2a19365f0>>, 'Alt-R': <bound method AnimationWidget._replace_keyframe_callback of <napari_animation._qt.animation_widget.AnimationWidget object at 0x2a19365f0>>, 'Alt-D': <bound method AnimationWidget._delete_keyframe_callback of <napari_animation._qt.animation_widget.AnimationWidget object at 0x2a19365f0>>, 'Alt-A': <function AnimationWidget._add_keybind_callbacks.<locals>.<lambda> at 0x287bc5d80>, 'Alt-B': <function AnimationWidget._add_keybind_callbacks.<locals>.<lambda> at 0x287bc7400>})
        self = <napari_animation.frame_sequence.FrameSequence object at 0x287a2ef20>
    163     self._current_index = index

File ~/Documents/dev/napari-animation/napari_animation/viewer_state.py:62, in ViewerState.apply(self=ViewerState(camera={'center': (62.99999999999997...aults': Empty DataFrame
Columns: []
Index: [0]}}), viewer=Viewer(camera=Camera(center=(62.99999999999997, ...ind_callbacks.<locals>.<lambda> at 0x287bc7400>}))
     59 original_value = layer_attributes[attribute_name]
     60 # Only setattr if value has changed to avoid expensive redraws
     61 # dicts can hold arrays, e.g. `color`, requiring comparisons of key/value pairs
---> 62 if layer_attribute_changed(value, original_value):
        value = {'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object}
        original_value = {'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object}
     63     with contextlib.suppress(AttributeError):
     64         setattr(layer, attribute_name, value)

File ~/Documents/dev/napari-animation/napari_animation/utils.py:44, in layer_attribute_changed(value={'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object}, original_value={'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object})
     39     if (
     40         not isinstance(original_value, dict)
     41         or value.keys() != original_value.keys()
     42     ):
     43         return True
---> 44     return any(
        value = {'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object}
        original_value = {'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object}
     45         layer_attribute_changed(value[key], original_value[key])
     46         for key in value
     47     )
     48 return not np.array_equal(value, original_value)

File ~/Documents/dev/napari-animation/napari_animation/utils.py:45, in <genexpr>(.0=<dict_keyiterator object>)
     39     if (
     40         not isinstance(original_value, dict)
     41         or value.keys() != original_value.keys()
     42     ):
     43         return True
     44     return any(
---> 45         layer_attribute_changed(value[key], original_value[key])
        value = {'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object}
        original_value = {'path_id': <class 'numpy.ndarray'> (6,) int64, 'spline_color': <class 'numpy.ndarray'> (6,) object}
        value[key] = <class 'numpy.ndarray'> (6,) object
        original_value[key] = <class 'numpy.ndarray'> (6,) object
        key = 'spline_color'
     46         for key in value
     47     )
     48 return not np.array_equal(value, original_value)

File ~/Documents/dev/napari-animation/napari_animation/utils.py:48, in layer_attribute_changed(value=<class 'numpy.ndarray'> (6,) object, original_value=<class 'numpy.ndarray'> (6,) object)
     43         return True
     44     return any(
     45         layer_attribute_changed(value[key], original_value[key])
     46         for key in value
     47     )
---> 48 return not np.array_equal(value, original_value)
        value = <class 'numpy.ndarray'> (6,) object
        np.array_equal = <function array_equal at 0x106899630>
        original_value = <class 'numpy.ndarray'> (6,) object
        np = <module 'numpy' from '/Users/sobolp/micromamba/envs/napari-418/lib/python3.10/site-packages/numpy/__init__.py'>

File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/numpy/core/numeric.py:2439, in array_equal(a1=<class 'numpy.ndarray'> (6,) object, a2=<class 'numpy.ndarray'> (6,) object, equal_nan=False)
   2437     return False
   2438 if not equal_nan:
-> 2439     return bool(asarray(a1 == a2).all())
        a1 = <class 'numpy.ndarray'> (6,) object
        a2 = <class 'numpy.ndarray'> (6,) object
   2440 # Handling NaN values if equal_nan is True
   2441 a1nan, a2nan = isnan(a1), isnan(a2)

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Looks like what I implemented in:
#181
doesn't cover path_id and spline_color properly.

Edit:
Aha, they're features in the points layer:
viewer.layers['n3d paths'].as_layer_data_tuple()

(array([[ 63.        ,  77.71789097,  64.83292989],
        [ 63.        , 109.17496836,  37.81894252],
        [ 63.        ,  19.60227341,  98.4226905 ],
        [ 76.        ,  83.93821701, 102.86578053],
        [ 76.        , 108.99724476,  71.76415033],
        [ 76.        ,  62.25593767,  21.46837122]]),
 {'name': 'n3d paths',
  'metadata': {'n3d_metadata': {'annotation_type': 'path'}},
  'scale': [1.0, 1.0, 1.0],
  'translate': [0.0, 0.0, 0.0],
  'rotate': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]],
  'shear': [0.0, 0.0, 0.0],
  'affine': array([[1., 0., 0., 0.],
         [0., 1., 0., 0.],
         [0., 0., 1., 0.],
         [0., 0., 0., 1.]]),
  'opacity': 1.0,
  'blending': 'translucent',
  'visible': True,
  'experimental_clipping_planes': [],
  'symbol': array([<Symbol.DISC: 'disc'>, <Symbol.DISC: 'disc'>,
         <Symbol.DISC: 'disc'>, <Symbol.DISC: 'disc'>,
         <Symbol.DISC: 'disc'>, <Symbol.DISC: 'disc'>], dtype=object),
  'edge_width': array([0.05, 0.05, 0.05, 0.05, 0.05, 0.05]),
  'edge_width_is_relative': True,
  'face_color': array([[0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ]], dtype=float32),
  'face_color_cycle': array([[0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [1.        , 0.49803922, 0.05490196, 1.        ],
         [0.17254902, 0.627451  , 0.17254902, 1.        ],
         [0.8392157 , 0.15294118, 0.15686275, 1.        ],
         [0.5803922 , 0.40392157, 0.7411765 , 1.        ],
         [0.54901963, 0.3372549 , 0.29411766, 1.        ],
         [0.8901961 , 0.46666667, 0.7607843 , 1.        ],
         [0.49803922, 0.49803922, 0.49803922, 1.        ],
         [0.7372549 , 0.7411765 , 0.13333334, 1.        ],
         [0.09019608, 0.74509805, 0.8117647 , 1.        ]], dtype=float32),
  'face_colormap': 'viridis',
  'face_contrast_limits': None,
  'edge_color': array([[0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ]]),
  'edge_color_cycle': array([[1., 1., 1., 1.]], dtype=float32),
  'edge_colormap': 'viridis',
  'edge_contrast_limits': None,
  'properties': {'path_id': array([0, 0, 0, 0, 0, 0]),
   'spline_color': array([array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32)],
         dtype=object)},
  'property_choices': {},
  'text': {'string': {'constant': array('', dtype='<U1'),
    'encoding_type': 'ConstantStringEncoding'},
   'color': {'constant': array([0., 1., 1., 1.], dtype=float32),
    'encoding_type': 'ConstantColorEncoding'},
   'visible': True,
   'size': 12,
   'blending': <Blending.TRANSLUCENT: 'translucent'>,
   'anchor': <Anchor.CENTER: 'center'>,
   'translation': array(0.),
   'rotation': 0.0},
  'out_of_slice_display': False,
  'n_dimensional': False,
  'size': array([10, 10, 10, 10, 10, 10]),
  'ndim': 3,
  'features':    path_id                              spline_color
  0        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  1        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  2        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  3        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  4        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  5        0  [0.12156863, 0.46666667, 0.7058824, 1.0],
  'feature_defaults':    path_id  spline_color
  0        0           NaN,
  'shading': <Shading.NONE: 'none'>,
  'antialiasing': 1,
  'canvas_size_limits': (2.0, 10000.0),
  'shown': array([ True,  True,  True,  True,  True,  True])},
 'points')

...and features are Pandas:

type(viewer.layers['n3d paths'].as_layer_data_tuple()[1]['features'])
Out[8]: pandas.core.frame.DataFrame

[kevinyamauchi]

Thanks for the feedback @psobolewskiPhD ! This might be related to the properties -> features migration on the napari side.

Unfortunately, I don't think I will have the time to push this across the line in the coming weeks, so I think it's best that we close this PR for now. Hopefully I (or someone else) can pick it up in the future!