Skip to content

use fully fledged async, NDPositions improvements#1050

Open
kushalkolar wants to merge 16 commits into
ndwidgetfrom
ndwidget-proper-async
Open

use fully fledged async, NDPositions improvements#1050
kushalkolar wants to merge 16 commits into
ndwidgetfrom
ndwidget-proper-async

Conversation

@kushalkolar
Copy link
Copy Markdown
Member

@kushalkolar kushalkolar commented May 9, 2026

This replaces the generator-as-a-couroutine with proper full async and uses rendercanvas's existing loop for scheduling. Throttling behavior is much more natural now and IO never blocks rendering even when waiting for Futures to resolve. The entire pipeline from getting the raw data --> window_funcs --> spatial_func is async and fully concurrent.

closes #1042

@apasarkar this should allow concurrent running of window_funcs and spatial_funcs written using torch that stay on the device until the very end, ex: torch.mean() for rolling mean windows instead of np.mean.

  • need to benchmark and verify that the performance is just as good as before, especially with cuda arrays from masknmf.
  • NDWSubplot.subplot property @apasarkar
  • NDGraphic._set_indices_() doesn't take any arguments, it retrieves the latest indices directly from the ref_index instance. This is because since _set_indices_() is a scheduled request AND the indices dict is produced at the time it's scheduled in _render_indices(), the indices argument will not reflect the latest indices at execution time. So NDGraphic._set_indices_() just always grabs the latest indices when it executes.
  • better x_range_mode logic in NDPostions, get rid of jittery and jigglyness when x_range_mode="auto", fix other bugs. _update_from_view_range() uses with block_indices_ctx() to any async scheduling of _set_indices_() when x_range_mode="auto" but we allow async scheduling to update all other NDGraphics. We then run_sync(self._set_indices_()) immediately so that the NDPositions graphic immediately reflects the latest polled camera state. This gets rid of the jitter and jiggly effect! 🥳
    • no x_range fighting when setting indices via _set_indices_
    • no x_range fighting with _update_from_view_range() - trying to figure this out
    • fix bug when width was 0 due to figure auto_scale when using VisibilitySelector and no lines are visible (i.e. initial selection is empty, blank subplot).
    • only one NDPositions LinearSelector per-subplot, so multiple timeseries can be overlaid in one subplot and they use the same linear selector.
  • better PlotArea.x_range and y_range setters and getters.
  • keep UI throttling for imgui slider, and also use Future cancellation in ReferenceIndex render request logic.
  • No throttling for other forms of viewing data, ex: play button, stepping through frames, using the LinearSelector, we want to see every single datapoint/frame of data when doing fine-grained inspection like this.
  • make sure new async works when slider is moved around very fast with multi-video decoding with fava - almost there
  • similarly, make sure new async works when viewing masknmf.DemixingResults and moving the slider very fast.
  • tested examples: masknmf single session, ephys, kcenia behavior, gerbils. WIP masknmf multi session, things wonky with timeseries.
  • cleanup comments and docstrings

@BalzaniEdoardo

@kushalkolar kushalkolar requested a review from clewis7 as a code owner May 9, 2026 08:48
@kushalkolar kushalkolar changed the title use fully fledged async use fully fledged async, NDPositions improvements May 10, 2026
@kushalkolar
Copy link
Copy Markdown
Member Author

kushalkolar commented May 11, 2026

No more jitter and jiggly effect in x_range_mode="auto"! Rock-solid.

no-jitter-2026-05-11_03.16.52.mp4

IO such as video decoding in an independent process + new async lets us get 120+fps easily. We still get ~100fps+ rendering when we set the playback fps to 100. With the previous generator-as-a-coroutine simpler async without rendercanvas scheduling we got ~60fps, so we can get almost double the fps with this proper async that uses rendercanvas scheduling! 🥳 . The decoder does get stuck for a few seconds if a huge backlog piles up I think, but from inspecting the stack this lag comes from the decoder. Anyways the frame corresponding to the latest indices do appear within a few seconds and I think it's unlikely a user will be scrolling such that they request 100 different random frames/second for long periods of time, or playback at > 100fps.

gerbies-async-2026-05-11_03.25.59.mp4

With masknmf demixing results with many subplots etc. (this) I get ~28fps with the new propery async, 17 fps before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant