Hey,

I am having the following usecase: frame sequence (t, y, x) as well as bounding boxes annotations (for example centerXY, width, height), and maybe additional label/flags etc. This is for vizualizing tracking data in a video, perhaps I can find a public sample to share later.

So I figured something out using the ImageWidget to show 1. the full screen and 2. a crop into the bounding box. The important part is that bounding boxes change in size and aspect ratio over time. So I couldn't precrop them all into a 2nd array and also want to see some margins outside the bounding box which gets tricky on the edges. Instead I just changed the pygfx camera which felt a bit awkward. And also needed me to double the data and bounding box lines. I feel like there just just be one scene with two cameras/views (as the two different subplots).
Perhaps as an advanced user I would also love to write my own fragment shader where I get the first resulting texture (cmap, lines, zoom) and my array data as uniforms to output a texture that is displayed in a second sbplot/figure. So I can implement my own zoom and other effects.

here is the snippet without mock data just yet.

# movie (t, y, x), bboxes (t, 4) xywh, lines (t, 5, 2)
iw = fpl.ImageWidget(data=[movie, movie], rgb=False, cmap="gray", figure_shape=(2,1), figure_kwargs={"size" : (650, 900)})
figure = iw.figure
bbox = figure[0,0].add_line(lines[0], colors="red")
bbox2 = figure[1,0].add_line(lines[0], colors="red", alpha=0.5)
figure[1,0].axes.visible = False
figure[1,0].toolbar = False
# TODO: disable camera control for crop, gang cmaps

def set_bbox(index):
    bbox.data[:,0:2] = lines[index["t"]]
    bbox2.data[:,0:2] = lines[index["t"]]

def set_crop(index):
    t = index["t"]
    long_edge = max(bboxes[t,2], bboxes[t,3])
    if long_edge: # to not crash on DIV by 0
      # zoom in so it's always showing the whole box + some margin
      scale = (1.0/long_edge) * 0.55
      # move the camera
      figure[1,0].camera.set_state({"position": (bboxes[t,0]*movie.shape[2], bboxes[t,1]*movie.shape[1], -1), "zoom": scale})

# TODO: initialize crop properly
# TODO labels as title

iw.add_event_handler(set_bbox)
iw.add_event_handler(set_crop)
iw.show()

I am writing the issue instead of a PR that adds an example to more ask a question if there ins't a better way. Also saw there is a rework for the image widget in progress but from quickly looking through that PR - I couldn't find something like this.