Technically the pygfx Orthographic camera is just the perspective camera with a fixed FOV of 0. https://github.com/pygfx/pygfx/blob/main/pygfx/cameras/_orthographic.py#L28

Therefore if we just use the perspective camera for everything it makes it much easier to switch controllers without making things wonky!