Adds CloudXR runtime auto-launch to teleop scripts by rwiltz · Pull Request #5187 · isaac-sim/IsaacLab

rwiltz · 2026-04-06T21:56:16Z

Description

Auto-launch the CloudXR runtime and WSS proxy when running teleop_se3_agent.py or record_demos.py via new --cloudxr_env and --auto_launch_cloudxr script-level CLI arguments. The default profile is cloudxrjs (Quest/Pico); use --cloudxr_env avp for Apple Vision Pro or --cloudxr_env none to disable.
Add cloudxr_env_file and auto_launch_cloudxr parameters to create_isaac_teleop_device(), IsaacTeleopDevice, and TeleopSessionLifecycle so CloudXR configuration flows from scripts into the session lifecycle without per-environment config boilerplate.
Ship two bundled .env profiles (CLOUDXR_JS_ENV, CLOUDXR_AVP_ENV) and update docs with simplified --cloudxr_env avp / --cloudxr_env cloudxrjs shorthands.

Fixes # (issue)

Type of change

New feature (non-breaking change which adds functionality)

Checklist

I have read and understood the contribution guidelines
I have run the pre-commit checks with ./isaaclab.sh --format
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
I have updated the changelog and the corresponding version in the extension's config/extension.toml file
I have added my name to the CONTRIBUTORS.md or my name already exists there

isaaclab-review-bot

Code Review — PR #5187: Auto-launch CloudXR runtime and ship device-specific .env profiles

Summary

This PR eliminates the multi-terminal CloudXR setup by auto-launching the runtime from TeleopSessionLifecycle.start() and shipping two device-specific .env profiles (Quest/Pico and Apple Vision Pro). It also promotes isaacteleop~=1.0.0 from an optional extra to a hard dependency. Solid quality-of-life improvement that simplifies the user workflow from 3 terminals + manual env-sourcing down to a single command.

Design Assessment

Good: Clean separation between CloudXRRuntimeCfg (data) and _ensure_cloudxr_runtime (lifecycle), lazy import of CloudXRLauncher, env-var escape hatch (ISAACLAB_CXR_SKIP_AUTOLAUNCH), and proper shutdown in stop(). The --cloudxr_env CLI flag provides a clean override mechanism.

Architecture Impact

Dependency change: isaacteleop~=1.0.0 becomes a hard install requirement. This means all isaaclab_teleop users will pull it, even if they only use keyboard/spacemouse teleop. This is a deliberate decision noted in the changelog; worth confirming with the team that the install-size/time trade-off is acceptable.
Blast radius: Limited to teleop workflows. Non-teleop tasks are unaffected. The CloudXRLauncher import is lazily deferred behind TYPE_CHECKING and a runtime conditional, so non-CloudXR paths don't pay import cost.

Implementation Verdict

Well-structured. A few issues noted below — one medium-severity robustness concern, one style point, and one doc nit.

Test Coverage

⚠️ No new tests. The checklist claims tests were added, but no test files were modified or created. At minimum, a unit test for _ensure_cloudxr_runtime with mocked CloudXRLauncher would verify the skip/launch logic paths. The ISAACLAB_CXR_SKIP_AUTOLAUNCH env-var check and auto_launch=False branches are trivially testable without any GPU/XR hardware.

CI Status

pre-commit: ✅ pass
license-check: ⏳ pending
Build Base Docker Image: ⏳ pending
Build Latest Docs: ⏳ pending
No failures at time of review.

Findings

See inline comments.

greptile-apps · 2026-04-06T22:01:56Z

Greptile Summary

This PR adds automatic lifecycle management for the CloudXR runtime and WSS proxy during IsaacTeleopDevice sessions. It ships two bundled .env profiles (cloudxrjs-cloudxr.env for Quest/Pico and avp-cloudxr.env for Apple Vision Pro), adds --cloudxr_env / --auto_launch_cloudxr CLI arguments to the two entry-point scripts, and promotes isaacteleop[retargeters,ui,cloudxr]~=1.2.0 from an optional extra to a required dependency.

Confidence Score: 5/5

Safe to merge — all remaining findings are P2 style/cleanup suggestions that do not affect correctness.

The PR is well-scoped, backed by thorough unit tests (env-var skip, idempotency, constructor args, start/stop integration), and the previous P1 concerns from the review thread have been addressed. The two remaining comments are P2: a narrow exception catch in stop() and a copy-paste helper that could drift.

session_lifecycle.py (stop() exception handling) and the two entry-point scripts (duplicated _resolve_cloudxr_env)

Important Files Changed

Filename	Overview
source/isaaclab_teleop/isaaclab_teleop/session_lifecycle.py	Adds _ensure_cloudxr_runtime() and CloudXR start/stop lifecycle; stop() catches only RuntimeError with a misleading "atexit cleanup" comment
scripts/environments/teleoperation/teleop_se3_agent.py	Adds --cloudxr_env / --auto_launch_cloudxr arguments; _resolve_cloudxr_env() is duplicated verbatim from record_demos.py
scripts/tools/record_demos.py	Adds --cloudxr_env / --auto_launch_cloudxr arguments; _resolve_cloudxr_env() is duplicated verbatim from teleop_se3_agent.py
source/isaaclab_teleop/test/test_cloudxr_lifecycle.py	Comprehensive unit tests for CloudXR auto-launch: env-var skip, idempotency, correct constructor args, start/stop integration, and .env path constants
source/isaaclab_teleop/setup.py	Promotes isaacteleop to required dependency with ~=1.2.0 pin; adds dex-retargeting as required on Linux x86_64; adds *.env to package_data
source/isaaclab_teleop/isaaclab_teleop/isaac_teleop_cfg.py	Adds CLOUDXR_AVP_ENV and CLOUDXR_JS_ENV module-level constants pointing to the bundled .env profiles
source/isaaclab_teleop/isaaclab_teleop/isaac_teleop_device.py	Threads cloudxr_env_file / auto_launch_cloudxr through IsaacTeleopDevice and create_isaac_teleop_device; no logic changes
source/isaaclab_teleop/isaaclab_teleop/init.pyi	Stub updated to export CLOUDXR_AVP_ENV and CLOUDXR_JS_ENV alongside existing public API

Sequence Diagram

sequenceDiagram
    participant Script as teleop_se3_agent.py
    participant Factory as create_isaac_teleop_device
    participant Device as IsaacTeleopDevice
    participant Lifecycle as TeleopSessionLifecycle
    participant CXR as CloudXRLauncher

    Script->>Script: _resolve_cloudxr_env("cloudxrjs")
    Script->>Factory: create_isaac_teleop_device(cfg, cloudxr_env_file)
    Factory->>Device: IsaacTeleopDevice(cfg, cloudxr_env_file)
    Device->>Lifecycle: TeleopSessionLifecycle(cfg, cloudxr_env_file)
    Script->>Device: __enter__()
    Device->>Lifecycle: start()
    Lifecycle->>Lifecycle: _ensure_cloudxr_runtime()
    Lifecycle->>CXR: CloudXRLauncher(install_dir, env_config, accept_eula=False)
    CXR-->>Lifecycle: launcher instance
    Lifecycle-->>Lifecycle: _cloudxr_launcher assigned

    loop Each simulation frame
        Script->>Device: advance()
        Device->>Lifecycle: step(anchor_world_matrix_fn)
        Lifecycle-->>Device: action tensor
    end

    Script->>Device: __exit__()
    Device->>Lifecycle: stop()
    Lifecycle->>CXR: stop()
    alt success
        CXR-->>Lifecycle: OK
        Lifecycle-->>Lifecycle: _cloudxr_launcher = None
    else RuntimeError
        Lifecycle-->>Lifecycle: retain launcher reference
    end

_{Reviews (4): Last reviewed commit: "Launch the CXR runtime automatiaclly whe..." | Re-trigger Greptile}

AntoineRichard

Thanks @rwiltz for the helping us bringing back XR workflows! There are a bunch of comments from the bots that I think are relevant could you address / answer / close them?

Thanks!

greptile-apps · 2026-04-09T21:19:47Z

Tip:

Greploop — Automatically fix all review issues by running /greploops in Claude Code. It iterates: fix, push, re-review, repeat until 5/5 confidence.

Use the Greptile plugin for Claude Code to query reviews, search comments, and manage custom context directly from your terminal.

rwiltz · 2026-04-09T21:36:20Z

@greptile

rwiltz · 2026-04-09T21:42:04Z

@isaaclab-review-bot

hougantc-nvda · 2026-04-15T00:08:24Z

I am able to run through the steps
./isaaclab.sh -i
installs isaacteleop automatically.

then running
./isaaclab.sh -p scripts/tools/record_demos.py --task Isaac-PickPlace-GR1T2-Abs-v0 --num_demos 5 --xr --visualizer kit

teleop automatically starts up for me and I can connect Quest3 to it after Start XR. Similar seamless flow to IL 2.3

Changes implemented

hougantc-nvda · 2026-04-15T22:22:09Z

LGTM besides the comment that we chatted about.

rwiltz · 2026-04-15T23:28:55Z

@greptile

hougantc-nvda · 2026-04-16T14:31:58Z

LGTM. Just retested on Quest3 and it was a seamless experience. Nice!

hougantc-nvda

LGTM

rwiltz requested review from Mayankm96, jtigue-bdai, kellyguo11 and ooctipus as code owners April 6, 2026 21:56

github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels Apr 6, 2026

isaaclab-review-bot Bot reviewed Apr 6, 2026

View reviewed changes

greptile-apps Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread source/isaaclab_teleop/docs/CHANGELOG.rst Outdated

Comment thread scripts/environments/teleoperation/teleop_se3_agent.py Outdated

Comment thread source/isaaclab_teleop/isaaclab_teleop/session_lifecycle.py Outdated

AntoineRichard previously requested changes Apr 7, 2026

View reviewed changes

Comment thread source/isaaclab_tasks/docs/CHANGELOG.rst

Comment thread source/isaaclab_teleop/docs/CHANGELOG.rst

Comment thread source/isaaclab_teleop/isaaclab_teleop/isaac_teleop_cfg.py Outdated

rwiltz marked this pull request as draft April 7, 2026 14:40

rwiltz force-pushed the rwiltz/launch-cxr branch from 59da18a to 2d4bcda Compare April 9, 2026 21:02

rwiltz marked this pull request as ready for review April 9, 2026 21:14

rwiltz requested a review from AntoineRichard April 9, 2026 21:36

rwiltz force-pushed the rwiltz/launch-cxr branch from 6c234d9 to 86a69fc Compare April 14, 2026 19:40

rwiltz requested a review from hougantc-nvda April 15, 2026 19:53

hougantc-nvda reviewed Apr 15, 2026

View reviewed changes

Comment thread ...aclab_tasks/manager_based/locomanipulation/pick_place/fixed_base_upper_body_ik_g1_env_cfg.py Outdated

Launch the CXR runtime automatiaclly when xr teleop is active

1f391a8

rwiltz force-pushed the rwiltz/launch-cxr branch from 92d1e71 to 1f391a8 Compare April 15, 2026 23:27

rwiltz changed the title ~~Auto-launch CloudXR runtime and ship device-specific .env profiles~~ Adds CloudXR runtime auto-launch to teleop scripts Apr 15, 2026

Merge branch 'develop' into rwiltz/launch-cxr

b5bd44b

hougantc-nvda approved these changes Apr 16, 2026

View reviewed changes

kellyguo11 approved these changes Apr 16, 2026

View reviewed changes

Merge branch 'develop' into rwiltz/launch-cxr

ce802e8

rwiltz merged commit 93ae322 into isaac-sim:develop Apr 16, 2026
10 checks passed

Conversation

rwiltz commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Checklist

Uh oh!

isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

Code Review — PR #5187: Auto-launch CloudXR runtime and ship device-specific .env profiles

Summary

Design Assessment

Architecture Impact

Implementation Verdict

Test Coverage

CI Status

Findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AntoineRichard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Apr 9, 2026

Uh oh!

rwiltz commented Apr 9, 2026

Uh oh!

rwiltz commented Apr 9, 2026

Uh oh!

hougantc-nvda commented Apr 15, 2026

Uh oh!

Uh oh!

hougantc-nvda commented Apr 15, 2026

Uh oh!

rwiltz commented Apr 15, 2026

Uh oh!

hougantc-nvda commented Apr 16, 2026

Uh oh!

hougantc-nvda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rwiltz commented Apr 6, 2026 •

edited

Loading

greptile-apps Bot commented Apr 6, 2026 •

edited

Loading