Skip to content

Commit a94859f

Browse files
owenthcareyowenthcarey
committed
test: make e2e open_demo state-aware to skip redundant relaunches
1 parent c7becef commit a94859f

3 files changed

Lines changed: 114 additions & 45 deletions

File tree

‎tests/e2e/AGENTS.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,12 @@ maestro test \
8383

8484
The build step (`pn run <platform> --no-logs`) only needs to run once per change to `app/`. After that, repeat-run the Maestro flow as you iterate.
8585

86+
### How `open_demo.yaml` decides what to do
87+
88+
`helpers/open_demo.yaml` is *state-aware* — it inspects the current screen and runs only the steps it needs to land on `Demo: <DEMO_TITLE>`. Same-category consecutive flows stay on the category screen between demos (no detour through home, no `launchApp`); cross-category transitions go via home; a dead app gets relaunched. The companion `helpers/close_demo.yaml` only pops one level (back to the category list) so the next flow's `open_demo` can pick up cheaply.
89+
90+
This is intentional and the source of the suite's speed. When debugging a flow, **don't** "simplify" `open_demo` to always `launchApp` + go home + go to category — that's the slow path the smart logic was written to avoid (about 15 min vs. 3 min of pure navigation overhead across the full iOS suite). If a flow needs a guaranteed clean app launch, set up that state in the flow itself.
91+
8692
### Suite-level retry
8793

8894
`scripts/run-e2e.sh` re-invokes the whole `maestro test` once if the first attempt exits non-zero. The retry exists to absorb Maestro's iOS XCUITest driver flake (transient `Application is not running` / `Request for viewHierarchy failed`) — not to paper over real failures.

‎tests/e2e/helpers/close_demo.yaml‎

Lines changed: 20 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,25 @@
1-
# Reusable cleanup: pop back to the home screen from a demo.
1+
# Reusable cleanup: pop from a demo screen back to its category list.
22
#
3-
# Run at the end of every demo flow so the suite can chain flows
4-
# without leaking state. Idempotent — calling this when already on
5-
# home is a no-op.
3+
# Each flow ends with this helper so the next flow's ``open_demo.yaml``
4+
# can pick up from a known state. ``open_demo`` is state-aware and will
5+
# handle category-to-category or home-to-category transitions itself,
6+
# so this helper only needs to take a single step out of the demo —
7+
# the suite stays in the category screen between consecutive demos in
8+
# the same category, which is most of the suite by volume and the main
9+
# source of the suite-wide speedup.
610
#
7-
# The category screen renders every demo button inside a ScrollView
8-
# and "Back to home" sits at the bottom. With 20+ demos in a category
9-
# (e.g. ``Components``) the button lands below the visible fold after
10-
# returning from a demo, and ``tapOn`` does not auto-scroll. iOS
11-
# additionally preserves a parent ScrollView's scroll offset across
12-
# navigation, so after a deep demo the category title can also start
13-
# *above* the visible fold — we scroll up to it first, then assert.
11+
# Callers pass:
12+
#
13+
# CATEGORY: "Components"
14+
#
15+
# The ``Demos in ${CATEGORY}`` assertion catches a broken "Back to list"
16+
# button right here, with a clear error, instead of letting the failure
17+
# surface as a confusing scroll-not-found in the next flow's
18+
# ``open_demo``.
19+
#
20+
# iOS preserves a parent ScrollView's offset across navigation, so the
21+
# category title can start above the visible fold after returning from
22+
# a deep demo — we scroll up to it before asserting.
1423
appId: ${APP_ID}
1524
---
1625
- tapOn: "Back to list"
@@ -20,17 +29,3 @@ appId: ${APP_ID}
2029
direction: UP
2130
timeout: 15000
2231
- assertVisible: "Demos in ${CATEGORY}"
23-
- scrollUntilVisible:
24-
element:
25-
text: "Back to home"
26-
direction: DOWN
27-
- tapOn: "Back to home"
28-
# Same scroll-position preservation applies to the home screen on iOS:
29-
# after popping back from a deep demo, the home ScrollView can still be
30-
# scrolled past its title. Scroll up to surface it before asserting.
31-
- scrollUntilVisible:
32-
element:
33-
text: "E2E Suite home"
34-
direction: UP
35-
timeout: 15000
36-
- assertVisible: "E2E Suite home"

‎tests/e2e/helpers/open_demo.yaml‎

Lines changed: 88 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,101 @@
1-
# Reusable navigation helper: launch the app, open a category, open a demo.
1+
# Reusable navigation helper: get the app to ``"Demo: <DEMO_TITLE>"``
2+
# from whatever screen we happen to be on.
23
#
34
# Callers pass two env vars:
45
#
56
# CATEGORY: "Hooks"
67
# DEMO_TITLE: "use_state"
78
#
8-
# This flow leaves the app on the corresponding demo screen with
9-
# ``"Demo: <DEMO_TITLE>"`` visible, ready for the caller's assertions.
9+
# This helper is *state-aware*: it does the minimum work needed to land
10+
# on the target demo. The full suite spends most of its time replaying
11+
# the same launch + nav-home + nav-category dance between every flow,
12+
# so the suite-wide speedup from short-circuiting these steps is
13+
# significant (and it removes a lot of ``launchApp`` calls, which is
14+
# where Maestro's iOS XCUITest driver tends to flake).
15+
#
16+
# State machine, evaluated top-to-bottom; each block runs only if the
17+
# previous one's effects still leave us off-target:
18+
#
19+
# 1. On a demo screen ("Back to list" visible)
20+
# -> tap "Back to list", now on some category screen.
21+
# 2. On the wrong category screen ("Back to home" visible, but title
22+
# != ``Demos in ${CATEGORY}``)
23+
# -> tap "Back to home", now on the home screen.
24+
# 3. App is dead (neither right category nor home visible)
25+
# -> ``launchApp``, now on the home screen.
26+
# 4. On the home screen (but not yet on the right category)
27+
# -> tap ``Open ${CATEGORY}``, now on the right category screen.
28+
# 5. On the right category screen
29+
# -> tap ``Open: ${DEMO_TITLE}``, now on the target demo.
1030
#
1131
# Both the home and category screens use a ScrollView and the target
12-
# button can be below the visible fold (especially in the Components
13-
# category, which lists 20+ demos). Maestro's ``tapOn`` does not
14-
# auto-scroll, so we explicitly ``scrollUntilVisible`` before each tap.
15-
# It's a no-op when the element is already on screen, so the helper is
16-
# safe to use for short lists too.
32+
# button can be below the visible fold (Components has 20+ demos).
33+
# Maestro's ``tapOn`` does not auto-scroll, so we ``scrollUntilVisible``
34+
# before each tap. iOS also preserves a ScrollView's offset across
35+
# navigation, so we scroll UP to surface the title before asserting.
1736
appId: ${APP_ID}
1837
---
19-
- launchApp
20-
- extendedWaitUntil:
21-
visible: "E2E Suite home"
22-
timeout: 60000
23-
- scrollUntilVisible:
24-
element:
25-
text: "Open ${CATEGORY}"
26-
direction: DOWN
27-
- tapOn: "Open ${CATEGORY}"
28-
- extendedWaitUntil:
29-
visible: "Demos in ${CATEGORY}"
30-
timeout: 15000
38+
# 1. If we landed on a demo screen, pop one level to its category list.
39+
- runFlow:
40+
when:
41+
visible: "Back to list"
42+
commands:
43+
- tapOn: "Back to list"
44+
- extendedWaitUntil:
45+
visible: "Back to home"
46+
timeout: 15000
47+
48+
# 2. If we're on the wrong category, go back to home.
49+
- runFlow:
50+
when:
51+
visible: "Back to home"
52+
notVisible: "Demos in ${CATEGORY}"
53+
commands:
54+
- scrollUntilVisible:
55+
element:
56+
text: "Back to home"
57+
direction: DOWN
58+
- tapOn: "Back to home"
59+
- scrollUntilVisible:
60+
element:
61+
text: "E2E Suite home"
62+
direction: UP
63+
timeout: 15000
64+
- assertVisible: "E2E Suite home"
65+
66+
# 3. If neither the right category nor the home screen is visible, the
67+
# app is either not running yet (first flow of the suite) or the driver
68+
# lost its connection. Relaunch as a cold start.
69+
- runFlow:
70+
when:
71+
notVisible: "Demos in ${CATEGORY}"
72+
commands:
73+
- runFlow:
74+
when:
75+
notVisible: "E2E Suite home"
76+
commands:
77+
- launchApp
78+
- extendedWaitUntil:
79+
visible: "E2E Suite home"
80+
timeout: 60000
81+
82+
# 4. From the home screen, navigate into the right category.
83+
- runFlow:
84+
when:
85+
visible: "E2E Suite home"
86+
notVisible: "Demos in ${CATEGORY}"
87+
commands:
88+
- scrollUntilVisible:
89+
element:
90+
text: "Open ${CATEGORY}"
91+
direction: DOWN
92+
- tapOn: "Open ${CATEGORY}"
93+
- extendedWaitUntil:
94+
visible: "Demos in ${CATEGORY}"
95+
timeout: 15000
96+
97+
# 5. We're now guaranteed to be on the right category list. Open the
98+
# requested demo.
3199
- scrollUntilVisible:
32100
element:
33101
text: "Open: ${DEMO_TITLE}"

0 commit comments

Comments
 (0)