Skip to content

Repoint more UI tests#73333

Draft
davidsbailey wants to merge 17 commits into
stagingfrom
repoint-more-ui-tests
Draft

Repoint more UI tests#73333
davidsbailey wants to merge 17 commits into
stagingfrom
repoint-more-ui-tests

Conversation

@davidsbailey

Copy link
Copy Markdown
Member

Testing story

Deployment notes

davidsbailey and others added 17 commits June 17, 2026 22:18
…things

The 'Desktop Minecraft puzzle using dark small footer' @eyes scenario only
loads a Minecraft level and checks the small footer and copyright modal; it
needs a Minecraft-kind level, not prod curriculum. Repoint it onto the
allthethings Minecraft sample lesson (position 25, level 1).

The 'Desktop Star Wars share small footer' scenario is left on prod starwars:
it presses runButton then waits for #finishButton and a sharing dialog, which
requires a completable puzzle. The allthethings Star Wars sample levels (event
freeplay and a blocks level) never reach the finish/share state, so repointing
there breaks the share flow. Flagged for the developer rather than forced.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lthethings

The 'minecraft house dialog' example in the Simple-blockly-level-page-view
@eyes outline only loads a level and snapshots its initial appearance. It
needs a Minecraft-kind level, not prod curriculum. Repoint onto the
allthethings Minecraft sample lesson (position 25, level 3), distinct from the
existing 'minecraft' example which uses level 1. The other examples already
target allthethingscourse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lthethings

The HOC Top Instructions @eyes scenario loads three lab kinds and snapshots
their top-instructions rendering. It needs a level of each kind, not prod
curriculum. Repoint:
  mc      L1/L4  -> allthethings Minecraft  L25/L1
  starwars L1/L15 -> allthethings Star wars L24/L1
  frozen  L1/L5  -> allthethings Artist     L3/L1   (Frozen is a Turtle/Artist lab)

The steps are visual-diff snapshots (no hard-coded instruction text in the
feature), so the task's content-coupling caution does not bite locally: with
disable_all_eyes_running the snapshot steps no-op and the scenario only drives
navigation + lab load, which all three allthethings sample levels satisfy. The
Applitools baselines will need re-accepting in CI since the level content
differs; these are incidental rendering checks, not text assertions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared-content-restriction test typed PII/profanity into a free-play
Studio game and asserted the server-side share filter (#share-fail-explanation).
Repoint off the prod 'playlab' course onto the allthethings fixture.

Target is allthethings lesson 22 (Studio) level 1, 'Resizing Test': a
free_play, non-channel-backed Studio game with sharing enabled. This
matches the original playlab_10's filtering path -- find_share_failure in
activities_controller scans the milestone program for STUDIO games whose
game.sharing_filtered? is true. The PlayLab lesson 5 candidates were
unsuitable: lesson5/level5 (allthethings-playlab-project-backed) is
project/channel-backed, so it routes through loadProjectBackedLevel_ in
project.js instead and never surfaces #share-fail-explanation (verified:
both scenarios timed out on it).

Verified against test-studio with local chromedriver: phone-number and
email scenarios pass (2 passed). The profanity scenario is @WebPurify
(skipped by default) and was left unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'bounce game' completion scenario ran a Bounce puzzle to the .congrats
win state. Repoint off the prod 'events' course onto the allthethings
fixture: events lesson 1 level 1 and allthethingscourse lesson 8 (Bounce)
level 1 are the same underlying level (bounce_1), so the repoint is a clean
swap -- run -> hold LEFT -> .congrats still completes.

Verified against test-studio with local chromedriver via --eyes (eyes
snapshots no-op'd by disable_all_eyes_running): full feature passes
(1 passed); the Bounce scenario alone also passes in isolation.

The 'freeplay playlab sharing' scenario (still on prod 'playlab') is NOT
repointed -- it is a finish/share flow (runButton -> finishButton ->
.congrats) and no acceptable allthethings level supports it: the only
free_play Studio level (lesson 22 'Resizing Test') pops an instructions
modal-backdrop that intercepts the finishButton click, and the PlayLab
lesson 5 levels are free_play=false so finishButton is not interactable.
Flagged for the developer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Step-mode maze levels now live in the allthethings fixture at lesson 2,
levels 8 (step-only) and 9 (step+run). Repoint:

  courses/step/units/1/lessons/1/levels/1 -> allthethingscourse/units/1/lessons/2/levels/8
  courses/step/units/1/lessons/1/levels/2 -> allthethingscourse/units/1/lessons/2/levels/9

Verified the 8/9 mapping by running: level 8's #runButton is hidden
(step-only) and level 9's is visible (step+run), matching the original
levels/1 and levels/2 scenarios. The completion dialog's hardcoded puzzle
number tracks the level's lesson position, so "Puzzle 1" becomes "Puzzle 8".

All 5 scenarios pass locally against test-studio.code.org.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cookie-banner @eyes test only needs any level page to load; the
destination is incidental. Repoint off the frozen production course onto
the allthethings Artist level (lesson 3, level 1):

  courses/frozen/units/1/lessons/1/levels/1 -> allthethingscourse/units/1/lessons/3/levels/1

Verified with --eyes against test-studio.code.org (eyes diffs no-op'd via
disable_all_eyes_running): banner shows, dismisses, and stays dismissed
after reload. Scenario passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Desktop Star Wars share small footer @eyes scenario navigated prod
starwars unit 1 lesson 1 level 15 (js_hoc2015_event_free, the freeplay
share level). Repoint onto the byte-identical level in allthethings:
courses/allthethingscourse/units/1/lessons/24/levels/9, same levelId
js_hoc2015_event_free. The scenario asserts no puzzle number, so the
position change (15 -> 9) needs no assertion edit. Verified with
--eyes against test-studio (1 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Hour of Code 2015 starwars tutorial drove prod course starwars
unit 1 lesson 1. Repoint onto the byte-identical hoc2015 levels in
allthethings lesson 24 (same js_hoc2015_* levelIds, verified live):
  starwars L1/levels 1..6 -> allthethingscourse L24/levels 3..8
    (move_right, move_right_down, move_backtrack, move_diagonal,
     move_around, move_finale -- identical levelIds, prod puzzle 1-6
     now at lesson positions 3-8 of a 9-level lesson)
  courses/starwars/units/1/reset -> courses/allthethingscourse/units/1/reset

Position-dependent assertion fix: the two "Solving puzzle 1" scenarios
assert header progress "for level 1" via header_bubble_selector, which
indexes the (N-1)th header bubble. The completed level now sits at
lesson position 3, so the assertion becomes "for level 3" to check the
correct bubble. No "Puzzle N of M" text is asserted anywhere, so the
lesson_total change (15 -> 9) needs no other edit.

Verified all 10 scenarios individually against test-studio (each 1
passed); @no_mobile runs in local Chrome.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The flappy completion tests drove the /flappy/N tutorial root (course
flappy). Repoint onto the byte-identical flappy levels in allthethings
lesson 7 (verified live by level key):
  /flappy/1 -> courses/allthethingscourse/units/1/lessons/7/levels/1 (flappy_1)
  /flappy/2 -> courses/allthethingscourse/units/1/lessons/7/levels/3 (flappy_2)

Position-dependent assertion fix: lesson 7 has only 3 levels, so flappy_2
at position 3 is the last level in the lesson (isLastLevelInLesson:true).
The lab then renders the finalStage congrats string instead of the
numbered one, so "Solving puzzle 2" now asserts "Congratulations! You
have completed the final puzzle." (was "You completed Puzzle 2"). flappy_1
at position 1 (puzzle_number 1, not last) still shows "You completed
Puzzle 1" -- unchanged. No /flappy reset refs exist in this file.

Verified all 3 scenarios against test-studio (3 scenarios, 3 passed);
@no_mobile runs in local Chrome.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bounce completion tests drove prod course events unit 1 lesson 1.
Repoint onto the byte-identical bounce levels in allthethings lesson 8
(verified live by level key):
  events L1/level1  (bounce_1)  -> allthethingscourse L8/level1
  events L1/level3  (bounce_3)  -> allthethingscourse L8/level3
  events L1/level5  (bounce_5)  -> allthethingscourse L8/level4
  events L1/level10 (bounce_12, -> allthethingscourse L8/level2
                     freeplay)

Position-dependent assertion fix: allthethings lesson 8 has only 4
levels, so bounce_5 at position 4 is the last level in the lesson
(isLastLevelInLesson:true). The lab then renders the finalStage congrats
string, so "Complete Level 5" now asserts "Congratulations! You have
completed the final puzzle." (was "...You completed Puzzle 5."). The
"Complete Level 1"/"Complete Level 3" puzzle numbers (1, 3) already
match the new positions (not last-in-lesson, so numbered). The freeplay
scenario asserts only that .congrats is visible, no puzzle number.

Verified all 5 scenarios against test-studio (5 scenarios, 5 passed);
@single_session, @no_mobile runs in local Chrome.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…allthethings

The @eyes "freeplay playlab sharing" scenario navigated to
courses/playlab/units/1/lessons/1/levels/10 (a production course).
Repoint onto the allthethings PlayLab lesson at
courses/allthethingscourse/units/1/lessons/5/levels/7, the same prod
playlab_10 level key wired onto test-studio. Position 7 is the last
level of L5 (isLastLevelInLesson:true), so a completion congrats may
render the finalStage string, but the scenario only asserts .congrats
visibility, not its text, so no assertion edit is needed.

Verified locally against test-studio with --eyes (1 passed), baseline
and changed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five scenarios navigated to courses/dance/units/1/lessons/1/levels/{1,8,
12,13} (a production course). Repoint onto the allthethings Dance Lab
lesson at courses/allthethingscourse/units/1/lessons/37/levels/{5,6,7,8}
respectively. The Dance_Party_* level keys are byte-identical to prod at
the new positions (Dance_Party_01, _09, _11_5, _12), so tint-block,
winning-sequence, sprite-count, song-selector, and share behavior are
unchanged. No assertion is position-dependent (all key off level keys,
not puzzle numbers); the "winning dance level 8 blocks" step text labels
the lab's internal level, not the URL position, so it stays.

Verified locally against test-studio: 6 scenarios passed, baseline and
changed. The @cloudfront_key restricted-audio scenario (levels/1 ->
levels/5) auto-skips locally; its target page loads and carries the same
Dance_Party_01 key, but it is exercised only in CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The four Oceans ML HoC scenarios navigated to
courses/oceans/units/1/lessons/1/levels/{2,3,4,6,8} (a production
course). Repoint onto the allthethings ML Fish lesson at
courses/allthethingscourse/units/1/lessons/42/levels/{2,3,4,5,6}
respectively. The Oceans_* level keys are byte-identical to prod at the
new positions (Oceans_FishVTrash, _CreaturesVTrashDemo, _CreaturesVTrash,
_Short, _Long), so the training/sorting/pond flow and button text the
scenarios assert are unchanged. The Long Word List scenario asserts the
pond-screen "Finish" button: L42/6 (Oceans_Long) is the last level in
lesson 42, matching prod, so "Finish" (not "Continue") still renders.

This feature is @no_ci (also @no_mobile/@no_safari) so it won't run in
Drone, but it runs in local Chrome: verified locally against test-studio,
4 passed baseline and 4 passed changed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These four were seeded only for UI tests that have now been repointed onto
allthethings (events→Bounce levels, step→step-mode maze levels, playlab→
PlayLab/Studio levels) or that use only standalone /projects/<type> routes
(artist, playlab), which load global custom levels via STANDALONE_PROJECTS
(Level.find_by_key), independent of the script.

No remaining UI test navigates to courses/{artist,events,playlab,step} or
/s/<slug>, and the levels these scripts contribute are seeded globally by
custom_levels / deprecated_blockly_levels (SCRIPTS_DEPENDENCIES, which run
before scripts_ui_tests), so allthethings and ui-test-csf keep working.

Removed from both UI_TEST_SCRIPTS and courses_ui_tests. [reset db] so drone
wipes the DB before `rake seed:ui_test`, validating seeding without them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
oceans's only remaining UI-test dependents are the two oceans-certificate
@eyes tests (hoc_certificates "oceans course certificates",
hoc_batch_certificates) — its catalog tests merged off it via
repoint-oceans-ui-tests, and ml_hoc was repointed onto allthethings.

Those cert tests don't need oceans seeded: the cert template is chosen by
course-name string (CertificateImage.certificate_template_for: `course ==
OCEANS_NAME` -> oceans_hoc_certificate.png), it's a prefilled-title template
(only the student name is overlaid, no seeded course title), and the
/api/hour/finish/oceans tutorial registry is Contentful-backed
(HocLegacy::Tutorials.get), independent of UI_TEST_SCRIPTS/courses_ui_tests.

Removed from both lists. [reset db] so drone wipes the DB before
`rake seed:ui_test`, validating seeding (and the oceans cert eyes tests)
without oceans seeded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant