Repoint more UI tests#73333
Draft
davidsbailey wants to merge 17 commits into
Draft
Conversation
…things The 'Desktop Minecraft puzzle using dark small footer' @eyes scenario only loads a Minecraft level and checks the small footer and copyright modal; it needs a Minecraft-kind level, not prod curriculum. Repoint it onto the allthethings Minecraft sample lesson (position 25, level 1). The 'Desktop Star Wars share small footer' scenario is left on prod starwars: it presses runButton then waits for #finishButton and a sharing dialog, which requires a completable puzzle. The allthethings Star Wars sample levels (event freeplay and a blocks level) never reach the finish/share state, so repointing there breaks the share flow. Flagged for the developer rather than forced. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lthethings The 'minecraft house dialog' example in the Simple-blockly-level-page-view @eyes outline only loads a level and snapshots its initial appearance. It needs a Minecraft-kind level, not prod curriculum. Repoint onto the allthethings Minecraft sample lesson (position 25, level 3), distinct from the existing 'minecraft' example which uses level 1. The other examples already target allthethingscourse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lthethings The HOC Top Instructions @eyes scenario loads three lab kinds and snapshots their top-instructions rendering. It needs a level of each kind, not prod curriculum. Repoint: mc L1/L4 -> allthethings Minecraft L25/L1 starwars L1/L15 -> allthethings Star wars L24/L1 frozen L1/L5 -> allthethings Artist L3/L1 (Frozen is a Turtle/Artist lab) The steps are visual-diff snapshots (no hard-coded instruction text in the feature), so the task's content-coupling caution does not bite locally: with disable_all_eyes_running the snapshot steps no-op and the scenario only drives navigation + lab load, which all three allthethings sample levels satisfy. The Applitools baselines will need re-accepting in CI since the level content differs; these are incidental rendering checks, not text assertions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared-content-restriction test typed PII/profanity into a free-play Studio game and asserted the server-side share filter (#share-fail-explanation). Repoint off the prod 'playlab' course onto the allthethings fixture. Target is allthethings lesson 22 (Studio) level 1, 'Resizing Test': a free_play, non-channel-backed Studio game with sharing enabled. This matches the original playlab_10's filtering path -- find_share_failure in activities_controller scans the milestone program for STUDIO games whose game.sharing_filtered? is true. The PlayLab lesson 5 candidates were unsuitable: lesson5/level5 (allthethings-playlab-project-backed) is project/channel-backed, so it routes through loadProjectBackedLevel_ in project.js instead and never surfaces #share-fail-explanation (verified: both scenarios timed out on it). Verified against test-studio with local chromedriver: phone-number and email scenarios pass (2 passed). The profanity scenario is @WebPurify (skipped by default) and was left unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'bounce game' completion scenario ran a Bounce puzzle to the .congrats win state. Repoint off the prod 'events' course onto the allthethings fixture: events lesson 1 level 1 and allthethingscourse lesson 8 (Bounce) level 1 are the same underlying level (bounce_1), so the repoint is a clean swap -- run -> hold LEFT -> .congrats still completes. Verified against test-studio with local chromedriver via --eyes (eyes snapshots no-op'd by disable_all_eyes_running): full feature passes (1 passed); the Bounce scenario alone also passes in isolation. The 'freeplay playlab sharing' scenario (still on prod 'playlab') is NOT repointed -- it is a finish/share flow (runButton -> finishButton -> .congrats) and no acceptable allthethings level supports it: the only free_play Studio level (lesson 22 'Resizing Test') pops an instructions modal-backdrop that intercepts the finishButton click, and the PlayLab lesson 5 levels are free_play=false so finishButton is not interactable. Flagged for the developer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Step-mode maze levels now live in the allthethings fixture at lesson 2, levels 8 (step-only) and 9 (step+run). Repoint: courses/step/units/1/lessons/1/levels/1 -> allthethingscourse/units/1/lessons/2/levels/8 courses/step/units/1/lessons/1/levels/2 -> allthethingscourse/units/1/lessons/2/levels/9 Verified the 8/9 mapping by running: level 8's #runButton is hidden (step-only) and level 9's is visible (step+run), matching the original levels/1 and levels/2 scenarios. The completion dialog's hardcoded puzzle number tracks the level's lesson position, so "Puzzle 1" becomes "Puzzle 8". All 5 scenarios pass locally against test-studio.code.org. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cookie-banner @eyes test only needs any level page to load; the destination is incidental. Repoint off the frozen production course onto the allthethings Artist level (lesson 3, level 1): courses/frozen/units/1/lessons/1/levels/1 -> allthethingscourse/units/1/lessons/3/levels/1 Verified with --eyes against test-studio.code.org (eyes diffs no-op'd via disable_all_eyes_running): banner shows, dismisses, and stays dismissed after reload. Scenario passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Desktop Star Wars share small footer @eyes scenario navigated prod starwars unit 1 lesson 1 level 15 (js_hoc2015_event_free, the freeplay share level). Repoint onto the byte-identical level in allthethings: courses/allthethingscourse/units/1/lessons/24/levels/9, same levelId js_hoc2015_event_free. The scenario asserts no puzzle number, so the position change (15 -> 9) needs no assertion edit. Verified with --eyes against test-studio (1 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Hour of Code 2015 starwars tutorial drove prod course starwars
unit 1 lesson 1. Repoint onto the byte-identical hoc2015 levels in
allthethings lesson 24 (same js_hoc2015_* levelIds, verified live):
starwars L1/levels 1..6 -> allthethingscourse L24/levels 3..8
(move_right, move_right_down, move_backtrack, move_diagonal,
move_around, move_finale -- identical levelIds, prod puzzle 1-6
now at lesson positions 3-8 of a 9-level lesson)
courses/starwars/units/1/reset -> courses/allthethingscourse/units/1/reset
Position-dependent assertion fix: the two "Solving puzzle 1" scenarios
assert header progress "for level 1" via header_bubble_selector, which
indexes the (N-1)th header bubble. The completed level now sits at
lesson position 3, so the assertion becomes "for level 3" to check the
correct bubble. No "Puzzle N of M" text is asserted anywhere, so the
lesson_total change (15 -> 9) needs no other edit.
Verified all 10 scenarios individually against test-studio (each 1
passed); @no_mobile runs in local Chrome.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The flappy completion tests drove the /flappy/N tutorial root (course flappy). Repoint onto the byte-identical flappy levels in allthethings lesson 7 (verified live by level key): /flappy/1 -> courses/allthethingscourse/units/1/lessons/7/levels/1 (flappy_1) /flappy/2 -> courses/allthethingscourse/units/1/lessons/7/levels/3 (flappy_2) Position-dependent assertion fix: lesson 7 has only 3 levels, so flappy_2 at position 3 is the last level in the lesson (isLastLevelInLesson:true). The lab then renders the finalStage congrats string instead of the numbered one, so "Solving puzzle 2" now asserts "Congratulations! You have completed the final puzzle." (was "You completed Puzzle 2"). flappy_1 at position 1 (puzzle_number 1, not last) still shows "You completed Puzzle 1" -- unchanged. No /flappy reset refs exist in this file. Verified all 3 scenarios against test-studio (3 scenarios, 3 passed); @no_mobile runs in local Chrome. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bounce completion tests drove prod course events unit 1 lesson 1.
Repoint onto the byte-identical bounce levels in allthethings lesson 8
(verified live by level key):
events L1/level1 (bounce_1) -> allthethingscourse L8/level1
events L1/level3 (bounce_3) -> allthethingscourse L8/level3
events L1/level5 (bounce_5) -> allthethingscourse L8/level4
events L1/level10 (bounce_12, -> allthethingscourse L8/level2
freeplay)
Position-dependent assertion fix: allthethings lesson 8 has only 4
levels, so bounce_5 at position 4 is the last level in the lesson
(isLastLevelInLesson:true). The lab then renders the finalStage congrats
string, so "Complete Level 5" now asserts "Congratulations! You have
completed the final puzzle." (was "...You completed Puzzle 5."). The
"Complete Level 1"/"Complete Level 3" puzzle numbers (1, 3) already
match the new positions (not last-in-lesson, so numbered). The freeplay
scenario asserts only that .congrats is visible, no puzzle number.
Verified all 5 scenarios against test-studio (5 scenarios, 5 passed);
@single_session, @no_mobile runs in local Chrome.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…allthethings The @eyes "freeplay playlab sharing" scenario navigated to courses/playlab/units/1/lessons/1/levels/10 (a production course). Repoint onto the allthethings PlayLab lesson at courses/allthethingscourse/units/1/lessons/5/levels/7, the same prod playlab_10 level key wired onto test-studio. Position 7 is the last level of L5 (isLastLevelInLesson:true), so a completion congrats may render the finalStage string, but the scenario only asserts .congrats visibility, not its text, so no assertion edit is needed. Verified locally against test-studio with --eyes (1 passed), baseline and changed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five scenarios navigated to courses/dance/units/1/lessons/1/levels/{1,8,
12,13} (a production course). Repoint onto the allthethings Dance Lab
lesson at courses/allthethingscourse/units/1/lessons/37/levels/{5,6,7,8}
respectively. The Dance_Party_* level keys are byte-identical to prod at
the new positions (Dance_Party_01, _09, _11_5, _12), so tint-block,
winning-sequence, sprite-count, song-selector, and share behavior are
unchanged. No assertion is position-dependent (all key off level keys,
not puzzle numbers); the "winning dance level 8 blocks" step text labels
the lab's internal level, not the URL position, so it stays.
Verified locally against test-studio: 6 scenarios passed, baseline and
changed. The @cloudfront_key restricted-audio scenario (levels/1 ->
levels/5) auto-skips locally; its target page loads and carries the same
Dance_Party_01 key, but it is exercised only in CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The four Oceans ML HoC scenarios navigated to
courses/oceans/units/1/lessons/1/levels/{2,3,4,6,8} (a production
course). Repoint onto the allthethings ML Fish lesson at
courses/allthethingscourse/units/1/lessons/42/levels/{2,3,4,5,6}
respectively. The Oceans_* level keys are byte-identical to prod at the
new positions (Oceans_FishVTrash, _CreaturesVTrashDemo, _CreaturesVTrash,
_Short, _Long), so the training/sorting/pond flow and button text the
scenarios assert are unchanged. The Long Word List scenario asserts the
pond-screen "Finish" button: L42/6 (Oceans_Long) is the last level in
lesson 42, matching prod, so "Finish" (not "Continue") still renders.
This feature is @no_ci (also @no_mobile/@no_safari) so it won't run in
Drone, but it runs in local Chrome: verified locally against test-studio,
4 passed baseline and 4 passed changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These four were seeded only for UI tests that have now been repointed onto
allthethings (events→Bounce levels, step→step-mode maze levels, playlab→
PlayLab/Studio levels) or that use only standalone /projects/<type> routes
(artist, playlab), which load global custom levels via STANDALONE_PROJECTS
(Level.find_by_key), independent of the script.
No remaining UI test navigates to courses/{artist,events,playlab,step} or
/s/<slug>, and the levels these scripts contribute are seeded globally by
custom_levels / deprecated_blockly_levels (SCRIPTS_DEPENDENCIES, which run
before scripts_ui_tests), so allthethings and ui-test-csf keep working.
Removed from both UI_TEST_SCRIPTS and courses_ui_tests. [reset db] so drone
wipes the DB before `rake seed:ui_test`, validating seeding without them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
oceans's only remaining UI-test dependents are the two oceans-certificate @eyes tests (hoc_certificates "oceans course certificates", hoc_batch_certificates) — its catalog tests merged off it via repoint-oceans-ui-tests, and ml_hoc was repointed onto allthethings. Those cert tests don't need oceans seeded: the cert template is chosen by course-name string (CertificateImage.certificate_template_for: `course == OCEANS_NAME` -> oceans_hoc_certificate.png), it's a prefilled-title template (only the student name is overlaid, no seeded course title), and the /api/hour/finish/oceans tutorial registry is Contentful-backed (HocLegacy::Tutorials.get), independent of UI_TEST_SCRIPTS/courses_ui_tests. Removed from both lists. [reset db] so drone wipes the DB before `rake seed:ui_test`, validating seeding (and the oceans cert eyes tests) without oceans seeded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Testing story
Deployment notes