Sprite Lab in Lab2: updates#73973
Conversation
Preview (the Code tab) ran the same scene-jump handlers as Play, so a previewed scene containing a goToScene block navigated away from the scene being edited. Gate the handlers to Play mode; while previewing, cancel the jump so the tick timer resumes on the current scene instead of freezing, and skip the cover flash. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s scene Play now (re)starts from a tracked start scene. The Play tab button starts from the beginning (default scene); clicking the live preview opens Play on the scene being previewed. A Start-over button (Play tab only, beside the Play button) re-runs the current start scene from the top. The preview click is caught by a transparent overlay so the game canvas stays non-interactive in preview and the location picker is untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The live preview shares the window keyboard with Blockly (p5 listens on window), so a game reading arrows/space ate those keys while the student was editing on the Code tab. While not in Play, swallow the game control keys at document — after Blockly's own handlers, before p5's window listener — so block editing, text fields, and workspace nav keep working and only the game is cut off. Verified in-browser: Code tab lets 0 arrow presses through to the game, Play tab receives them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The AI codegen emits gamelab_whileKey / gamelab_whenKey for 'while/when [key] pressed', but the /s/allthethings lab2-showcase level's toolbox only offered the combined gamelab_keyPressed, so students couldn't find the same blocks the AI produced. Add the dedicated blocks to the Events category, with while-left and when-space defaults for platformer controls. Verified in-browser: both register and render in the Events flyout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first cut of the scenes variant dropped the World tab. The restructure makes World and Code per-scene sub-tabs, so re-enable World in the tab bar. Still the placeholder grid editor; the grouped scene-nav and per-scene world wiring follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restructure the tab bar into the agreed layout: Images and Play are top-level tabs; the scene picker and the World/Code sub-tabs sit together in one bordered scene segment between them. Picking a scene now governs both sub-tabs (it was coupled to the Code tab only). TabShell renders explicit segments rather than a generic tab map; the old single-tab tabGroup styles are replaced by sceneGroup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
World placement now lives on each scene (SpriteLab2Scene.world) rather than a single global grid, so the scene picker switches the world along with the code. Retires the unused global worlds/activeWorldId/SpriteLab2World. Write and read mirror the per-scene source path (getScenes().map by active id / activeScene.world). Static checks pass and the World tab renders; a live paint-isolation test wasn't possible on the showcase level (empty costume palette, nothing to paint with). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the window.prompt scene-naming placeholder with a DS Dialog plus a text field: Create is disabled until a name is entered; Cancel and the close button dismiss. Verified in-browser: the '+ New scene' option opens the dialog, and entering a name and pressing Create adds and selects the scene. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… scene Add a world-type dropdown to the World tab, persisted per scene (SpriteLab2Scene.worldType). 'none' (the default) hides the grid and shows a hint; 'platform' and 'topdown' show the placement grid. The type will drive runtime placement + gravity in Phase 3. Verified in-browser: none hides the grid, platform/topdown show it, and the choice round-trips. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… blocks Plan change: no world editor. The central area is the scene picker + Code again; platform setup will come from composite blocks (make platform player, make platform blocks using grid) instead of a painted grid. Removes WorldTab, the per-scene world/worldType data, gridConstants, and the World tab styles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Third generate type alongside sprite and background: a square tile drawn full-bleed (no green-screen cut) so copies laid on the grid butt seamlessly. Tagged with a 'blocks' category, which the platform-blocks grid block will draw from. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Back to the pre-restructure middle tab: the Code button leads, the scene dropdown follows, and the pair reads as one segmented control with the original styling (group carries the active background; whole group is the Code tab's click target). The picker-first sceneGroup variant is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two blocks that collapse the ~15-block platformer setup: - 'make platform player [costume] at grid location [grid]': a player sprite at the marked 8x8 cell (same bitmap widget as make-sprites-using-grid), created directly in the players group — zGameDev then applies gravity, wall collisions, and the cell-sized default — plus while-left/right movement at the sprite's own speed and a space jump that requires standing on a wall. Existing blocks still override the defaults. - 'make [costume] platform blocks using grid [grid]': the pieces plus environment (walls) typing in one call. Group-keyed on purpose: an earlier costume-keyed draft orphaned the player from its physics whenever the animation label failed to round-trip. Verified numerically on the showcase level via the engine's sprite map: spawn at the marked cell (175,225), gravity-fall and land exactly on the floor (y=325, resting vy 0.75), walk right 175->260 while held, space jump to vy -7.75 and back down. Both blocks lead the Sprites toolbox category. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tegory Auto-append the two composite blocks to the Sprites toolbox category (ensurePlatformerBlocks, mirroring ensureSceneBlocks): they're lab-owned blocks, so DB-authored toolboxes don't know them. Skipped when a level curates its own entries. Toolbox defaults: the player marked mid-air over a full bottom-row floor. Also corrects the entries' category hint to Sprites. Verified on the dev level (94409): both lead the Sprites flyout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hour2026 leads the toolbox in every level: the two platformer composites plus set background, when-sprite-clicked, when-touching, at-time, sprite say, and the two go-to-scene blocks. A block can appear in any number of toolbox categories; entries curated elsewhere in the level's toolbox are cloned so their shadows/defaults carry over, and unregistered types still get filtered per level. The platform player's grid is now a bitmapSingle input — a position picker, not a painter: at most one set cell, a new mark replaces the old (validator keeps the newest cell). Verified on the dev level: category order and lineup as specified; marking a second cell drops the first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A platform patrol for the walls group: walks left/right, turning at a platform edge (footing probed half a grid cell ahead via isDirectlyAbove, so the sprite turns when its center reaches the edge), at the playspace bounds, or when a wall collision pushes the step back (this tick's x differs from where last tick left it). The playspace floor counts as footing, so a floor patroller just walks the bounds. Known tradeoff: the half-cell probe sees across a one-cell gap, so patrollers fall into them. Verified numerically on the dev level: a patroller on a 3-cell platform (x 100-250) walked 107<->247 across multiple passes with y constant — turning at both edges, never falling off. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The previous commit's 'falls into one-cell gaps' claim was wrong twice over: a cell-sized sprite spans a one-cell gap, so the collide holds it up from one side or the other and it bridges — EXCEPT at the gap's exact center, where it touches both sides with zero overlap, the collide stops holding (isDirectlyAbove, edge-inclusive, still reports supported), and it drops through the seam. Whether a walk lands on that knife-edge depends on step parity, so patrollers sometimes crossed fine and sometimes fell through. Detect the drop as grounded-last-tick/airborne-now and recover: step past the seam, lift back to the walking line, cancel the fall. The blocked-flip check is evaluated before the recovery nudges x, so recovery can't read as a wall bump. Two-cell gaps still read as edges via the probe. Verified numerically: platform / one-cell gap / platform / open end — the patroller swept 35<->295 with y constant across repeated seam crossings in both directions, turning at the open end and the playspace bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A cell-wide patroller (a square AI-generated costume) used to bridge a one-cell gap: its full-width collider always overlapped a platform on one side, so isDirectlyAbove could never see the hole — confusing next to narrower characters, which turn. New native command hasSupportAt(sprite, offsetX, target): a point probe at foot level — a point can see a gap of any width, and a point on the seam between adjacent blocks is inside both, so contiguous ground never reads as a gap. The patrol behavior probes half a grid cell ahead of center, giving every patroller the same rule: turn when the ground half a block ahead disappears. The knife-edge recovery stays as a safety net for sprites shoved onto a seam. Verified numerically: the cell-wide patroller that previously swept across the gap (35<->295) now patrols only its own platform (31<->123), turning before the gap, crossing internal block seams without false turns; the narrow-costume case behaves as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ask for a block that extends to the very edges on all four sides — no margins or frame — while allowing a few empty pixels for a slightly irregular outline, so side-by-side copies read as continuous ground. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Blocks now go through the green-screen pipeline and come out actually transparent at their irregular edges, then get cropped to the opaque bounds so the delivered image fills its own frame — grid-placed copies butt cleanly however much margin the model left. The prompt relaxes to 'mostly-square block on green' accordingly. Corner-seeded keying is wrong for blocks: the subject may reach the corners (the fill would key the block itself), and mid-edge background slivers aren't corner-connected. New MatteOptions.edgeSeededKey keys against explicit green, flooding from every border pixel — subject border pixels are dropped by the same threshold check that keeps the fill from crossing the subject. Sprites keep the corner-seeded behavior unchanged. cropToContent reuses imageTrim's findOpaqueBounds. Unit-tested: mid-edge sliver keyed while a corner-touching subject is kept; enclosed key color preserved; green-corner case unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second injected category, after Platform: scene-driven storytelling — set background, make new sprite at, set size, say / say for N seconds, when clicked, the two go-to-scene blocks, at N seconds, begins-behavior, and the moving-left and patrolling behaviors. The injection step is now data-driven (INJECTED_CATEGORIES) so further categories are one entry. Verified on the dev level: category order Platform, Story, then the level's own; Story lineup as specified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… crop Plan change: rather than an edge-seeded key for blocks that may touch the corners, the block prompt asks for a clear green margin on all four sides, so the sprite pipeline's corner-seeded key applies unchanged; the crop to opaque bounds then trims the padding so grid-placed copies butt cleanly. Removes the just-added edgeSeededKey option and its tests. Tradeoff: a generation that ignores the padding request keys badly and needs a regenerate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ound / block) Three-way split, mirroring how the background picker already worked: backgrounds and blocks each list only their own image category; costume pickers list everything else — so platform tiles don't show up as characters and vice versa. New blockPicker customInput drives the platform-blocks block's costume field; the showcase toolbox entry drops its hardcoded sprite default accordingly. Verified live with one image per category: make-new-sprite and platform-player pickers list only the sprite; platform-blocks only the block; set-background only the background. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cabulary The model now opens its pseudocode with 'profile: platform' or 'profile: story' — chosen from the user's request — and may only use that profile's commands, mirroring the Platform and Story toolbox categories. The parser enforces it: out-of-profile commands are skipped (with a console warning), so the emitted workspace only ever holds the chosen category's blocks. Without a profile line the legacy vocabulary is unchanged. New pseudocode commands to cover the categories: platform_player and platform_blocks (the composites, same 8x8 grid rows as make_grid; platform_blocks validates names against the project's block images, which the prompt now lists), when_click and at_time event hats, and say_for (spriteSayTime). Story behaviors are limited to the category's two (moving left, patrolling left and right). The prompt teaches one worked example per profile; the platformer example is now two setup lines instead of the old ~15-block recipe. 35 unit tests pass (6 new: hats, say_for, composites, name validation, and per-profile filtering). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Play is framed as one experience with a variable starting point: the Play tab always starts from the beginning; clicking a scene preview starts from that scene. While playing from a scene, the tab bar shows 'Playing from: <scene>' plus a Play-from-beginning button; from the beginning, just Start over — the label's absence marks the default. Previewing the first scene counts as the beginning. Start over keeps re-running the current start point. Verified in-browser: preview-click on a second scene shows the label and both buttons; Play-from-beginning clears them and stays in Play. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unselected tabs shared the bar's background with only a faint outline — the design system's *disabled* recipe (its segmented control gives unselected segments a solid neutral fill and marks selection with color). Adopt that: every tab is a solid neutral chip with a tertiary hover; the active tab or group carries a 2px teal inset edge plus the existing bold. The faint look is now reserved for a genuinely disabled state. Verified in both themes; the teal edge reads especially well in light mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ig story sprites Three playtest follow-ups: - New 'make new sprite at grid location' block in the Platform category: a plain single-sprite creator on the same 8x8 grid as the composites, using the single-cell picker like the platform player — placement only, no group or controls. - The Play controls are now two always-present buttons, Restart game (from the first scene) and Restart scene (whatever scene is on stage right now, tracked through scene jumps, external ones included). Replaces the 'Playing from' label + conditional Play-from-beginning button. - The AI's story profile now instructs a set_size of 150-250 after every make_sprite: zGameDev sets the global default sprite size to one 50px cell, which suits platformers but left story characters tiny. Sizing stays visible as blocks rather than hidden defaults. Verified in-browser: the new block sits third in the Platform category and its grid keeps only the newest mark; the Play tab shows exactly the two Restart buttons. 48 lab2 unit tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Platform leads with set background, platform blocks, platform player, then the grid sprite; Story moves at-time just below when-clicked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review of the branch against apps/src/AGENTS.md: - share one bitmap input factory instead of two near-identical entries - @cdo alias for the deep-relative imageTrim import - rename ensurePlatformerBlocks to ensureInjectedCategories (it builds Platform and Story) - name the jump strength; fix comments describing removed UI ("Playing from" label, World tab) and blocks' old full-bleed handling Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A category the level authors itself is now used exactly as written, and an empty authored one suppresses the category entirely; the default lineup only appears when the level doesn't mention the category at all. Previously the default lineup was appended to authored categories, so a levelbuilder couldn't trim them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| ].join('\n'), | ||
| }, | ||
| { | ||
| name: 'spritelab2_patrollingOnBlocks', |
There was a problem hiding this comment.
Can we write these blocks in modern JSON format, like these?
There was a problem hiding this comment.
From Claude; curious what you think:
These are intentionally in the DB pool-block shape (config + interpreted helperCode) rather than the JSON format — the plan of record is to promote them into a new spritelab2 block pool once the set stabilizes (see the TODO at the top of the file), and config+helperCode is the format the blocks table holds. They also lean on installCustomBlocks for the behavior styling and the costume/bitmap custom inputs, which the JSON path doesn't provide. The blockDefinitions blocks are JSON because they're lab-permanent and have no interpreted helper half. If we end up deciding against the DB pool move, converting these would make sense as part of that.
There was a problem hiding this comment.
Hmm I'd still be curious what the blockly JSON version of these blocks would look like, at least to compare. The DB pool-block shape is relevant if we expect levelbuilders to manage and edit these, but I'm not sure if that's the case with these blocks.
There was a problem hiding this comment.
From Claude:
Went ahead and did it — you were right, and the comparison convinced us. The lab-owned blocks are now JSON definitions in
blockDefinitions/extraBlocks.ts(174c39a), and the DB-pool plan for them is dead.What sold it: TypeScript started checking things the classic config path never sees. The conversion immediately surfaced a stale
Globals.d.tsstub that was typing the whole@blockly/field-bitmapplugin asany(removed — the plugin ships real types), and abuttonsconfig missing a required key. Two real latent problems for the price of a format change.The parts that survive either format, for the record: the costume/block/grid pickers needed registered
fromJsonfield types (imagePickerFields.ts+ the grid fields inextraBlocks.ts, same pattern asSceneDropdown), and the interpreted helperCode rides a parallelSPRITELAB2_HELPER_CODElist the engine prepends, since JSON has no slot for it.Block types, field names, and generated code are unchanged, so saved projects and the AI codegen emitters are unaffected — verified in the browser (blocks render with their grid defaults, programs execute, behaviors still plug into
add behavior).
| }); | ||
|
|
||
| // The scenes UI variant replaces the World tab with per-scene code workspaces. | ||
| const ENABLED_TABS: readonly SpriteLab2Tab[] = SCENES_UI_VARIANT |
There was a problem hiding this comment.
Can we kill all the SCENES_UI_VARIANT branching from this file and remove the experiment flag now?
There was a problem hiding this comment.
From Claude:
Done — removed in e45202c, along with
experiments.ts(that flag was all it held). The!SCENES_UI_VARIANTbranches were already dead sincegetScenesmigrates single-source projects into a default scene.
The scenes UI is the only variant now; the flag was hardcoded true and every !SCENES_UI_VARIANT branch was dead (getScenes already migrates single-source projects into a default scene). Per review on #73973. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
What extraSharedBlocks.ts's blocks look like in the modern JSON format (per review discussion on #73973): the simplest behavior and the hardest composite, plus the fromJson field wrappers they need. Also removes the stale Globals.d.ts stub for @blockly/field-bitmap — the plugin ships real types, and the shorthand declaration was shadowing them (the whole module was `any`). Not wired into the lab; typechecks, nothing imports it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The behaviors and grid composites move from the pool-block config shape (extraSharedBlocks.ts) to JSON definitions with standalone generators (blockDefinitions/extraBlocks.ts), per review on #73973. Block types, field names, generated code, and helperCode are unchanged, so saved projects and the AI codegen emitters are unaffected; verified in the browser (all 7 blocks register, composites render their two-row layout with grid defaults, a makeSpriteAtGrid program creates its sprite). Supporting changes: the image pickers move to imagePickerFields.ts and gain registered field types alongside the classic customInputTypes; the 8x8 grid fields get registered types (single-select as its own type); BlockStyles gains SPRITE; the engine prepends SPRITELAB2_HELPER_CODE (JSON has no helperCode slot, so the interpreted half rides a parallel list shaped like sharedBlocks entries). Typechecking immediately caught real looseness: a stale Globals.d.ts stub was typing the whole @blockly/field-bitmap plugin as any (removed; the plugin ships real types), and the bitmap buttons config was missing a required key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Style pass on the conversion: refreshAnimationDropdownThumbnails moves to imagePickerFields (the dropdowns' module, and setup.ts's last reason to import the field class); fromJson builds instances of its own field subclass; the ExtraBlock interface sits above its uses; one comment still said "this pool". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| if (itemType === 'sprite') { | ||
| fullPrompt = `${fullPrompt} Use a plain solid bright green (#00FF00) background that extends to all edges. Do not include any scenery, ground, sky, or other background elements — only the subject on a flat green background.`; | ||
| } else if (itemType === 'block') { | ||
| fullPrompt = `${fullPrompt} Draw one large square block centered in the image, with a clear margin of plain solid bright green (#00FF00) background around all four sides, extending to the image edges. No background scene — just the block on the flat green background.`; |
There was a problem hiding this comment.
Should the 'green' be configurable or omitted so that the user's prompt can set the color?
| // animationList slice, so the model only references images that actually | ||
| // exist. Exported so the generate flow can validate the model's output | ||
| // against the same lists. | ||
| export function getAvailableImageNames(): { |
There was a problem hiding this comment.
Maybe not for this change, but feels like this could be a selector that's called in the view (GenerateSpriteLab) and then passed to askSpriteLabAi as a param.
|
|
||
| // A behavior block: a value block generating `func()`, whose helperCode | ||
| // defines func as a factory returning {func: perTickFn, name}. | ||
| function behavior( |
There was a problem hiding this comment.
I'd prefer we settle on a consistent format rather than adding more types; can we define JSON-first blocks in separate files similar to goToExternalScene.ts, etc, and export a single list from index.ts? We can add helperCode as a new export from the block files that need them since that gets fed to the engine separately anyway (though I wonder; if the helper code is declared inline with the generator output, does that also just work? or alternatively; is it worth having a library of shareable helper code that lives outside of block definitions?)
Also happy to consider an alternative, just want to make sure we don't have too many paradigms floating around 🙂
| 'makePlatformBlocks', | ||
| [ | ||
| 'function makePlatformBlocks(animation, layout) {', | ||
| " makeEnvironmentSprites(animation, 'walls', layout);", | ||
| '}', | ||
| ] |
There was a problem hiding this comment.
This is an example where it feels like helperCode is redundant; the generator could just call through to makeEnvironmentSprites(animation, 'walls', layout);
[written by Claude]
This PR proposes focusing this lab on two kinds of scenes: platformers and stories. The impetus was playtesting: the direction worked, but it took a dozen-plus blocks of setup before anything moved on screen. With this change a working platformer is two blocks, stories get big characters and timed dialogue, and AI code generation sticks to whichever of the two the student is going for.
How we got here
The first plan was a World tab: the scene picker governing World and Code sub-tabs, a WYSIWYG placement grid with real costume images and a world-type picker (none / platform / topdown), and the runtime reading the placements — instantiating the sprites, applying that world type's gravity — while the student wired up controls and jump with visible blocks. We built that far enough to try it, and decided against it: it split scene authoring across two places, and the placement grid duplicated the grid widget blocks already have. So the World tab is gone from this branch, and placement moved into the blocks themselves: the same 8x8 grid widget, embedded in the composite blocks below, does what the world editor would have done. A scene's world is just code like everything else, which also means the AI can write it: generated platformer programs went from ~15 blocks to 2.
Example
make platform blocks(paint the level on the grid, pick a block image) plusmake platform player(pick a costume and a starting cell) is a playable platformer: arrows move, space jumps when standing on a wall, gravity and collisions come from the zGameDev helper library. The composites are optional — they're a fast start, and the full existing block set still works alongside them, both for building scenes without them and for layering onto them (set speed, gravity, more events and behaviors).Platform
Beyond the two composites,
make new sprite at grid locationplaces any other sprite on the same grid. A new patrolling left and right on blocks behavior walks platforms and turns at gaps, edges, walls, and the playspace bounds; it looks ahead with a point probe (new nativehasSupportAtcommand) rather than body collision, so even a cell-wide patroller turns at a one-cell gap instead of crossing it. A new "block" AI image type asks the model for a square tile with a green margin, then runs the same green-screen key as sprites and crops to content, so grid-placed copies tile seamlessly.Story
Story scenes lean on set background / make sprite / say / timed say, click and at-time events, scene jumps, and the simple movement behaviors. AI guidance sizes story characters at 150–250px so they fill the scene.
Toolbox and AI
Platform and Story categories are injected at the top of every level's toolbox, each with a default lineup; the level's existing categories and blocks are left intact below them. Levelbuilders keep control: a level that authors its own Platform or Story category uses it exactly as written, and authoring an empty one suppresses the category, so a level can offer both, one, or neither. Image pickers filter to the matching category (sprite / background / block) the way set-background already did. AI codegen opens its pseudocode with
profile: platform|storyand is restricted to that profile's vocabulary, so generated programs only draw on the matching category; programs without a profile line keep the legacy vocabulary.Preview vs play
Clicking the live preview opens Play from that scene as a starting point. The Play tab always shows Restart game and Restart scene, the game only owns the keyboard while on the Play tab (so arrow/space programs don't fight block editing), and scene-jump blocks don't navigate while previewing. The tab bar is restyled as chips per the design system so unselected tabs no longer look disabled.
Follow-ups
toolboxDefinitionbypasses it. The ensure-steps should move to the JSON representation and level files should migrate; the levelbuilder-override semantics carry over unchanged.