[Impeller] Namespace user-supplied shaders to prevent entrypoint collisions#186332
[Impeller] Namespace user-supplied shaders to prevent entrypoint collisions#186332bdero wants to merge 7 commits into
Conversation
… engine-internal shaders Flutter GPU user shaders, FragmentProgram (RuntimeEffect) shaders, and Impeller's own entity-contents shaders all register into the same per-Context impeller::ShaderLibrary, keyed only on (entrypoint name, stage). Because impellerc derives entrypoint names from source file stems with no scope prefix, a user-supplied shader whose .frag stem matches an Impeller-internal one would either shadow it on lookup or, in the FragmentProgram case (which intentionally evicts the previous function on dirty re-load to support hot reload), actively unregister the engine's pipeline function. Add a small ShaderKey::MakeUserScopedName helper that prefixes user shader entrypoints with "<scope>:<library_id>:" before they go into the shared registry. Engine-internal entrypoints stay unprefixed and are not spoofable, since impellerc-generated names are valid identifiers and cannot contain ':'. The library_id is anchored to the asset path the shader was loaded from (FragmentProgram via SetLibraryId on RuntimeStage; Flutter GPU via the asset name plumbed through ShaderLibrary::MakeFromFlatbuffer), so hot reload of the same asset continues to evict and replace the same registry slot, and shaders loaded from different packages or different asset bundles never collide. Fixes flutter#175464.
There was a problem hiding this comment.
Code Review
This pull request introduces a namespacing mechanism for user-supplied shaders to prevent collisions with engine-internal entrypoints by adding a library_id to RuntimeStage and Shader objects. Feedback focuses on performance and maintainability, specifically suggesting the caching of scoped names to avoid per-frame allocations, refactoring duplicated fallback ID generation logic, and using more idiomatic string appending.
…k id, tighten append - Cache the scoped fragment registry name on RuntimeEffectContents at SetRuntimeStage time so that the per-frame Render path no longer re-allocates and re-concatenates it on every call. - Move the fallback library-id helper (used when no asset path is available) into ShaderKey::MakeFallbackLibraryId so the runtime stage decode path and the Flutter GPU bundle path share one source of truth and one process-wide counter, and use std::to_string instead of std::ostringstream. - Tighten ShaderKey::MakeUserScopedName to use std::string::append with std::string_view directly.
impeller/renderer (where ShaderKey lives) already depends on impeller/runtime_stage, so runtime_stage.cc cannot pull in ShaderKey::MakeFallbackLibraryId without creating a cycle. Restore a TU-local helper in runtime_stage.cc and document why; flutter::gpu continues to use the shared ShaderKey helper since lib/gpu already depends on impeller/renderer. Both helpers use std::to_string per the earlier review feedback.
|
Ah yeah that should mostly resolve the user<->engine case. However, a deeper motivation for this change is actually to namespace user shaders against each other. Will retitle the PR. Concretely, this prevents:
|
| std::string ShaderKey::MakeUserScopedName(std::string_view scope, | ||
| std::string_view library_id, | ||
| std::string_view entrypoint) { | ||
| std::string out; |
There was a problem hiding this comment.
absl::StrCat please (better performance and ergonomics)
| ShaderKey::kScopeFlutterGPU, | ||
| "packages/pkg_a/assets/shaders.shaderbundle", | ||
| "solid_fill_fragment_main"), | ||
| "fg:packages/pkg_a/assets/shaders.shaderbundle:solid_fill_fragment_main"); |
There was a problem hiding this comment.
how long can we make these before we get a problem?
There was a problem hiding this comment.
Done in 09272b8: new ShaderKeyTest.MakeUserScopedNameHandlesLongInputs covers a 4096-byte library_id and a 2048-byte entrypoint. The scoped name is only used as a std::string map key, so there is no fixed length limit; this just pins that expectation.
| std::shared_ptr<const impeller::ShaderFunction> Shader::GetFunctionFromLibrary( | ||
| impeller::ShaderLibrary& library) { | ||
| return library.GetFunction(entrypoint_, stage_); | ||
| return library.GetFunction(GetScopedName(), stage_); |
There was a problem hiding this comment.
I don't think there is a test exercising this, is there?
There was a problem hiding this comment.
Done in 3617309: added Two shader bundles with the same entrypoint names do not collide in gpu_test.dart, which loads test.shaderbundle and a new test_alt.shaderbundle (both exporting UnlitFragment / UnlitVertex) and renders with a RenderPipeline from each.
| // same asset evicts the previous registration at the same scoped key, | ||
| // and so two different FragmentProgram assets cannot collide with each | ||
| // other or with engine-internal entrypoint names. | ||
| runtime_stage->SetLibraryId(asset_name); |
There was a problem hiding this comment.
I don't think there is a test for this either.
There was a problem hiding this comment.
Done in 3617309: added FragmentPrograms from different asset paths do not collide in fragment_shader_test.dart, which loads no_uniforms.frag.iplr and a byte-identical alias no_uniforms_alt.frag.iplr and paints a Picture with each.
* `ShaderKey::MakeUserScopedName` and `ShaderKey::MakeFallbackLibraryId` now use `absl::StrCat` instead of the manual `reserve` / `append` / `push_back` pattern. Improves readability and matches the prevailing style used elsewhere in Impeller. * Adds `ShaderKeyTest.MakeUserScopedNameHandlesLongInputs` covering a 4096-byte `library_id` and a 2048-byte `entrypoint`. The scoped name is only used as a `std::string` key in the per-process shader library registry, so there is no fixed length limit; this pins that expectation against future regressions. * Adds the `absl::strings` dependency to `impeller/renderer/BUILD.gn` for `absl::StrCat`.
…tPrograms Adds Dart-level integration tests that pin the user-visible behavior the shader namespacing change is supposed to deliver: two distinct user shader sources that share an embedded entrypoint name can be loaded and used in the same process without colliding in the shared shader library. * `testing/dart/gpu_test.dart`: new `Two shader bundles with the same entrypoint names do not collide` test that loads `test.shaderbundle` and a new `test_alt.shaderbundle` (both exporting `UnlitFragment` and `UnlitVertex`), constructs a `RenderPipeline` from each, and renders with both. Pre-namespacing this would have torn one of the two pipelines down at registration time. * `lib/ui/fixtures/shaders/BUILD.gn`: new `flutter_gpu_shaders_alt` impellerc target that produces `test_alt.shaderbundle` with the same `UnlitFragment` / `UnlitVertex` entrypoints as the existing bundle. * `testing/dart/fragment_shader_test.dart`: new `FragmentPrograms from different asset paths do not collide` test that loads `no_uniforms.frag.iplr` and a byte-identical alias `no_uniforms_alt.frag.iplr`, gets a `FragmentShader` from each, and paints a Picture with both. `no_uniforms.frag` was chosen so the test does not need to thread sampler or uniform setup through the FragmentShader to validate it. * `lib/ui/fixtures/shaders/general_shaders/BUILD.gn`: new `no_uniforms_alt` copy action that publishes `no_uniforms.frag.iplr` under a second asset name so FragmentProgram tracks the two loads under distinct library ids. Verified locally on both `flutter_tester` (Metal) and `flutter_tester_opengles` (SwANGLE). The unrelated `impellerc produces reasonable JSON encoded IPLR files` failure in `fragment_shader_test.dart` is pre-existing on master.
|
autosubmit label was removed for flutter/flutter/186332, because - The status or check suite Linux linux_unopt has failed. Please fix the issues identified (or deflake) before re-applying this label. |
…sion test CI analyzer flagged `omit_obvious_local_variable_types` on four locals in the new `FragmentPrograms from different asset paths do not collide` test where the right-hand side is a constructor call (`Paint()`, `PictureRecorder()`, `Canvas(...)`) or a typed list literal (loop variable). Drop the annotations so the lint passes.
|
Unfortunately hit lint churn. Added stuff to my pre commit hooks to hopefully prevent in the future. |
The prior fix for `omit_obvious_local_variable_types` overshot by also dropping the type annotation on `final picture = recorder.endRecording()` at fragment_shader_test.dart:88. `PictureRecorder.endRecording()` is a method call, not a constructor, so the analyzer considers the type non-obvious and the `specify_nonobvious_local_variable_types` lint fired on the next CI run. Restore `final Picture picture = ...` to keep the annotation on the function-call return while leaving the constructor locals (`Paint paint = Paint()`, `PictureRecorder recorder = ...`, `Canvas canvas = Canvas(...)`) without explicit types. Verified locally: `engine/src/flutter/ci/analyze.sh` reports "No issues found!".
Resolves #175464
Flutter GPU user shaders, FragmentProgram (RuntimeEffect) shaders, and Impeller's own entity-contents shaders all register into the same per-Context
impeller::ShaderLibrary, keyed only on(entrypoint name, stage). Because impellerc derives entrypoint names from source file stems with no scope prefix, a user-supplied shader whose.fragsource file matches an Impeller-internal one will collide. In the FragmentProgram case the collision is worse than a silent shadow:RuntimeEffectContents::RegisterShaderactively callsUnregisterFunctionon the existing entry when a new dirty stage arrives at the same key (this eviction path was added intentionally to support shader hot reload), so a user could in principle evict an engine pipeline function.This change introduces a small
ShaderKey::MakeUserScopedName(scope, library_id, entrypoint)helper that prefixes user-shader entrypoints with<scope>:<library_id>:before they go into the shared registry. Engine-internal entrypoints stay unprefixed and remain unspoofable: impellerc-generated names are valid identifiers and cannot contain:. AllRegisterFunction/UnregisterFunction/GetFunctionsites in the user-shader paths (and the runtime-effect pipeline cache key inContentContext) are routed through the helper. Engine-internal lookup sites (PipelineBuilder,ComputePipelineBuilder, the runtime-effect vertex stage inRuntimeEffectContents::CreatePipeline) are unchanged because they already use compile-time constant entrypoint names.The
library_idis anchored to a stable per-source identifier so that hot reload semantics are preserved:FragmentProgram::initFromAssetcallsRuntimeStage::SetLibraryId(asset_name). Hot reload of the same.fragasset produces a newRuntimeStagewith the samelibrary_id, so the existing eviction path atRuntimeEffectContents::RegisterShadercontinues to evict and replace at the same scoped key.flutter::gpu::ShaderLibrary::MakeFromAssetplumbs the asset name intoMakeFromFlatbuffer, which propagates it to each ownedflutter::gpu::Shader. A future hot reload story for Flutter GPU can land its own dirty/evict logic against this same scoped key.This also covers the multi-package case for free: Flutter's asset manifest already disambiguates package-shipped assets via the
packages/<pkg>/...prefix, so two packages each shipping a shader with the same internal name land at distinct asset paths and therefore distinctlibrary_idvalues.Tests
ShaderKey::MakeUserScopedName(correct format, contains the:separator that prevents engine-internal collisions, different library ids and different scopes do not collide).RuntimeStage::GetLibraryId(auto-assigned and unique by default, overridable viaSetLibraryId).All added under
impeller/runtime_stage/runtime_stage_unittests.cc.Pre-launch Checklist
///).