Skip to content

refactor: package-per-provider using uv workspaces and entry points; declarative distributions#5492

Draft
eoinfennessy wants to merge 36 commits intoogx-ai:mainfrom
eoinfennessy:poc/package-per-provider
Draft

refactor: package-per-provider using uv workspaces and entry points; declarative distributions#5492
eoinfennessy wants to merge 36 commits intoogx-ai:mainfrom
eoinfennessy:poc/package-per-provider

Conversation

@eoinfennessy
Copy link
Copy Markdown
Contributor

@eoinfennessy eoinfennessy commented Apr 8, 2026 •
edited
Loading

Summary

Proof-of-concept for #5478. Extracts the Ollama inference provider into a separate workspace package with entry-point-based discovery (924088e), and introduces a declarative distribution pattern built on top of it (ce99839). Commits after this (9762790 onwards) mock up a phased implementation of a full migration.

Provider extraction

See commit 924088e, which introduces uv workspaces, Python entry points, and moves the remote::ollama provider to its own package in the workspace.

  • Adds [tool.uv.workspace] to root pyproject.toml
  • Creates packages/llama-stack-provider-inference-ollama/ as a workspace member with its own pyproject.toml, dependencies, and entry point registration
  • Adds _discover_providers_from_entry_points() to get_provider_registry() — entry-point providers supplement registry-file providers
  • Removes the original Ollama provider and its registry entry to validate entry-point-only discovery
  • Container builds remain functional via the provider package name in pip_packages

Declarative distributions

See commit ce99839 for the example implementation.

  • Introduces distributions as standalone Python packages: a pyproject.toml declaring core + provider dependencies, a CONFIGS_DIR Path exported via the llama_stack.distributions entry point group, and optional YAML patches deep-merged onto an auto-generated base config
  • Config generation (llama stack config generate) discovers installed providers via entry points and builds a complete run config — no Python-as-config, no codegen scripts
  • Patch chaining via _base allows shared base patches (e.g. server settings) across variants
  • Config resolution (llama stack config show, llama stack run) discovers installed distribution packages via entry points, with distro::variant syntax for multiple configs
  • llama stack run auto-selects the sole installed distribution when no config is specified
  • Includes packages/llama-stack-distribution-demo/ as a working example using ollama inference
  • Adds scripts/distro_generate_config.sh for in-tree developers to generate configs in an isolated uv run --project environment

Validated

  • uv lock and uv sync resolve the workspace correctly
  • Entry point inference.remote.ollama is discoverable via importlib.metadata
  • get_provider_registry() discovers the provider via entry point when the registry file entry is absent
  • uv build produces a valid sdist + wheel with lockstep versioning via setuptools-scm
  • Published to TestPyPI and verified entry-point discovery from a clean install
  • llama stack config generate produces correct config from entry-point providers + patches
  • llama stack config show demo resolves and prints config from installed distribution package
  • All pre-commit hooks pass
  • 290 unit tests pass, 0 regressions

Test plan

  • uv lock && uv sync succeeds
  • uv run python -c "from importlib.metadata import entry_points; print(entry_points(group='llama_stack.providers'))" returns the Ollama entry point
  • uv run python -m pytest tests/unit/distribution/ tests/unit/providers/utils/ --tb=short — 290 passed
  • uv build --package efenness-llama-stack-provider-inference-ollama produces valid wheel
  • Published to TestPyPI and verified in clean environment
  • ./scripts/distro_generate_config.sh packages/llama-stack-distribution-demo --patch packages/llama-stack-distribution-demo/patches/config.yaml generates correct config
  • uv run llama stack config show demo resolves installed distribution and prints config
  • Pre-commit hooks pass (ruff, mypy, all checks)

🤖 Generated with Claude Code

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Apr 8, 2026
@eoinfennessy @claude
poc: extract Ollama provider into separate package with entry-point d…
924088e 
…iscovery

Set up a uv workspace and extract the Ollama inference provider into a
separate package (efenness-llama-stack-provider-inference-ollama) to
validate the package-per-provider approach proposed in llamastack#5478.

Key changes:
- Add [tool.uv.workspace] to root pyproject.toml with members for
  src/llama_stack_api and packages/*
- Create packages/llama-stack-provider-inference-ollama with its own
  pyproject.toml, entry point registration, and provider source code
- Add _discover_providers_from_entry_points() to distribution.py for
  entry-point-based provider discovery alongside existing registry files
- Remove the original Ollama provider from src/llama_stack/providers/
  and its registry entry, proving the entry-point path works standalone
- Update distribution configs via codegen, tests, and the native
  messages module path list

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy eoinfennessy force-pushed the poc/package-per-provider branch from a2c190b to 924088e Compare April 9, 2026 08:17
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 9, 2026

This pull request has merge conflicts that must be resolved before it can be merged. @eoinfennessy please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Apr 9, 2026
eoinfennessy and others added 2 commits April 13, 2026 13:11
@eoinfennessy @claude
fix: include entry-point providers in registry available_providers()
915f277 
Add discover_entry_point_providers() to distribution.py and
merge_entry_point_providers() utility in the registry package so that
available_providers() includes entry-point-discovered providers
alongside in-tree ones. This enables distribution config codegen to
pick up providers that have been extracted into separate packages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy
Merge branch 'main' into poc/package-per-provider
ff5343b
@mergify mergify Bot removed the needs-rebase label Apr 13, 2026
eoinfennessy and others added 3 commits April 13, 2026 14:02
@eoinfennessy @claude
fix: use empty pip_packages for workspace-managed provider package
f1e94fa 
The ollama provider package dependencies are managed by its
pyproject.toml and installed via uv sync as a workspace member. Listing
the package name in pip_packages caused uv pip install (which is not
workspace-aware) to resolve it from PyPI, pulling in a stale version of
llama-stack that overwrote the editable install.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
fix: sync all workspace packages before distro codegen
ef3cd66 
The distro codegen pre-commit hook needs entry-point-migrated providers
to be installed so their provider specs are discoverable. Change the
hook to sync all workspace packages before running codegen, ensuring
providers in packages/ are always available regardless of which
dependency groups are active.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
fix: skip unknown providers in get_provider_dependencies instead of c…
658714b 
…rashing

In environments where an entry-point provider package is not installed
(e.g. container builds), the provider type won't be in the registry.
Log a warning and skip instead of raising ValueError, since the
provider's own package manages its dependencies.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy eoinfennessy changed the title poc: package-per-provider using uv workspaces and entry points refactor: package-per-provider using uv workspaces and entry points Apr 13, 2026
@eoinfennessy @claude
feat: add declarative distribution config generation with entry-point…
ce99839 
… discovery

Introduce a new pattern for building Llama Stack distributions as standalone
Python packages. Distributions declare their providers as dependencies in
pyproject.toml and register a CONFIGS_DIR via the llama_stack.distributions
entry point group. Config generation discovers installed providers via entry
points, builds a base config with default storage/server settings, and
deep-merges YAML patches (with _base chaining) on top.

New CLI commands:
- llama stack config generate: generate a run config from the current environment
- llama stack config show: print a resolved config to stdout

Also extends config resolution to discover installed distribution packages,
adds auto-selection of a sole installed distribution for llama stack run,
and includes a demo distribution package using ollama inference.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy eoinfennessy changed the title refactor: package-per-provider using uv workspaces and entry points refactor: package-per-provider using uv workspaces and entry points; declarative distributions Apr 14, 2026
eoinfennessy and others added 11 commits April 14, 2026 21:37
@eoinfennessy @claude
refactor: clean up core dependencies and move router utilities into core
9762790 
Move InferenceStore and parse_filter from providers/utils/ into
core/routers/ where they belong, since core routers are the only
consumers. Backward-compatible re-export shims remain at the old
import paths for existing provider code.

Remove 8 packages from core dependencies that are only used by
providers: numpy, oci, oracledb, fire, prompt-toolkit, jsonschema,
tornado, and urllib3. Move tornado and urllib3 security pins to
constraint-dependencies where they apply transitively. Move tiktoken
to the dev dependency group since it is only needed by provider tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
refactor: extract shared provider utilities into separate workspace p…
2a10c78 
…ackages

Move provider utilities from src/llama_stack/providers/utils/ into five
dedicated workspace packages: llama-stack-utils-inference,
llama-stack-utils-vector-io, llama-stack-utils-safety,
llama-stack-utils-bedrock, and llama-stack-utils-common. Each package has
its own pyproject.toml with only the dependencies it needs.

The original file paths are preserved as sys.modules aliases so that all
existing provider imports and unittest.mock.patch calls continue to work
without modification. These shims will be removed as providers are
extracted into their own packages in later phases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
refactor: extract torch-dependent providers into separate workspace p…
89cf36d 
…ackages

Extract the 4 torch-dependent providers into their own packages under
packages/, removing them from the in-tree registries and discovering
them via entry points instead. This isolates heavy torch/transformers
dependencies so they are only installed when the provider is needed.

New packages:
- llama-stack-provider-inference-sentence-transformers
- llama-stack-provider-inference-transformers
- llama-stack-provider-safety-prompt-guard
- llama-stack-provider-tool-runtime-file-search

Also adds merge_entry_point_providers() to the safety and tool_runtime
registries (inference already had it) and fixes the list-deps test to
use a provider that still has pip_packages declared.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
refactor: extract all remote inference providers into separate worksp…
be5c9a0 
…ace packages

Extract all 19 remaining remote inference providers from the in-tree
registry into their own workspace packages under packages/. The
inference registry is now empty - all providers are discovered via
entry points.

New packages: anthropic, azure, bedrock, cerebras, databricks,
fireworks, gemini, groq, llama-cpp-server, llama-openai-compat,
nvidia, oci, openai, passthrough, runpod, sambanova, together,
vertexai, vllm, watsonx.

Also regenerates distribution configs and provider docs to reflect the
new module paths and entry-point-based provider ordering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
refactor: extract all vector I/O providers into separate workspace pa…
686261c 
…ckages

Extract all 13 vector I/O providers (faiss, sqlite-vec, chromadb, milvus,
qdrant, pgvector, weaviate, elasticsearch, oci, infinispan) into individual
workspace packages discovered via entry points. For chroma, milvus, and
qdrant which share code between inline and remote variants, inline configs
are moved into the remote package and the inline package depends on the
remote package to avoid circular dependencies. The vector_io registry is
now fully empty with all providers discovered via entry points.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
fix: rename ollama provider package and update its imports to use uti…
948d392 
…lity packages

The PoC ollama package was still using the efenness- prefix and importing
from the old in-tree llama_stack.providers.utils paths. This renames it to
llama-stack-provider-inference-ollama, adds llama-stack-utils-inference as a
dependency, and rewrites imports to use the extracted utility packages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
feat: extract safety, tool runtime, files, and file processor provide…
56b1541 
…rs into separate workspace packages

Extract 16 providers into individual workspace packages with entry-point
discovery, completing Phase 6 of the package-per-provider refactoring:

Safety (6): llama-guard, code-scanner, bedrock, nvidia, passthrough, sambanova
Tool Runtime (5): brave-search, bing-search, tavily-search, wolfram-alpha, model-context-protocol
Files (3): localfs, s3, openai
File Processors (2): pypdf, docling

All four registry files (safety, tool_runtime, files, file_processors) now
use merge_entry_point_providers with an empty in-tree list, making them
fully entry-point driven. Provider imports are rewritten to use the
extracted utility packages (llama-stack-utils-inference, llama-stack-utils-safety,
llama-stack-utils-bedrock, llama-stack-utils-common, llama-stack-utils-vector-io).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
feat: extract batches, interactions, messages, and responses provider…
fdc558d 
…s into separate workspace packages

Move the remaining providers with registry files into their own workspace
packages with entry-point discovery. Registry files for batches, interactions,
messages, and responses are now fully entry-point driven, completing the
migration of all registry-based providers to the package-per-provider model.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
feat: extract distribution configs into separate workspace packages
dfc6bb7 
Create 7 distribution packages (starter, ci-tests, nvidia, oci,
open-benchmark, watsonx, postgres-demo) that register via
llama_stack.distributions entry points and expose their config YAML
files. This enables distributions to be installed independently and
discovered at runtime via importlib.metadata.

Update distro_codegen.py to sync generated configs into the
distribution package directories alongside the in-tree copies.
The in-tree distribution configs will eventually be removed once
config generation moves to `llama stack config generate`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
refactor: move llama-stack-api package to packages/ workspace directory
e810f54 
Move llama-stack-api from src/llama_stack_api/ to
packages/llama-stack-api/src/llama_stack_api/ so it follows the same
src-layout as all other workspace members under packages/*.

Update all references in pre-commit hooks, GitHub Actions workflows,
scripts, mypy config, grandfathered file paths, and conformance docs.
The workspace members glob (packages/*) now covers this package
without a separate entry. Switch the setuptools config from an
explicit package/module list to packages.find with where=["src"].

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
refactor: move core llama-stack package to packages/ workspace directory
469b2df 
Move src/llama_stack/ to packages/llama-stack/src/llama_stack/ so the
core package becomes a workspace member alongside llama-stack-api and
all provider/distribution packages. The root pyproject.toml is now a
workspace-only config file with no [project] section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 15, 2026

This pull request has merge conflicts that must be resolved before it can be merged. @eoinfennessy please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Apr 15, 2026
eoinfennessy and others added 3 commits April 15, 2026 13:53
@eoinfennessy @claude
chore: normalize pyproject.toml structure and versioning across all p…
657e9a6 
…ackages

Standardize all 73 workspace packages to use consistent setuptools-scm
versioning with fallback_version = "0.7.2.dev0" and reorder TOML sections
to follow a canonical layout: [project] -> [project.*] -> [build-system]
-> [tool.*].

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
chore: update release pipeline for workspace-wide package publishing
119d8b7 
Update all three release workflows to handle ~73 workspace packages instead
of just llama-stack and llama-stack-api. pypi.yml now builds all packages in
a single job via `uv build --all-packages`. prepare-release.yml pins all
internal workspace dependencies to ==${VERSION} at release time.
post-release.yml bumps fallback_version across all workspace packages.

Also normalizes llama-stack package name from underscore to hyphen form.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
Merge upstream/main into poc/providers-over-grpc
652e48e 
Resolve conflicts:
- providers-build.yml: keep workspace path, add upstream's PIP_CONSTRAINT_FILE arg
- pypi.yml: take upstream's pypi-publish v1.14.0 bump, keep our publish structure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@mergify mergify Bot removed the needs-rebase label Apr 15, 2026
eoinfennessy and others added 9 commits April 15, 2026 15:17
@eoinfennessy @claude
fix: remove --with-editable from unit-tests.sh for workspace-only root
44edd52 
The root pyproject.toml is now a workspace config without a [project]
table, so uv cannot build it as an editable wheel. Workspace packages
are already available via uv sync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
fix: update REPO_ROOT paths for workspace directory structure
d415526 
The llama-stack package moved from src/ to packages/llama-stack/src/,
adding two extra directory levels. The REPO_ROOT calculations using
Path(__file__).parent chains were too shallow, causing the integration
test recording/replay system to look for recordings in the wrong
directory and fail to find them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
fix: patch extracted provider classes in test recording system
7f810f4 
The recording system was monkeypatching TavilySearchToolRuntimeImpl and
PyPDFFileProcessorAdapter from in-tree paths, but the server loads these
classes from the extracted workspace packages via entry points. Since
they are different class objects, the patches had no effect and web
search tests made unrecorded live calls that failed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
fix: update CI workflows and Containerfile for workspace-only root py…
565063e 
…project.toml

The root pyproject.toml is now a workspace definition without a [project]
table, so `uv pip install -e .` fails. Update all CI workflows and the
Containerfile to install the core package at packages/llama-stack instead.
Also sync the in-tree messages impl.py with cache metrics changes that
landed on the extracted package during the main merge, and fix the
backward-compat workflow to handle both old and new distribution paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
feat: remove in-tree provider copies in favor of extracted packages
0bfe44f 
Delete all duplicated provider source code from the monolith now that
every extracted provider lives in its own package under packages/. This
eliminates the dual-copy divergence problem where merging main would
update one copy but not the other (e.g., the messages cache metrics bug).

Key changes:
- Delete ~53 provider directories from providers/inline/ and providers/remote/
- Delete providers/utils/ (all utils extracted to llama-stack-utils-* packages)
- Move AssistantMessageWithReasoning to llama-stack-utils-inference
- Update ~225 test imports to use extracted package paths
- Update distribution config imports (starter, ci-tests, open-benchmark, etc.)
- Fix _NATIVE_INTERACTIONS_MODULES to use extracted package path
- Fix cross-package imports (CompactionConfig, forward_headers, connectors/mcp)

Providers not yet extracted are left in-tree: agents, datasetio, eval,
scoring, post_training, ios, inference/meta_reference, tool_runtime/rag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
chore: remove unused provider deps and add missing unit test deps
b19c72f 
Remove 21 provider and utils packages from llama-stack dependencies
that are no longer directly imported after deleting in-tree copies.
Add 5 provider packages newly needed by distribution configs
(files-localfs, file-processor-pypdf, safety-nvidia,
tool-runtime-brave-search, tool-runtime-tavily-search).

Also add llama-stack-client and qdrant-client to the unit test
dependency group so all unit tests can run with --group unit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
chore: remove duplicate ruff config from llama-stack-api package
6c06cbb 
The ruff configuration in llama-stack-api's pyproject.toml was a
duplicate of the root config, left over from before the workspace
migration. Remove it and move the F403 per-file-ignore for API
__init__.py files to the root config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
chore: remove duplicate uv required-version from llama-stack-api
ea44dd2 
Already specified in the root pyproject.toml which covers the workspace.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
chore: remove starter optional dependency extra
1a66f1f 
Dependencies are now resolved from the llama-stack-distribution-starter
package's own pyproject.toml. The starter extra was a legacy flat list
of ~50 third-party libraries predating the workspace structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 15, 2026

This pull request has merge conflicts that must be resolved before it can be merged. @eoinfennessy please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Apr 15, 2026
eoinfennessy and others added 4 commits April 16, 2026 01:27
@eoinfennessy @claude
feat: add declarative config overlay system with typed schema
0174eb9 
Replace the Python-based distribution config generators (starter.py at
418 lines of Python and build.yaml at 56 lines of YAML, ci_tests.py at
120 lines of Python) with a declarative overlay system using structured
YAML files validated by Pydantic. Overlays support four top-level keys
with clear execution ordering:

  1. base — chain to a parent overlay file
  2. composition — pre-merge directives (e.g. storage: postgres)
  3. patch — strategic merge data (providers merged by provider_type)
  4. finalizers — post-merge transforms (e.g. conditional provider IDs)

The starter distribution is now two overlay files (56 lines of YAML)
and ci-tests extends it with a 55-line overlay that adds watsonx, test
connectors, pre-registered models, and auth config. Generated configs
are verified semantically identical to the originals via yq.

Also removes the demo distribution package and in-tree starter and
ci-tests distribution copies, as both are superseded by the overlay
system and extracted packages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
feat: convert remaining 5 distributions to declarative config overlays
52309f7 
Convert nvidia, watsonx, oci, open-benchmark, and postgres-demo
distributions to use the new overlay-based config generation system,
replacing 1730 lines of Python config generators and static YAML with
128 lines of declarative overlay YAML.

Add MergeStrategy enum and MergeDirectives model to ConfigOverlay
schema, enabling per-API control over how provider lists are merged
(strategic_merge vs replace).

Remove the in-tree distributions/ directory entirely, along with the
DistributionTemplate system and 25 provider dependencies from the
core llama-stack package that were only needed by the old generators.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
refactor: move MCP client and TTLDict from llama-stack-utils-common i…
2b65ee3 
…nto core

Move mcp.py into llama_stack.core.mcp and TTLDict into
llama_stack.core.utils.ttl_dict, eliminating a circular dependency
between llama-stack and llama-stack-utils-common. Remove both
llama-stack-utils-common and llama-stack-utils-inference as
dependencies of the core llama-stack package.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
fix: restore rich provider descriptions lost during package extraction
6800caf 
When providers were extracted to separate packages, their provider.py
entry point files received placeholder one-liner descriptions instead
of the rich multi-paragraph documentation from the original registry
files. This caused provider_codegen.py to generate degraded docs
missing Features, Usage, Installation, and Search Modes sections.

Restores descriptions for 10 providers with rich content (faiss,
sqlite-vec, chromadb inline/remote, qdrant inline, weaviate, milvus,
pgvector, vertexai, azure) and fixes 4 minor wording differences
(llama-guard, milvus-inline, qdrant remote, s3). Also removes unused
BUILTIN_DEPS constant from inference registry and regenerates all
provider documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
cdoern
cdoern requested changes Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@cdoern cdoern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initial review regarding our release workflow actions. Generally, there seems to have been a huge amount of change to prepare-release and pypi yml actions. Let's see which of these changes are actually necessary and only introduce those. These actions are very crucial and easy to break. Breaking these would halt any future releases until they are fixed so we need to be careful.

Comment thread .github/workflows/prepare-release.yml

# --- Update llama-stack-client pins in pyproject.toml ---
# Update the pin in [project.optional-dependencies] and [dependency-groups]
# Match patterns like: "llama-stack-client>=X.Y.Z" or "llama-stack-client==X.Y.Z"
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets try and preserve these comments if we can

Comment thread .github/workflows/prepare-release.yml
# Show what changed
echo "=== Changes ==="
git diff
git diff --stat
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

Comment thread .github/workflows/pypi.yml
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "Computed version: ${VERSION}"

# Build and validate release artifacts
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happened to the - name: Check if package should be built step?

Copy link
Copy Markdown
Contributor Author

@eoinfennessy eoinfennessy Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new workflow uses uv build --all-packages to build everything in a single job, so the per-package check is no longer needed. Now workspace packages are always built together.

Comment thread .github/workflows/pypi.yml
id: client-ref
run: |
if [ -n "${{ inputs.client_ref }}" ] && [ "${{ inputs.client_ref }}" != "main" ]; then
# Explicit override from workflow_dispatch
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please preserve these comments, all of the ones removed here.

Comment thread .github/workflows/pypi.yml
# === PYTHON SETUP (for all Python packages) ===
# === PYTHON SETUP ===
- name: Set up Python
if: steps.should-build.outputs.skip != 'true' && matrix.registry == 'pypi'
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why removed?

Comment thread .github/workflows/pypi.yml
if [[ ! "$VERSION" =~ \.dev ]]; then
echo "Checking if ${PACKAGE} ${VERSION} exists on PyPI..."

# Parse version components
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preserve comments

Comment thread .github/workflows/pypi.yml
done

if [ "$VERSION" != "$BASE_VERSION" ]; then
echo "::notice::Bumped ${MATRIX_PACKAGE} (PyPI: ${PACKAGE}) version from ${BASE_VERSION} to ${VERSION} (already existed on PyPI)"
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preserve this if else logic

Comment thread .github/workflows/pypi.yml
ls -la dist/

- name: Upload artifacts (local)
if: steps.should-build.outputs.skip != 'true' && matrix.type == 'local'
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like we removed the differentiation between an external and local build (the clients are external). why?

Copy link
Copy Markdown
Contributor Author

@eoinfennessy eoinfennessy Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

External and local builds have been split from a single build-package into two jobs: build-local-packages and build-external-package

Comment thread .github/workflows/pypi.yml
# Order: llama-stack-client-python, llama-stack-client-typescript, llama-stack-api, llama-stack
publish-packages:
name: Publish ${{ matrix.package }}
# Publish all local workspace packages to PyPI in a single job
Copy link
Copy Markdown
Collaborator

@cdoern cdoern Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the point in this change?

eoinfennessy and others added 2 commits April 21, 2026 10:51
@eoinfennessy @claude
refactor: simplify config generation to rely on isolated venv for pro…
3e75d08 
…vider filtering

Remove the distribution-package-based provider filtering logic from the
config generator. Instead, rely on the isolated venv created by
distro_generate_config.sh to ensure only the correct providers are
discovered via entry points. This eliminates the need to inspect
distribution dependencies at generation time.

Also update distro_generate_config.sh to create a proper temporary venv
instead of using uv run --project, and add --distribution flag support
for environments with multiple distributions installed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy @claude
feat: add minimal eoin distribution with OpenAI inference provider
45eaaa0 
Add a new lightweight distribution package containing only the OpenAI
inference provider, demonstrating the minimum viable distribution for
the package-per-provider architecture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Eoin Fennessy <efenness@redhat.com>
@eoinfennessy eoinfennessy mentioned this pull request Apr 22, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cdoern cdoern cdoern requested changes

@raghotham raghotham Awaiting requested review from raghotham raghotham will be requested when the pull request is marked ready for review raghotham is a code owner

@leseb leseb Awaiting requested review from leseb leseb will be requested when the pull request is marked ready for review leseb is a code owner

@bbrowning bbrowning Awaiting requested review from bbrowning bbrowning will be requested when the pull request is marked ready for review bbrowning is a code owner

@mattf mattf Awaiting requested review from mattf mattf will be requested when the pull request is marked ready for review mattf is a code owner

@franciscojavierarceo franciscojavierarceo Awaiting requested review from franciscojavierarceo franciscojavierarceo will be requested when the pull request is marked ready for review franciscojavierarceo is a code owner

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Meta Open Source bot. needs-rebase

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@eoinfennessy @cdoern