feat(postgresql): add support for table partitioning#7497
Merged
B4nan merged 57 commits intomikro-orm:nextfrom Apr 19, 2026
Merged
feat(postgresql): add support for table partitioning#7497B4nan merged 57 commits intomikro-orm:nextfrom
B4nan merged 57 commits intomikro-orm:nextfrom
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## next #7497 +/- ##
==========================================
- Coverage 99.65% 99.65% -0.01%
==========================================
Files 265 267 +2
Lines 27621 28087 +466
Branches 7169 7775 +606
==========================================
+ Hits 27526 27989 +463
- Misses 89 92 +3
Partials 6 6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
B4nan
requested changes
Apr 6, 2026
Member
B4nan
left a comment
There was a problem hiding this comment.
Thanks! Apart from the broken CI checks, please also use lowercase for sql. Also ideally get to a complete code coverage.
Author
|
Old habits die hard, I keep subconsciously capitalizing SQL queries. |
Member
|
I'll try to have a closer look later this week, for now, this is what claude things about the PR: |
…orm#7347) ## Summary - Add `migration:log` and `migration:unlog` CLI commands - `migration:log` marks a migration as executed without running it - `migration:unlog` removes it from the executed list without reverting - Add entries to the migrations documentation Closes mikro-orm#5390 --------- Co-authored-by: Martin Adámek <banan23@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary Adds a `discovery:export` CLI command that scans entity source files and generates a TypeScript barrel file (mikro-orm#7323). The generated file provides: - `export const entities = [...] as const` — for use in ORM config - `export type Database = ...` — for typed `getKysely<Database>()` in DI contexts (NestJS, etc.) ### Command usage ```bash mikro-orm discovery:export [--path './src/entities/*.ts'] [--out ./entities.generated.ts] [--dump] ``` ### Files changed - **New:** `packages/cli/src/commands/DiscoveryExportCommand.ts` — the command implementation - **Modified:** `packages/cli/src/CLIConfigurator.ts` — register the command - **New:** `tests/features/cli/DiscoveryExportCommand.test.ts` — 14 tests - **Modified:** `tests/features/cli/CLIHelper.test.ts` — updated command list assertion - **Modified:** `docs/docs/kysely.md` and `docs/versioned_docs/version-7.0/kysely.md` — documentation ## Test plan - [x] 14 unit tests covering: dump mode, file write, ESM/CJS imports, EntitySchema dedup, decorator entities, multiple entities, driver package resolution, config path fallback, error cases - [x] `yarn build` passes (CLI package) - [x] `yarn tsc-check-tests` passes - [x] Full test suite passes (5314/5314) Closes mikro-orm#7323 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…hatch (mikro-orm#7587) ## Summary Three additive, non-breaking features for to-one relations, prompted by [discussion mikro-orm#7585](mikro-orm#7585). ### `LazyRef<T>` Type-only marker for `@ManyToOne` / `@OneToOne`. Runtime is a plain entity (no `Reference` wrapper, `instanceof User` returns `true`). TypeScript restricts access to the primary key until `Loaded<>` narrows it back to the full entity — no `.$` / `.get()` indirection needed on loaded relations. ```ts @manytoone(() => Author) author!: LazyRef<Author>; // or via defineEntity: author: () => p.manyToOne(AuthorSchema).lazyRef() book.author.id; // ok — PK always accessible book.author.name; // compile error — not loaded const loaded = await em.findOneOrFail(Book, 1, { populate: ['author'] }); loaded.author.name; // ok — Loaded<> strips the brand ``` Scope is to-one only. Collections keep `Collection<T>`. ### `Loadable` mixin Adds `load()` / `loadOrFail()` to an entity's prototype — Reference-style ergonomics for direct and LazyRef-typed relations. Ships as `Loadable(Base)` plus pre-composed `LoadableBaseEntity`. `BaseEntity` itself is deliberately untouched so `load` / `loadOrFail` aren't reserved on existing subclasses. ```ts class User extends LoadableBaseEntity { /* ... */ } await user.load(); // Promise<User | null> await user.loadOrFail(); // Promise<User> ``` ### `unref()` Typed escape hatch — inverse of the existing `ref()` helper. Narrows `Ref<T>` | `LazyRef<T>` | `T` down to `T` for cases where you know the relation is populated but can't thread `Loaded<>` through a function signature. ```ts import { unref } from '@mikro-orm/core'; function logAuthor(book: Book) { // `book.author` is typed as `LazyRef<Author>` — `.name` would be a compile error console.log(unref(book.author).name); } ``` Works for `Ref<T>` (calls `.unwrap()`), `LazyRef<T>` (identity cast), plain `T` (passthrough), and `null` / `undefined` (passthrough). ### Non-breaking - `BaseEntity`, `Ref<T>`, plain relations, and `@ManyToOne({ ref: true })` all unchanged. - `LazyRef<T>` is opt-in per property. - `.lazyRef()` is opt-in per builder chain and leaves runtime metadata untouched. ### Type performance `LazyRef` integration is isolated to `AutoPath` / `Loaded<>` narrowing via a new `ExtractTypeWithLazyRef` helper — the existing `ExtractType` hot path (`EntityData`, `RequiredEntityData`, `IsOptional`) pays no cost. New benches in `tests/bench/types/lazy-ref.ts` compare like-for-like against `Ref<T>`; LazyRef is slightly cheaper for loaded narrowing (no `LoadedReference` intersection). All existing benchmark baselines have been refreshed and the full `yarn bench:types` run shows 0% delta.
…ator gaps - preserve double quotes and internal whitespace inside single-quoted SQL literals in partitioning normalizers (quote-aware mapping helper) - align partition expression parsing with MetadataValidator for comma-separated identifier strings - thread platform.quoteIdentifier through getTablePartitioning so generated DDL emits properly-quoted partition key columns - split schema-qualified partition names on the first dot (schema.name) instead of keeping multi-dot names whole - emit comments/indexes/checks/triggers before partition children in createTable for more readable generated scripts - emit partitionBy entity option from entity-generator source files - drop partitioning docs from the frozen v7.0 snapshot (docs live under docs/docs only)
- exercise remaining branches in partitioning helpers: callback-expression passthrough, raw-SQL fallback, physical column-name lookup, empty-key guard, `from` bound without `to`, and empty bound string - cover MetadataValidator partition paths: callback/comma-separated/raw expressions, unique-property & unique-constraint coverage errors, no-op paths for unresolved keys and property-less uniques - short-circuit path in PostgreSqlSchemaHelper.getPartitions when all schema buckets are empty
Adds entity-generator and partitioning-helper coverage for callback `expression` values so the quoter-aware branches are reached both with and without a custom quoter.
Co-authored-by: Martin Adámek <banan23@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Schema Generator is the right home for partition documentation; the defineEntity page is for explaining the helper, not for platform-specific table features.
- `Platform.validateMetadata` now rejects `partitionBy` on platforms that don't support partitioned tables, so the error surfaces at ORM init (discovery) rather than only at schema-update time; MongoDB/MSSQL overrides delegate to super. The duplicate check in `DatabaseSchema.fromMetadata` is removed. - `MetadataValidator.getPartitionKeyFields` now resolves each key independently: resolvable keys still drive PK/unique-coverage checks even when other tokens in a comma-separated expression can't be mapped to a property or column.
- preserve `TO` tokens inside quoted literals when normalizing `from ... to ...` range bounds - parse compact `HASH(expr)` definitions without dropping the expression - return empty string for empty partition bounds instead of malformed `for values` - resolve partition keys via `meta.root.properties` to match `MetadataValidator` - drop the unreachable callback branch from the entity generator `partitionBy` declaration (only string/array expressions reach the generator via `toEntityPartitionBy`)
…partition-support # Conflicts: # packages/core/src/metadata/types.ts # packages/sql/src/dialects/postgresql/PostgreSqlSchemaHelper.ts # packages/sql/src/schema/DatabaseTable.ts # packages/sql/src/typings.ts
- only fold midnight timestamps with no offset or UTC offset in partition diffing (non-UTC offsets are semantically distinct) - emit function-typed partition expressions verbatim in entity generator instead of stringifying the source as a quoted string literal - include from/to definitions in the unsupported-repartition error so normalization mismatches are easier to diagnose - attribute partition validation errors to the root entity (where the unique constraint lives) rather than the validated meta
…og introspection Addresses staff-review findings on the partitioning work: - splice `partition by …` into `create table` via slice instead of `.replace(/;$/, …)` so dollar-quoted literals, `$&`, and `$1..$9` in user expressions are not interpreted as regex back-references - throw on unresolved partition keys during validation (and in the DDL path) so DDL cannot silently emit references to unknown columns - require an explicit offset on `YYYY-MM-DD 00:00:00` literal stripping to avoid false-negative diffs when a text/varchar list partition value legitimately looks like a midnight timestamp - quote-aware split of `type ` prefix in `toEntityPartitionBy` so bounds containing spaces inside quoted literals round-trip correctly - filter child partitions in `getListTablesSQL` via a (schema, relname) anti-join so children living in non-`search_path` schemas are not surfaced as untracked tables - collapse the per-schema OR tree in `getPartitions` into a single `values(…)` join keyed by (schema, table) so the query stays compact and sargable - mark `setPartitioning` `@internal` to match peer `DatabaseTable` mutators - fail loud in the entity generator if a callback-form `partitionBy.expression` ever reaches source emission, since `fn.toString()` would produce uncompilable TypeScript - propagate the `Entity` generic to `EntityMetadata.partitionBy` so internal code retains narrowed column types in callback expressions - document the opaque-expression coverage-skip in `getPartitionKeyFields` and simplify `normalizePartitionBound` to a single keyword-lowering pass that stays inside `mapOutsideLiterals` - add a `defineEntity` section to the docs so the schema-generator reference has a matching partition example on the other side
…ation
- extend `partitionBy.partitions` for `hash` to accept a name array alongside the
numeric count (names may carry a `schema.name` or quoted `"schema"."name"` prefix)
- have `splitPartitionName` honour double-quoted identifiers so partition names
containing `.` (e.g. `"my.schema"."part"`) are parsed correctly
- also strip the `00:00:00` time component when followed by an explicit
`::timestamp[tz]` cast, so `timestamp without time zone` range partitions diff
cleanly on round-trips (catalog output carries no offset for non-tz)
- declare the legacy `inheritance?: 'tpt'` field on `EntityMetadata` so the
validator's inheritance check no longer needs an inline cast
- preserve round-tripped hash partition names when the catalog names deviate from
the default `${tableName}_N` pattern, and drop the redundant `public.` prefix
when the child's schema matches the parent's
- document the symmetric `::text` strip behaviour in `normalizePartitionLiterals`
- drop the PG-only partitioning callout from `defineEntity` docs (one canonical
entry in `schema-generator.md` is enough)
…ator output - entity-generator now quotes custom hash partition names so introspected `partitions: ['a', 'b']` arrays round-trip as valid TypeScript. - `getPartitions` resolves undefined schema buckets against `current_schema()` instead of wildcarding across every schema that shares the table name. - `splitCommaSeparatedIdentifiers` (new shared util) respects double-quoted identifiers containing commas; replaces the previous regex + naive split. - `MetadataValidator.hasPartitionExpression` guards against non-string array entries so bad input surfaces as a clean `MetadataError`. - `skipQuotedLiteral` now returns past the end for unterminated literals so callers don't silently drop the tail character. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ate partition names PostgreSQL's pg_get_expr emits MINVALUE, MAXVALUE, and NULL in uppercase, but user-supplied lowercase bounds were not normalized, producing a permanent false-positive diff that could not be resolved (changing partitioning is blocked by getPreAlterTable). Add them to the bound keyword regex so case-folding happens symmetrically on both sides. Validate duplicate partition names for both hash (array form) and list/range partitions so users get a clear MetadataError at discovery time instead of a late, opaque PG DDL failure. Distinguish the composite-column case from the unknown-key case in resolvePartitionKey / resolvePartitionKeyField, naming the physical columns so the remedy is actionable. Also drop a redundant normalizePartitionLiterals call in normalizePartitionBound; normalizePartitionSqlFragment already runs it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Partition name collisions that only surface after PostgreSQL's identifier
folding (e.g. `Part_1` vs `part_1`) or between an explicit name and another
partition's auto-generated `${tableName}_${index}` default previously slipped
past the validator and only failed at DDL execution. Normalize names the way
PG resolves them — case-fold unquoted segments, preserve quoted ones — and
include the defaulted names in the list/range duplicate check.
Also document the raw-SQL branch of `resolvePartitionExpression` (identifiers
inside opaque expressions are emitted verbatim) and add an integration test
that exercises partitioning alongside indexes, check constraints, and
triggers.
…ts filter The `pg_inherits` anti-join in `getListTablesSQL` was rewritten to compare on (schema, table) pairs so cross-schema partitions get filtered correctly even when their schema isn't on the session `search_path`. The Migrator test's expected query string was still on the old `table_name not in (...)` form.
…verage Closes the remaining coverage gaps on the new partition helpers — quoted identifiers with embedded "" escape sequences, and the empty/malformed definition fallbacks in normalizePartitionDefinition and toEntityPartitionBy.
…branches Pins coverage on the last four branches codecov flagged: hash partition names with more than one dot, partition keys that resolve to multi-column properties, MetadataDiscovery surfacing the unsupported-platform error through MikroORM.init, and the entity generator's refusal to emit callback-form partitionBy.expression.
The `connect: false` flag is honored at runtime but isn't part of the driver's Options type; cast through the driver's Options to keep the typecheck green (mirrors advanced-index-features.postgres.test.ts).
…partition-support # Conflicts: # packages/sql/src/schema/DatabaseTable.ts
B4nan
approved these changes
Apr 19, 2026
B4nan
added a commit
that referenced
this pull request
Apr 20, 2026
## Summary Adds PostgreSQL declarative partitioning via `EntityOptions.partitionBy`, extending existing entity options rather than introducing a new decorator (per discussion on #6944). Closes #6944. ## Changes - add `partitionBy` to `EntityOptions` and entity metadata - validate partition metadata during discovery (PK / unique-constraint coverage, inheritance / view / virtual guards) - support `hash`, `list`, and `range` partition definitions with array / comma-separated / raw SQL / callback expressions - generate `partition by …` parent DDL and explicit child partitions in PostgreSQL schema generation - introspect partitioned tables from `pg_partitioned_table` / `pg_inherits` to avoid perpetual diffs - surface partition changes through the schema comparator - emit `partitionBy` in entity-generator output for introspected tables - document the new API in `schema-generator` and `defineEntity` docs ## Example ```ts @entity({ partitionBy: { type: 'hash', expression: ['type'], partitions: 16, }, }) export class Event { @PrimaryKey() type!: string; @PrimaryKey() id!: number; } ``` ## Notes - PostgreSQL-only. Other SQL platforms reject `partitionBy` at discovery with a platform-specific error. - Changing the partition definition of an existing table is not auto-migrated — write a manual migration for repartitioning. - Multi-level (sub-)partitioning is not modelled in declarative metadata and is intentionally invisible to schema diffing. --------- Co-authored-by: Eyüp Can Akman <eyupcanakman@gmail.com> Co-authored-by: Martin Adámek <banan23@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Vasil Rangelov <boen.robot@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds PostgreSQL declarative partitioning via
EntityOptions.partitionBy, extending existing entity options rather than introducing a new decorator (per discussion on #6944).Closes #6944.
Changes
partitionBytoEntityOptionsand entity metadatahash,list, andrangepartition definitions with array / comma-separated / raw SQL / callback expressionspartition by …parent DDL and explicit child partitions in PostgreSQL schema generationpg_partitioned_table/pg_inheritsto avoid perpetual diffspartitionByin entity-generator output for introspected tablesschema-generatoranddefineEntitydocsExample
Notes
partitionByat discovery with a platform-specific error.