Skip to content

perf(sql): resolve a joined relation's mapped columns once per query - #8465

Open
ybai08 wants to merge 1 commit into
mikro-orm:masterfrom
ybai08:perf/8460-joined-partial-hydration
Open

ybai08 wants to merge 1 commit into
mikro-orm:masterfrom
ybai08:perf/8460-joined-partial-hydration

Conversation

@ybai08

@ybai08 ybai08 commented Oct 17, 2026

Copy link
Copy Markdown

Refs #8460. This is the first of the two paths in the issue: the per-row work in AbstractSqlDriver.mapJoinedProps. The second, EntityFactory.mergeData re-diffing an identity-map entity on every row, touches the merge rules for user-changed values. I'll send it as a separate PR, so this one stays reviewable on its own.

For every row of a joined result, mapJoinedProps filtered all of the target's properties (shouldHaveColumn), filtered them again for the non-persistent ones, and called mapJoinedProp for each. That call builds `${relationAlias}__${name}` and finds no such column for anything outside fields, and the delete loop then visits the same props. With fields: ['to.pid'] on a wide target, nearly all of that is spent on columns that were never selected.

Every row of one result has the same columns. So on the first row of each join path, this resolves the target props that have a column in the row (typeof root[alias] !== 'undefined', the same test mapJoinedProp uses to skip a prop), plus the non-persistent ones. Both lists are kept in the per-query map that QueryBuilder already passes to mapResult, keyed by the join path, and reused for the remaining rows. A row whose relation is null still has the columns (with null values), so it gives the same list. Callers that don't pass a map get a fresh one per row, as before, so nothing is shared beyond one result.

Numbers

These come from the issue's reproduction (SQLite in memory, 5,000 edges to 500 targets, find(Edge, {}, { populate: ['to'], fields: ['to.pid'], strategy: 'joined' })) on macOS arm64 with Node 26. Each figure is the median of 15 runs after 3 warm-ups. I ran master and this branch alternately, twice each:

Target columns master this PR
2 25.0 / 25.4 ms 24.7 / 24.1 ms
102 91.7 / 91.1 ms 47.6 / 46.4 ms
202 172.0 / 171.5 ms 79.5 / 79.8 ms

The remaining growth with column count is the mergeData path.

Tests

tests/issues/GH8460.test.ts (SQLite), three tests:

  • A partial joined to with fields: ['to.pid'] over 5 rows. The first row's relation is null, so the list is resolved from an all-null row. pid is right on every row, and the unselected columns stay unloaded.
  • Two join paths to the same entity with different fields (to.pid and from.col1, plus from.owner.name). Each path gets its own columns, and the populate takes one query (mockLogger).
  • A fully loaded to.owner keeps every column.

They pass on master too, since this doesn't change behaviour. They guard the cache. Each mutation below was run against GH8460.test.ts and the SQLite partial-loading/GH* tests (14 tests, all passing unmutated):

Mutation Result
One cache entry for all join paths 3 tests in 2 files fail
Presence check inverted 7 tests in 3 files fail
Cache keyed by the target entity instead of the path 1 test fails. The entity loader then reloads what wasn't mapped, so the values come out right, but the single-query assertion fails

One mutation fails nothing: dropping the non-persistent-props loop. It fails no test here, nor in EntityManager.pglite.test.ts, GH7023 or the polymorphic SQLite tests, so that branch (from the :ref refactor) has no test on master either. The list itself is unchanged, only computed once.

What I ran:

  • The whole suite (yarn vitest run) on this branch and on master 733c05d3, against the repo's docker-compose Postgres, MySQL 9 and MariaDB, with JSON reports. The set of failing tests is identical on both: nothing fails only on this branch, and nothing only on master. About half of the failing files are the MongoDB, MSSQL and Oracle suites, which I don't have running. I haven't set up the /etc/hosts entries either, and I didn't go through the remaining shared failures one by one.
  • yarn build in packages/sql, and yarn lint (oxlint with type-aware rules and tsgo): clean.

mapJoinedProps filtered every target property for every row, then built
and checked an aliased key for each one, so hydrating a partially loaded
joined relation cost time proportional to the target entity's column
count instead of the selected columns. The columns are the same on every
row, so resolve the target props that have one, and the non-persistent
ones, on the first row of each join path and reuse them.

Refs mikro-orm#8460

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant