Skip to content

Move result projection onto Table (Table::projectAs/list); drop SelectQuery::reshape() - #19484

Draft
dereuromark wants to merge 2 commits into
5.nextfrom
find-honesty-5next
Draft

Move result projection onto Table (Table::projectAs/list); drop SelectQuery::reshape()#19484
dereuromark wants to merge 2 commits into
5.nextfrom
find-honesty-5next

Conversation

@dereuromark

@dereuromark dereuromark commented May 30, 2026

Copy link
Copy Markdown
Member

Draft. Part of the find()-honesty discussion in #19482.

Problem

$x = $table->find('auth')->first() is statically typed as the entity even when a finder reshapes each row via formatResults(..., OVERWRITE). The static type lies at runtime, and the reshape is invisible at the assignment site.

Direction (revised)

Rather than adding result-reshaping methods to SelectQuery, this keeps SelectQuery a query builder (it returns entities or arrays) and puts shape-changing transforms on the Table (repository), where the honest return type can be a concrete array. This follows the same trajectory as disableHydration() -> Table::unhydratedFind()/UnhydratedSelectQuery.

It also avoids the overwrite-formatter stacking limitation: the terminal repository methods consume a query and return an array instead of mutating a chainable formatter.

What this adds

  • Table::projectAs(class-string $class, ?SelectQuery $query = null): array returns a concrete list<T> of DTOs. Delegates to the existing query projection path, so rows are mapped from the raw selected columns (entity visibility/accessors do not affect the DTO data).
  • Table::list(?SelectQuery $query = null, ...): array is the terminal counterpart to the list finder and returns the combined array directly.
  • Removes SelectQuery::reshape() (never released).
  • Soft-deprecates SelectQuery::projectAs() (doc-only) in favor of Table::projectAs().
$dtos = $articles->projectAs(ArticleDto::class);

$options = $articles->list(
    $articles->find()->where(['published' => true]),
    valueField: 'title',
);

Out of scope (need design discussion first, see #19482)

  • A shared bound so createFromArray()-style DTOs are type-tracked too.
  • Removing SelectQuery::formatResults()/projectAs() in 6.0, which needs an internal result-decorator pipeline first (find('list')/find('threaded')/associations still use formatResults() internally).

@dereuromark
dereuromark force-pushed the find-honesty-5next branch from 7e254e3 to 8bd8f36 Compare May 30, 2026 15:04
@dereuromark dereuromark added this to the 5.4.0 milestone May 30, 2026
@dereuromark
dereuromark force-pushed the find-honesty-5next branch from 8bd8f36 to b7ee889 Compare May 30, 2026 15:29
find()->first() is statically typed as the entity even when a formatter
reshapes each row, so reading the result of find(...)->first() gets a
type that lies at runtime and hides the reshape from the call site.

reshape() makes the reshape explicit and rebinds the result generic, so
first()/firstOrFail()/all() resolve to the new shape instead of the
entity. Runtime behavior is identical to formatResults(..., OVERWRITE);
the only addition is the honest static type via a TNew template bound to
EntityInterface or array.

Named reshape() rather than map() to avoid confusion with the collection
map(), which applies per element and returns a collection; this operates
on the whole result set and returns the query.
@dereuromark
dereuromark force-pushed the find-honesty-5next branch from b7ee889 to 31cbdb7 Compare May 30, 2026 16:00
@dereuromark dereuromark changed the title Add SelectQuery::map() for type-tracked result reshaping Add SelectQuery::reshape() for type-tracked result reshaping May 30, 2026
Comment thread src/ORM/Query/SelectQuery.php Outdated
*/
public function reshape(Closure $callback): SelectQuery
{
$this->formatResults($callback, self::OVERWRITE);

@ADmad ADmad May 30, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the overwrite mode means this method can only be used once? Or rather if used multiple times only the last call will work.

This means it can't be used inside custom finders as if you chain finders like $table->find('foo')->find('bar') and both the finders use reshape(), the callback set by findFoo() will get overwritten.

If finder stacking can't be used then I question the need for this method. One can just do $table->find('foo')->find('bar')->all()->somecollectionMethd() instead.

@LordSimal

Copy link
Copy Markdown
Contributor

I am fine with this as it is.

Looking at our docs we only mention formatResults() to be used for adding calculated fields even though it can also be used for other stuff as well.

But I know this is primarily focused on getting the static typing move more forward, so we just have to make sure we document this new method properly so users understand why it exsist, when to use it, and when to user alternatives.

@dereuromark

Copy link
Copy Markdown
Member Author

Based on the open topics above I think we need to first talk more about the topic itself in the parent issue ( #19482 ).

Pivots the find-honesty work from augmenting the query class to terminal
repository methods, per review feedback: SelectQuery stays a query builder
that yields entities or arrays, and shape-changing transforms live on the
Table (repository) where the honest return type can be a concrete array.
This also sidesteps the overwrite-formatter stacking limit of reshape(),
since the terminal methods consume a query and return an array rather than
mutating a chainable formatter.

- Add Table::projectAs(class-string, ?SelectQuery): list of DTOs. Delegates
  to the existing query projection path, so rows are mapped from the raw
  selected columns (entity visibility and accessors do not affect the data).
- Add Table::list(?SelectQuery, ...): terminal counterpart to the list
  finder, returning the combined array directly.
- Remove SelectQuery::reshape() (never released).
- Soft-deprecate SelectQuery::projectAs() in favor of Table::projectAs();
  doc-only for now since it is still used by createFromArray-style DTOs.
@dereuromark

dereuromark commented Jun 14, 2026

Copy link
Copy Markdown
Member Author

Pivoted this PR based on the review feedback.

The objection to reshape() was twofold:

  1. OVERWRITE mode means only the last reshape()/formatter wins, so it cannot be used inside stacked finders (find('foo')->find('bar')).
  2. More fundamentally, augmenting the result set on the query class violates SRP. SelectQuery should stay a query builder that returns entities or arrays; transforms that change the result shape belong on the repository.

This is also the direction the rest of the find-honesty work already took (disableHydration() was deprecated in favor of Table::unhydratedFind()/UnhydratedSelectQuery), so honest-typed reads keep moving onto Table.

What changed here

  • reshape() is removed (it was never released).
  • Table::projectAs(class-string $class, ?SelectQuery $query = null): array returns a concrete list<T> of DTOs. It delegates to the existing query projection path, so rows are mapped from the raw selected columns (entity visibility/accessors do not alter the DTO data). Build and stack finders on $query first, then hand it in; it is terminal, so the stacking limitation does not apply.
  • Table::list(?SelectQuery $query = null, ...): array is the terminal counterpart to the list finder and returns the combined array directly.
  • SelectQuery::projectAs() is soft-deprecated (doc-only) in favor of Table::projectAs().

Open points for discussion

  • Table::projectAs() is type-tracked for constructor-mapped DTOs (the core DtoMapper). The createFromArray() factory style (cakephp-dto) cannot be statically narrowed to T without an interface bound, so those DTOs still work at runtime but stay on the fluent SelectQuery::projectAs() path for now. A shared bound is the remaining decision (same one noted in the original out-of-scope list).
  • Naming: Table::list() - toList() is an alternative if preferred.

@dereuromark dereuromark changed the title Add SelectQuery::reshape() for type-tracked result reshaping Move result projection onto Table (Table::projectAs/list); drop SelectQuery::reshape() Jun 14, 2026
@rochamarcelo

Copy link
Copy Markdown
Contributor

Yield could be good use in Table::projectAs

Comment on lines +1554 to +1557
* @deprecated 5.4.0 Use {@see \Cake\ORM\Table::projectAs()} instead. The
* terminal repository method returns a concrete `list<T>` of DTOs, while
* this fluent toggle keeps the query typed as a collection of entities
* and so lies about the projected result shape. Removed in 6.0.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunate that we're churning a new method. Did we miss something during the design of the feature?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we did the Query part, which now looks fine.
But we didnt tackle the even more problematic formatResults() and result formatting etc, which literally is "mixed" at this point, due to the way we inject them before we execute.
The more correct approach would be to separate those concerns, always return either entity collections or array, and then pass those to the processing that are return type safe then.

I had hoped people would help the last 7 weeks to find a way to way here forward that would minimize the fallout while keeping it clean moving forward.

Comment thread src/ORM/Table.php
$query ??= $this->find();

$results = [];
foreach ($query->projectAs($class)->all() as $dto) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this method deprecated?

@rochamarcelo

Copy link
Copy Markdown
Contributor

What do you think about creating a resolver class to handler projection?

Using DTO class

$resolver = $this->Table->resolver(UserDto::class);
$user = $resolver->first($query);//Return one record using the UserDto, instanceof check
$users = $resolver->all($query);//Return all records using the UserDto, instanceof check
Using entity (second arg = false), dont set dto class in the query, it only perform instanceof check

$resolver = $this->Table->resolver(User::class, false);
$user = $resolver->first($query); //Return one record checking it is User
$users = $resolver->all($query); //Return all records checking they are User

@rochamarcelo

Copy link
Copy Markdown
Contributor

Maybe have resolver at query level

$resolver = $query->resolver(UserDto::class);//Return a resolver classs instead of Query
$user = $resolver->first();//Return one record using the UserDto, instanceof check
$users = $resolver->all();//Return all records using the UserDto, instanceof check

@dereuromark

Copy link
Copy Markdown
Member Author

Good ideas in general.

The part I want to take: there is currently no honest-typed single result. projectAs() only gives list<T>, so getting one row means $table->projectAs(UserDto::class, $query)[0] ?? null, which is clunky and materializes the whole set to take one element. That gap is worth closing, and I will add a terminal counterpart:

/**
 * @template T of object
 * @param class-string<T> $class
 * @return T|null
 */
public function projectOne(string $class, ?SelectQuery $query = null): ?object

So the pair becomes projectAs() -> list<T> and projectOne() -> T|null.

Where I would not go is the resolver object itself, for a few reasons:

It carries no state. With the query passed per call (first($query) / all($query)), the resolver holds exactly one field, the class name. Two methods on Table give the same ergonomics and the same static typing, with no new class in the public API. A template parameter on a plain method narrows T just as well as one carried on an object, so the object is not buying us anything on the PHPStan side. If the query moved into the constructor it would at least be coherent state, but then the resolver is per-query and the reuse argument disappears.

The entity mode duplicates what already works. $table->find()->first() is already statically typed as the entity through the table's own generic, so resolver(User::class, false) is a second way to spell an existing, already-honest call. Adding a parallel path is the API growth that #19482 is trying to reduce, not an example of it. And the runtime instanceof check there is defensive against our own ORM: if find() can hand back something that is not a User, that is a core bug to fix, not something call sites should be guarding.

The query-level variant walks back the pivot. The reason reshape() was dropped was to get shape-changing transforms off SelectQuery and onto the repository. $query->resolver(UserDto::class) puts the entry point back on the query. It is genuinely better than reshape() was, since it returns a separate object instead of mutating a chainable formatter, so it does not lie about the type of $query itself. But it re-splits projection across Table and SelectQuery, which is exactly what the pivot merged. If a resolver ever does land, it should be on Table only.

Also, minor naming point: "resolver" is fairly loaded in PHP already (DI containers, GraphQL). If we ever do want a dedicated object, something like Projection would read closer to what it does.

On your earlier point about yield: that is a real tension I should address in the PR rather than leave hanging. list<T> is honest but eager, so projecting a very large result set materializes all of it. A Generator<T> return is equally honest statically and stays lazy, but the two cannot live on one method signature. My inclination is to keep the terminal methods eager, since they line up with the array-returning list() counterpart and most call sites are small result sets, and add a streaming variant later if there is demand for it. Happy to be argued out of that.

One more thing your comments surfaced indirectly: the current Table::projectAs() implementation filters with instanceof inside the loop, which silently drops any row that does not map. That is bad behavior regardless of which API shape we settle on, so I will make it throw instead of quietly returning a short list.

@rochamarcelo

Copy link
Copy Markdown
Contributor

The idea for resolver class was to separete specific methods from Table class in a dedicated class, in case we need more methods in the future (generator and array related); but this could be too much now for little benefit.

The addition of projectOne is good, I also aggree with throwing exceptions.

@markstory markstory modified the milestones: 5.4.0, 5.5.0 Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants