Skip to content

optimize getting of line number in pytest discovery#26017

Open
vaclavHala wants to merge 1 commit into
microsoft:mainfrom
vaclavHala:optimize-pytest-discovery-lineno
Open

optimize getting of line number in pytest discovery#26017
vaclavHala wants to merge 1 commit into
microsoft:mainfrom
vaclavHala:optimize-pytest-discovery-lineno

Conversation

@vaclavHala

Copy link
Copy Markdown

The discovery for pytest is still quite a bit slower than running plain -m pytest --collect-only so I investigated a bit more and the next most significant (at least for our test suite) slowdown is caused by the pytest.Item.location call used to get line number of the test:

profile_location

This is because the .location returns tuple of path and line number, of which only the line number is used by the discovery, but computing the path is rather expensive. This PR proposes an optimization which uses internal pytest API in exactly the same way as the location function itself, but without also computing the path.

In particular, the location call hierarchy is:

This PR calls Code.from_function directly and only gets .firstlineno from it.

profile_firstlineno

Both profiles are captured with this PR cherry-picked to main branch prior to the discovery result compaction introduced in 5389068 which causes significant performance hit that overshadows the optimization done here, for that I created a separate PR #26016 with proposed fix.

This change makes the discovery about 20% faster for our suite, now almost on par with the raw pytest call

…xpensive call before falling back to location query
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant