Feature request
I'd like to simulate git bisect command in PHPStan. Because PHPStan is distributed as a PHAR and always like that, typical projects are not compatible with being analysed by phpstan-src for numerous reasons.
If there's a regression, hard to debug regression, between two releases, and the regression is only reproducible in complex scenarios with project configuration, it's hard to pin down the "first bad commit" and we only have the "good" release version and the "bad" release version.
I'd like this bisect command to:
- be an interactive command that asks for a "good" release and a "bad" release
- it also asks for a single file to analyse (optional). If not provided, it will run on specified paths, or on the paths in config
- support all parameters that
analyse command accepts, so that it can pass them to the analyse command
- Query GitHub API with a token found in
~/.composer/auth.json 🙈 Query the phpstan/phpstan repository for a commit list between two releases
- Do a binary search like
git bisect does, looking for first offending commit where a regression appears
- For each evaluated commit during the binary search in phpstan/phpstan, it downloads
phpstan.phar file with GitHub API in an efficient way
- It then runs this downloaded phpstan.phar and reports results, and ask the user whether this is good or bad
- It then continues the binary search in the right direction based on the answer
- In the end we have a first
phpstan/phpstan commit where the regression appears, but it might actually be a result of a push of more commits. This is readable from Git history (Again, ask GitHub API, do not clone phpstan/phpstan). The long commit description of a commit in phpstan/phpstan contains the list of pushed commits to phpstan-src:
https://github.com/phpstan/phpstan-src/commit/b4c5164f9804ac18ce7862760694bbd22c28b729 Update BetterReflection
https://github.com/phpstan/phpstan-src/commit/edd5f9f1537a7245f8a66ac1dee554bc67c609d8 Merge branch 2.1.x into 2.2.x
- Once the first bad commit is found, print the commit in phpstan/phpstan alongside this long description
Verify the behaviour of this command by looking at release notes https://github.com/phpstan/phpstan/releases, pick a "bugfixed" issue, download the code snippet for this issue and verify that the bisect command finds the right commit that changed the behaviour for this snippet by running it with "old" (previous release) and "new" (the release that fixed it). You might need to inverse the "good" and "bad" meaning for this test.
Did PHPStan help you today? Did it make you happy in any way?
No response
Feature request
I'd like to simulate
git bisectcommand in PHPStan. Because PHPStan is distributed as a PHAR and always like that, typical projects are not compatible with being analysed by phpstan-src for numerous reasons.If there's a regression, hard to debug regression, between two releases, and the regression is only reproducible in complex scenarios with project configuration, it's hard to pin down the "first bad commit" and we only have the "good" release version and the "bad" release version.
I'd like this bisect command to:
analysecommand accepts, so that it can pass them to theanalysecommand~/.composer/auth.json🙈 Query thephpstan/phpstanrepository for a commit list between two releasesgit bisectdoes, looking for first offending commit where a regression appearsphpstan.pharfile with GitHub API in an efficient wayphpstan/phpstancommit where the regression appears, but it might actually be a result of a push of more commits. This is readable from Git history (Again, ask GitHub API, do not clone phpstan/phpstan). The long commit description of a commit inphpstan/phpstancontains the list of pushed commits to phpstan-src:Verify the behaviour of this command by looking at release notes https://github.com/phpstan/phpstan/releases, pick a "bugfixed" issue, download the code snippet for this issue and verify that the
bisectcommand finds the right commit that changed the behaviour for this snippet by running it with "old" (previous release) and "new" (the release that fixed it). You might need to inverse the "good" and "bad" meaning for this test.Did PHPStan help you today? Did it make you happy in any way?
No response