Summary

Generated qstr.i.last files (output from the "pp" phase) can be very large due to inclusion of large HAL headers. Yet only a small amount of this data is actually needed by the "split" phase of makeqstrdefs.py.

This commit improves the situation by filtering out unneeded lines as early as possible: in the "pp" phase the C preprocessor runs, its output is parsed via a pipe connection, and only those lines that are needed are written out to the qstr.i.last file. When the "split" phase runs it then sees only the necessary lines (it does further regex filtering to get exactly the lines it needs for the given mode).

Windows already does this filtering (it does not use makeqstrdefs.py for the preprocessor phase) in its ConcatPreProcFiles function, and that was added long ago in 29c8c8a.

This change significantly reduces the output build size and improves the build speed:

• stm32 BOARD=PYBV10: build output goes from 152M down to 32M, and build is 10% faster (29s down to 26s). Building all stm32 boards, the total build size goes from about 11GB down to about 2.5GB.

• rp2 BOARD=RPI_PICO: build output goes from 87M down to 39M, and build is about 5% faster (23s down to 21.6s).

• esp32 BOARD=ESP32_GENERIC: build output goes from 307M down to 230M, and build is about 6% faster (101s down to 95s).

Testing

Tested all three ports above, that the files in build/genhdr are equivalent (except of course qstr.i.last), and that PYBV10 is binary equivalent before and after this change.

Trade-offs and Alternatives

Tried using gzip in #19312, but the approach here is far better.

Now need to keep the regex's in sync, between the "pp" and "split" phases.

Generative AI

Not used.