python-stdlib: Add typing modules that can be frozen or mip installable.#1051
python-stdlib: Add typing modules that can be frozen or mip installable.#1051Josverl wants to merge 5 commits into
Conversation
2bab830 to
6868a06
Compare
|
@stinos, could you please review ? |
|
To make comparisons easier, could you share gcc/micropython version and a script which you used, or even just a set of commands, to get these results? Having rebased my code on master and comparing sizes and results, things seem different for me so I want to make sure we're doing roughly the same thing; for instance:
In any case, I'm hitting something funny; I made a windows msvc build with your typing bundle, I can see frozen_content.c including the frozen typing code code, yet I get an ImportError trying to import any of the modules contained in the manifest. Probably I'm gonna try and sort that out first. |
|
@stinos , https://gist.github.com/Josverl/f953fb49a4cac63759bd2056f728a725#file-readme-md I built locally on WSL Ubuntu 22.04.5 LTS with :
|
Oops, my MICROPYPATH didn't have a .frozen entry.
Ok thanks. I was doing almost the same, however using multiple variants in different branches where I might have messed something up at the time I wrote the previous comment, and using gcc 9.4 or so. Will check again next week. Not sure if this is of any use, but here's a PS script for comparing typing variants: https://gist.github.com/stinos/2ee6138679a39a75628f85b5765a71b1 |
| extras.append("expected failures=%d" % res.expectedFailuresNum) | ||
| if res.unexpectedSuccessesNum > 0: | ||
| extras.append("unexpected successes=%d" % res.unexpectedSuccessesNum) | ||
| if ( |
There was a problem hiding this comment.
if (res.failuresNum + res.errorsNum + res.unexpectedSuccessesNum) > 0: saves 9 bytes here, although it won't work correctly if any of those values ends up being negative.
There was a problem hiding this comment.
In my tests your optimization works, and I think the counts will never will be negative.
the changes to unittest are a spin-off for a make_report.py ( need marketing advise here) tool that runs part of the test suite across multiple variants , and provides a more readable format of the results and the firmware sizes.
report.md
I think that make_report , and the these changes to unittest may be useful to compare other implementations ( such as enum) and these should that should be its own PR(s). I kept it in here for now to avoid needing to manage even more git worktrees
it is still WIP, also as the xfail/xpass Pytest concepts don't fit unittest.
20e3a20 to
c5c42c0
Compare
typing, typing_extensions and collections.abc Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
`__future__`, `abc`, `collection` Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
This is needed to access the the "__future__" package. Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
The could be submitted as a separate PR, but is included here to allow better reporting of the tests results of different typing variants. Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
|
I think this PR is now feature complete. (As it has changed significantly - I have updated the initial PR comment.) Now it comes to the real decision: which of the various alternatives is the best- and by what measure. Clearly I like this PR, but even if this would be merged that still leaves the question of which typing modules to freeze, or leave them all to be installed on demand. To help make that decision there is this summary, and a more complete report is linked below.
Legend:
|
As this PR have changed very significantly I have completely rewritten the PR Description.
The old description can be found at the end.
This is a MicroPython alternative to:
based on all the above and tuned for reduced firmware/frozen size, while keeping provable consistence with the relevant typing PEPs for features that are supported by MicroPython.
It consists of a number of modules that can be mip-installed or frozen as part of a firmware..
Note that the modules have been optimized for size - somewhat impacting readability - but where relevant the orignal code has been kept as comments.
For other related changes see later in this section.
Fixes: #190
Closes: micropython/micropython#15911
Closes: #584
I want to explicitly recognize that this work is in part based on and inspired the work by in PRs @stinos and @andrewleech and other individuals that have directly contributed to the typing stubs on the MicroPython-Stubs repo.
Summary of the modules:
__future__: contains theannotationsfeature, which allows for postponed evaluationtyping: contains the core typing features, such asTypeVar,Generic,Protocol,Union,Optional,Callableand more.typing_extensions: features that are/were not yet part of the standard library for Python 3.4/3.5.collections.abc: contains the abstract base classes for collections, such asIterable,Sequence,Mappingand more.abc: contains theABCandabstractmethodfeatures, while they are deprecated in Python 3.11, they are still relevant.Implementation details:
The
__future__already existed as a basic list of assignments, and has been extended with only theannotationsfeature.In order to save space in the firmware or on the file system the
typingmodule uses a module level__getattr__to lazily load the features of the module, rather than implement each typing class, method or decorator in detail. Care has been taken to match the runtime signatures of the CPython/Micropythn runtime , and reduce runtime overhead as much as possible.The
typing_extensions,abcandcollections.abcmodules re-use the same__getattr__from thetypingmodule and also include only limited implementations, again to save space.each module can be frozen or installed individually, it will automatically pull in any required dependencies.
In order to further simplify installation or freezing a
bundle-typinghas been created that contains:__future__typingtyping_extensionsAnd optionally ( extensions=true ) also:
abccollections.abcThe bundle also set the
opt_levelto 3, which will further reduce the size of the modules, but may impact readability of tracebacks.Install from this PR:
The modules can be installed from this PR using the following command:
( Do try this at home :-)
Testing:
I have spent perhaps more time on creating the tests than on the implementation, as I wanted to make sure that the implementation is consistent with the relevant PEPs and also to have a good coverage of the implemented features.
Currently there are 187 typing runtime tests, which are based on:
test_typingandtest_typing_extensions.Tests and test results
micropython:tests/typing_runtimeunittesthas been added tomicropython-lib:python-stdlib/unittest/testsTest matrix
in order to compare the different optionas for implementation on both size and functionaly the followin variants have been tested:
require("typing")from this PR, which is optimized for size and uses lazy loading.require("bundle-typing")which includes the core typing features and is optimized for size.require("bundle-typing", extensions=True)with which includes all typing modules and is optimized for size.mip installall the typing modules.require("bundle-typing", extensions=True).mpremote mip install <all typing modules>Summary of test results
Legend:
size <binary>.typing_runtime/check_all_modules.py
See the full report.md
PR and merge dependencies
See the companion PR: micropython/micropython#19310
I'm open to recommendation on how the different parts needed to implement and test this functionality should be seperated into different PRs, across the different repos.
Apart from this PR there are at the following components needed:
unittest@0.11.0In order to be able to create tests, and clear reporting across the alternative implementations, I needed the test suite to be able to make use
unittest.expectedFailure, and forunittestto report the Expected Failures and unexpected successes to therun-test.pytest runner. This allows to distinguish on things that are not implemented in MicroPython, but are expected to fail ( User defined Generics[], MetaClass), from things that are implemented but not working as expected.micropython:tests/run-test.pyIn turn, the
run-test.pytest runner needed to be updated to handle the additional information.micropython:tools/make_report.pyIn order to keep an overview of all test results I have created a new tool
make_report.pythat builds a markdown report of the test results, which can be used to compare the results of different implementations and variants. This tool could also be used to compare other sets of PRs that implement the same features.Example report runner
Trade-offs and Alternatives
Size versus accuracy:
The implementation is optimized for size, which may impact readability and traceability of errors.
Also due to the use of
__getattr__for lazy loading, some features may appear to be available, while they are actually not.For example:
from typing import NoSuchThingwill not raise anImportErrorat runtime.import typing; dir(typing)will not work as expected but show a list of all QSTR's in the firmware.This is deemed acceptable as static type checkers and linter will catch this at development time.
Size versus composability:
While the freeezing of all .py files is larger than a fuctionally equivalant C implementation, it has the advanatge that is can be
mip installedand also easily extended with new features, without the need to recompile the firmware.or any median can be accomplished by freezing only the core
bundle-typingandmip installingother modules as appropriate.vfs size impact:
Freezing methods:
durign the quest to save firmware size, I have experimented with different freezing method. I noticed in particular that the
__future__made the fimware size increase by more that its .py file size. and could actually be stored smaller usingfreeze_as_str()at the cost of additional runtime overhead while loading. This was explained in DiscussionThis PR currently uses the
require()method for all modules,to reduce or avoid memory fragmentation that is likely caused by compilation at runtime.unittest@0.11.0andtools/run-tests.py:In order to be able to create tests for the typing modules, and to have clear reporting of expected failures and unexpected successes, I have added CPython compatible functionality to
unittestThat was defintly needed in order to develop this PR, but just guarding against regressiosn could be done using
unittest@0.10.5, but that would require either removing.chaining the expected failure tests, or putting them behind a version check, which would make the tests less clear and more difficult to maintain.The update to
tools/run-tests.pyis minor, and adds just the abaility to parse the additional information, and is backward compatible withunittest@0.10.5.There is however no way for (a) unittest to check by which version of
tools/run-tests.pyit is being run.Generative AI
I used generative AI tools when creating this PR, but a human has checked the
code and is responsible for the code and the description above.
Outdated Original PR Summary
This is a improved MicroPython alternative to
based on all the above and tuned for reduced firmware/frozen size, while keeping provable consistence with the relevant typing PEPs for features that are supported by MicroPython.
It consists of a number of modules that can be mip-installed or frozen as part of a firmware..
Note that the modules have been optimized for size - somewhat impacting readability - but where relevant the orignal code has been kept as comments.
cloases: #190
Documentation
Documentation is not created yet.
Testing:
The same set of tests has been run against both implementations and while there are a few differences , both pass the majority of tests.
The remaining differences to CPython should be documented as such, and then can be excluded from the verification tests.
The tests are created at part of MicroPython's tests suite , and are mot included in this PR.
Currently they are part of micropython/micropython#15911, but could be separated for testing ( may require a new variant for testing in this in the MicroPython repo)
The PR includes the
bundle-typingto allow simple addition to a board defintitionThis also defaults the mpy-cross opt level to 3, to reduce firmware size.
The impact to firmware size of the
modtyping.cand thetyping.pyvariant are surprisingly close,although I think there may be room for additional optimisation for the
modtyping.calternative.Comparison without collections.abc
Note: both PRs are changing , so the below is a point-in-time comparison ( 14/10/'25)
Test Results:
Comparison including collections.abc
Test results: