extmod, ports: add new vfs_rom_ioctl code to support partial prepare/erase, and implement on alif, samd, stm32#19348
Conversation
This allows devices to support partial update of the ROM segments. Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
|
@kwagyeman FYI this might be useful to you. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #19348 +/- ##
=======================================
Coverage 98.51% 98.51%
=======================================
Files 176 176
Lines 22903 22903
=======================================
Hits 22562 22562
Misses 341 341 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Code size report: |
This uses the new GET_MIN_PREPARE ioctl to support incremental preparation of the ROM partition. On very large partitions the preparation can take a long time (eg due to the erase time of many blocks) and this can lead to a timeout of mpremote. Using incremental preparation prevents the timeout and gives better feedback to the user as to the progress. Signed-off-by: Damien George <damien@micropython.org>
b07275b to
720f797
Compare
|
@dpgeorge Can you add the mimxrt too? |
The mimxrt port (and others like rp2, esp32) do not need this addition, because they return a block device from the The changes in this PR are only needed for ports that do not expose a block device for the ROMFS. |
|
Mmm, would be nice to have a unified api here. |
That's not easy to implement. Eg on STM32 parts with ROMFS in internal flash, the page erase size is sometimes very large (eg 256kb on F767) and it's not feasible to expose that as a block device object (because you can't write 256kb at once so it'd need to support partial writes and that makes it quite complicated). So the way it's designed is:
In the end it's not that hard to support the different ways that it works, especially writing it in Python is pretty straightforward. I can add some docs for that (see also #19127). If we really need to have a unified API for this... then it requires a redesign of the whole interface to come up with something that will suit all the ports and all the different ways of having writable ROM. (Note that the situation in master already has this non-unified API. This PR does not change that but rather extends the |
|
Well, I can just use exceptions on returned types and name errors to have one piece of code handle everything, so, I can do that to handle everything. |
|
I did some more thinking about this, and maybe it does need a bit of an overhaul to unify it, and also provide information about non-memory-mappable flash partitions that are used for filesystem only. |
Summary
At the moment, ROMFS on some ports (eg alif, samd, stm32) supports only a single large partition that must be prepared/erase in one go. This PR enhances these ports so that their ROMFS partition can be prepared/erased in blocks (down to the erase size of the underlying flash). This has two advantages:
mpremotewhen deploying very large (>8MB) ROM filesystems. This timeout would occur if the prepare stage took > 10 seconds. But now it's split up into many smaller prepare calls, each taking less than 1 second, so does not timeout anymore.Changes in this PR:
mpremoteto use the new ioctls when they are supported by a targetThe change is fully backwards compatible with boards that don't support the newer ioctls.
Testing
Tested on ALIF_ENSEMBLE, ADAFRUIT_ITSYBITSY_M0_EXPRESS, RPI_PICO2 and PYBD_SF6:
mpremotewith old firmware (before these changes)romfs queryandromfs deploympremotewith new firmwareTrade-offs and Alternatives
This adds a small amount of extra code to ports that use the new features. But ports that don't have the space do not have to implement it.
The impact to mpremote is minimal (some code was refactored there).
Generative AI
Not used.