Added errno attribute to OSError class.#2407
Conversation
|
It's neat, but did you find a particular need for it? Is there code out there that uses exc.errno instead of exc.args[0]? Well, I guess that there's one main thing this adds that exc.args[0] doesn't offer: if there are no arguments then exc.args[0] raises IndexError, whereas exc.errno just returns None. |
|
I had this problem when I was writing the tests for |
|
Grepping through my src directory I found several instances of the "except OSError as e: if e.errno != errno.ESOMETHING:raise" idiom (binwalk, apm_planner, openscad, libffi, cython, sel4). I think it's a common one in CPython. I first encountered it in some code I wrote myself. |
|
I'm not sure this change adheres to criteria outlined in https://github.com/micropython/micropython/wiki/ContributorGuidelines . |
|
@pfalcon this is like saying "there is a typo in your two-page article". Not very helpful. Would you mind telling us explicitly what needs to be corrected, instead of making us guess what you mean exactly? |
|
@pfalcon I agree it doesn't meet the criteria, but I think it's still a worthwhile investment. It makes porting a lot of CPython code easier, at the cost of two comparisons, branches and a function call. |
There's still a thousand of features which are missing which make porting of CPython code hard. Each of them is approximately few branches, few function calls (plus/minus, you know). Now, the link above comes into play to not repeat same old things over and over again: "We could spend years of development and implement everything what CPython can do, just in 50% of size." Spend years to implement those 1000 x "few branches, few function calls" things, hope it's clear. But: "If you want all CPython functionality, you can use CPython right now." And: "What we try to achieve with MicroPython is being order(s) of magnitude smaller than CPython, to use Python (the language) where CPython could not be used at all." There're many more arguments in that very detailed docs. Highly recommended to read to understand "MicroPython zen". |
|
Now, why StopIteration.value was implemented: because it's part of the language. OSError is however part of the library. MicroPython's doesn't implement CPython's standard library, only subset of it. In particular, MicroPython's OSError doesn't have .errno, but uses generic .args attribute. That way is also compatible with CPython. |
|
I was pretty sure that |
|
The line is fine, for sure. There're core language features, like iteration and generators, and StopIteration is part of implementation of these features. Everything else is another level. In particular, any I/O support is a separate level. First of all, it's possible to have no I/O at all. Secondly, I/O implementation may be different (not using open(), not using OSError). In all these cases it may be stil possible to do useful things, using just core language features. |
|
It's also perfectly possible to have no iterators (older python versions didn't have them, while they did have open() and OSError). The behavior of "open", "print" etc. is actually specified in core python documentation, and is officially part of the language, just as much as iterators are. Of course MicroPython is not a full implementation of the Python language, and a lot of compromises has been made. But I expect that each difference there is from the way Python language is defined is there for a good reason, either because it's faster, uses less ram at runtime or simply because it makes more sense in the constrained environment. What is the specific reason for adding this particular difference to the list of incompatibilities with Python? |
No, it's reversed logic. Any addition to MicroPython core should have very good reasons, and generally, the more features have been added, the harder (even more, of even better reasons) it should be to add more. Well, that's how I see it. Once again, the document linked above explains in detail ideas behind this. |
|
I think that your logic is reversed, and it doesn't much matter how many documents on the wiki you write about it. I just checked, out of curiosity, what is the price we are paying for the ability to know, in a Python-compatible way, what OSError was actually thrown. It adds 24 bytes to the 530+kB binary. To put that in perspective, I compared that against a feature you recently added to uzlib (definitely not part of the core python language specification) 1bc5cb4 and it adds 76 bytes. Was that justified? Why didn't you care about it then, but care about it now? |
That document was written by me, relatively long ago, based on FAQs asked, and based on my even longer contribution to MicroPython. It was reviewed by the MicroPython author and other contributors. It depicts requirements based on which MicroPython has been developed so far (2.5yrs). So unless you hear something new from MicroPython author, newcomers to uPy development, like you, are advised to treat it as the official project documentation, intended to get people like you right with uPy development.
That's exactly what I'm talking about - uPy needs builtin package manager to unleash its full power, and there's no way to make builtin package manager work on low-heap systems without that change. However, accessing OSError's errno is as simple as |
|
See also #2414 |
Right. And an ad-hoc implementation of a database. It's crucial, because every single microcontroller platform has to have a database and a way of downloading code from the Internet. I think that your longer contribution might have skewed a little bit your opinions about what makes a project successful.
This is far from simple. To figure this out, you would need to look into the C code and analyze it. Nobody in the Python world does it this way, the Python documentation does it differently, and there is no MicroPython documentation telling you about this. You are the only person who thinks it's simple, because you wrote it. To unleash its full power, MicroPython needs a lively and thriving community of users and contributors. For that, it has to be friendly to newcomers and easy to learn and use. That is how the original Python became successful, anyways. I am working hard on bringing as many users as possible into this great project. You seem to be working hard on becoming the only user and contributor left (though I do value your contributions and all the energy and time you are putting into this project, not all of us have the skill and time to devote like that). #2414 seems to like saying "ok, I had enough fun here, now that other people are joining development, let's lock it all up". |
|
The contributor document etc are not set in stone. They are guidelines and should be treated as such: guides not rules. In particular they should evolve over time to keep up to date with how uPy is progressing as a whole. It's becoming clear that uPy's distinguishing feature from CPython is low RAM usage, ability to run easily on bare-metal with no OS, and the fact that it's actually ported to various MCUs with decent support for their peripherals. stmhal and esp8266's firmware are definitely more than 50% made of peripheral drivers and bindings, so adding stuff to the core language, especially features that just take flash not RAM, is really a non-issue for these ports. And there is another big thing to consider when adding features (or not): the users. uPy was at the beginning a technical thing for me (and perhaps others), but now that focus has shifted. There are so many people using it, and some relying on it for their daily work, that users are now a very important part of the project, and we must weight up decisions with users in mind. I'm +0.5 on merging this. If a user came across it then most likely it's useful. It doesn't add anything to RAM usage (and may decrease it in some cases because errno is now interned to flash). It could be added with MICROPY_CPYTHON_COMPAT if that's a better option. |
|
How about guarding it with MICROPY_PY_UERRNO. Having errno only makes sense if you also have those constants. |
This is Nth time you show your ignorance of Python. Definitely not 1st, not 2nd, and not 3rd. Each time this happens (you talk nonsense, or give wrong advice to someone else), you're patiently and humbly hinted that you knowledge is far from complete and there're lot to learn. Time after time, you ignore any hints, lessons, and don't show any desire to learn. Only, based on your own words, desire to argue for purpose of argument and just flame. Actually, your behavior passed a line between "ignorance" and "arrogance" long ago. So, dude, do RTFM one of these days, then every other day too. For how exception arguments are handled, you do https://docs.python.org/3/tutorial/errors.html#handling-exceptions and https://docs.python.org/3/library/exceptions.html#BaseException . But don't stop there - Python has one of the greatest documentation out there, you should read it all. Then, you will be able to participate in discussion, not flames.
Exactly, to be real-world useful, for MicroPython it's not enough to have a solid core (which it already has), it should offer "full stack" application development model. That concept was proposed to community via the latest Kickstarter campaign, was approved, and now being worked on.
Either that, or you're an arrogant new kid on the block making too much noise and wasting people's time.
Dude, you must be kidding everyone. What dawned on you - that for a project to be successful and useful it needs confirmed userbase - was crystal clear to others years ago. Your "hard working" is little comparing to what other people have been doing for years. Get humbler, Mr. Arrogant. If you find this project great, figure out that it is so because other people worked on it before. Learn what they did and how. But first learn Python, as pointed above. Then keep working hard. And make sure the project stays great after you applied your hands to it. As for "friendly", there're different concepts of that. One can be friendly by promising everyone 5 minutes of fame by letting copy-pasting 2 lines and merge that. One can be friendly by providing documentation of a project paradigm and what constitutes a good and useful contribution - to avoid wasting hours, days, weeks, months of many peoples' month.
No, arrogant dude, there were always people who contributed to MicroPython. When maintainers' efforts to bring more people and contributors gave visible results, and there were wide spectrum of contributions, 1.5 years ago, https://github.com/micropython/micropython/wiki/ContributorGuidelines doc was written to acquaint newcomers with best practices accumulated over previous year of development and contributions. Yes, process-wise, it was written by a single person as an RFC, was reviewed over reasonable time by the BDFL and other contributors, no big concerns were raised, at this time, it's official guidelines to follow. Now, in 1.5 years, there's an obvious need to make next step - formalize criteria and verification of them. Or do some other changes based on maintainers decision. Anyway, as I already told, a lot of people contribute to MicroPython, and of course, as the number grows, there're marginal cases appear. You can't have mass project (event, etc.) and have everyone happy. Some people will be unhappy just from the feeling of contradiction. @deshipu, you're the lost cause for me. For last couple of months, you were in "soft ignore" mode for me, I tried to communicate with you only on technical matters. But you indeed have those flaming/arrogance traits which make productive communication with you hard, so you go in my permanent ignore list, I'm sorry. |
Ok, then we should discuss how they should progress/changed. As you figured, I think that they should be tightened/formalized, to avoid dilution of the project aims.
I'd say, that's too lowy aims. I always treated (and worked) on making MicroPython have the best functionality/size ratio. That includes ROM size, ability to be ported to as many MCUs as possible (not just "various MCUs"), etc. And of course, competitiveness with other projects, which is hard an aim on its own.
Of course, it's all about users. And that's what I call for - to not dilute, shift or twist the original project traits and requirements which won so many users. It's very hard and non-rewarding task, because each of these so many people would like to add a feature or two just for oneself, and that's quite understood (I, working with MicroPython daily and for long time, would like to add few dozens). But if everyone of so many will add a feature or two, there will be a monster, or just CPython no.2. It's maintainers' task to set direction of the project and guard it. The idea is that in the long run, many more people will appreciate firm and clear paradigm rather than not being of copy of something which existed all the time.
That's rather non-monotonic logic. I'm a user too, why my patch to add embedding support waited for a year? Why, while being blackmailed by upstream with "trademarks" in #2180, you bend to them instead of supporting innovation? Heck, I don't want to do that for myself - it's for users (not "a user") who experience memory problem. I'm literally waiting if George Robotics will change project name to avoid somebody's trademarks, or if I'll have to fork it to implement them. Oh, btw, I want uPy to make coffee for me. Straight in the core. make_coffee() builtin. (I mean real coffee.) So, sorry, but "if a user came across it" it doesn't mean much more than a user came across it, and found that it can be done by copy-pasting 2 lines (kudos to uPy codebase!). Let's keep it open for a year, and see if at small number of users would vote for it. Say, 100. No, if they vote for it, it shouldn't mean automagic approval - we already had that mass-voting, where random people voted to call an executable a_very_long_name. I wonder how many of them actually type it - ever, or daily. So, after those 100 people voted up, in a year they should be contacted and asked if they remember what they voted up and if they still care. If they do, can go to the next stage - ask that small number of people to find now not-so-small number of users to back that idea, say 1000. That's good number to make a final decision. Unless another 500 vote against, then it's still not decisive. That's pretty much a virtual example, but I'd really like to bring up matter of formalizing decision process. It of course should begin with discussing how disagreement among maintainers is handled, how one becomes a maintainer (gets commit access), what are responsibilities, etc.
This case is of course more on trivial side (but shows a need to upgrade decision process). But I'd really like one of the aims is that as time goes forward, minimal MicroPython binary size shrinks. That's damn hard - it's perhaps easy to shrink it by 2%, and hard by 5%. By growing by 100% is piece of cake. That's why all the fuzz about 24 bytes. To that end, I'd like to raise awareness of this aim and share the load with other contributors, and to that end I submitted #2414. That's of course and RFC, to see if other dedicated contributors share those ideas. Just the same as I wrote https://github.com/micropython/micropython/wiki/ContributorGuidelines to see if there's some basic agreement in non-conflict of interests. It showed there's a needed level of agreement, and lot of productive work ensued. Apparently, time has come for another reality check. In the meantime, for consistency with existing documents, I'm firm -1 on this patch:
It's not explained why it's beneficial to run these packages with MicroPython. Nor how many missing features preclude doing that still. There's existing topic on the forum for people interested to run existing projects on uPy: http://forum.micropython.org/viewtopic.php?f=15&t=104 , experience can be shared there. And I'm speaking exactly from experience - 2 line patches aren't going to help running existing (soft|bloat)ware, but few dozens of 2, 3, 4, 5 ... 20 - line patches will deteriorate uPy's original aims to run everywhere.
That's a pure contradiction. So, let me just joke it up that I also want to eat a cake and have it too. |
|
@pfalcon None of us, me in particular, have time to deal with your abrasive attitude that continues to crop up in discussions like these. More than one time you have scared away willing contributors, when the same thing said with friendlier wording would have made the difference and kept them on board. If you want to talk about writing guidelines then we first need to make a code of conduct. Right now there are a few contributors coming on board who really want to help and it's up to me to make them feel welcome, so we can be on a friendly footing and discuss technical things (and have disagreements, not arguments) without harsh words. You make that job very difficult. You need to get comfortable with @deshipu because he will be sticking around (I hope!). He has been contributing to uPy outside github, eg giving talks on it, promoting it. |
Really? Please elaborate on that (here or by other means). Take an example of the doc mentioned above and number of times it was sad "no" based on it. It was specifically written to avoid and minimize "bad things" like time spent on patches which can't go. Special attention was attention was paid to make that doc accessible, etc. That's my attitude. Sorry, but when a case arises to invoke that doc, I link to it, not read it aloud (or rewrite it from scratch inplace) - just as everyone, I don't have time for that, there're enough leads it to read it before.
That gentleman has abrasive attitude. Self-proclaimed flame/flood master, he's willing to flame with anyone who responds. Alas, in our project, there're not so many people to respond, and as my attitude is to motivate contributors by providing prompt responses, it oftentimes happens to be me. It goes in the same vein when he's involved and me not, e.g. http://forum.micropython.org/viewtopic.php?f=3&t=2369 . It goes in the same way just from the start when he appeared (his messages can be dug out), and all attempts to set a dialog on constructive rails were futile, so now I with from any communication with him. Unlike his understanding of others' roles, I very well understand that he contributes to MicroPython, appreciate that, and welcome him to do more (and better). It's of course not a good situation when contributors can't communicate directly, and even have different-direction aims. That's why I'm, one of few committers and # 2 contributor to codebase (based on number of commits), call you, @dpgeorge to decide/confirm how much I (or any other contributor) have say in you project. It never was said explicitly, because well, it's assumed that it's how open-source projects run, but now time has come to pronounce whether it's based on meritocracy on some other principles, how that maps to direct voting numbers, etc. That takes time, yeah. But already time was spent already, and only more is intended to be. I particularly spent enough time dealing with abrasive types like deshipu. Now that you accuse me of being an abrasive type, it can't be left as is (though that might have been the easiest solution on time and efforts), and needs decision. Thanks. |
|
All I ask is for more friendly wording of issues/comments/posts, and to give people the benefit of the doubt if they perhaps misunderstood something or came at something from a different angle/background than oneself. |
It's meant to illustrate that testing OSError.errno is common practice in CPython. Please don't take my comments out of context. It's an investment since compiling e.args[0] results in larger code than compiling e.errno. So if the attached (and compiled) library includes 12 instances of this test you break even (on the x86/unix port). If it has more you'll have saved a few bytes. The subprocess module from stdlib which I'm looking into porting has 8 of these. I'm more than happy to just drop the request if it doesn't meet the standards. I didn't expect such a small patch to start a flamewar, but rather a discussion. |
|
Im using this as a reference of how not to keep a civilized communication. This is just a toxic exchange of words. |
|
I was thinking about suggesting MicroPython for a new product development in my company, but thankfully I found this discussion before. I cannot promote a project where their main leaders treat people's ideas like this, no matter if they're right or wrong. Instead, I'll be backing the use of other engine, even when it doesn't support Python. |
|
@harrycallahan I'm sorry that you came in at the wrong end of this discussion, and judge the project on the above posts. Speaking for myself, I try to react quickly to issues/PRs (when possible) and be friendly in my responses. Please consider evaluating this project on the quality of its code, the active forum (https://forum.micropython.org) and the response that most of the issues/PRs get. Thanks! |
|
@harrycallahan dpgeorge is right: this is one out of a couple of thousand PRs and issues. Treating it as examplary and then making decisions based solely on that is a fallacy. |
@michaelschrijver , First of all, the "discussion" which ensued, has nothing to do with your patch, but rather with a communication manner of a particular person. That person is well aware that his manner is not productive, see for example #2172 (comment) , quoting: "(and there is that bad troll habit I'm trying to get rid of too)", and others are well-approaching to him, see for example continuation of the discussion above, hoping that he'll stop pushing communication in that direction. But it happens again and again. Granted, bad patterns are bound to burst, and that happened, spectacularly. I'm sorry it happened in comments to your pull request, and thanks for not taking that as related to you personally. So, there's no reason to close this PR only because of the "discussion" above. The only reason would be that you agreed to technical arguments provided, and you don't have to.
It depends what you mean by "common". One definition would be "used every ten lines". Regardless of that, there're many things which can be called "common practice in CPython". Random examples are multiple inheritance, inheriting native types, plethora of special methods, etc. These don't work (fully) in MicroPython. So, while I listened to your argument that "OSError.errno is common practice in CPython", I'm still not convinced that adding it would change much. But if some port has binary size of e.g. 131062 bytes, with flash size 128K then adding it will push binary over 128K and make MicroPython run on one less platform. And MicroPython should run everywhere (that's how I have seen it all this time, and worked for it, I may be yet caught by surprise if uPy author (suddenly) says he doesn't care).
Neither please take mine. When you submitted your pull request perhaps you paid attention that we have 60+ pull requests open. I'm not sure if you paid attention that we have 1180 closed. I'm not sure if you though about that or not, but among them, there were multiple PRs to add some feature to e.g. language core. Some were accepted, some not. My response to your patch isn't based solely on it, and not based on desire to say "no" to you (I regret that very much; so much, that I volunteered to write doc which explains how to minimize chance to get "no" response). It's based on few-years history of PRs (mine too, including not submitted, self-rejected), on attempts to run "big Python" code on uPy, and on consistent and fair treatment on various PRs. While I very much would like to say "yes" to PR of each newcomer, the best way I could do that is to write docs guiding newcomers how to prepare such PR. I can't say "no" to one patch and "yes" to another similar one. If I do like that, I became unfair to the previous submitter. I don't even want to use word "friendly" popping up above, because everyone treats it differently, but for me unfair == unfriendly.
There're multiple ways to assess efficiency, some contradicting the other. We usually use simple and direct measures - RAM usage, ROM usage. That's pretty fair assessment - just consider that if someone doesn't use e.errno, that they don't need to have it ROM. And it's not fair to add it to ROMs of thousands people if just one person needs it. Also 2, 5, 10. We may reconsider if there will be more (adjusted for flashmob effect). So, I proposed what to do - we leave this PR open, collecting grounded people's votes, experiences, arguments. You paid attention to 60+ open pull requests and 200+ open tickets, right? They're waiting their time, yours will be one of them. There're mine there too. (And for PRs which don't cause disagreement, we try to offer fast turnaround times - as fast as 5 mins.) |
|
@pfalcon Thank you for your friendly and insightful response. I hope my response wasn't perceived as an attempt to rush this PR, I understand everyone is busy and good decisions take time. If there's something I can do to improve the PR (eg #ifdef'ing it) please let me know. |
|
@Harry - open source projects don't have an HR department like companies Mike On Thu, Sep 15, 2016 at 11:02 PM, Harry Callahan notifications@github.com
Meski "Going to Starbucks for coffee is like going to prison for sex. Sure, |
|
Here is a good code of conduct meant for open source projects: http://contributor-covenant.org/version/1/4/ |
|
See #7136 for an alternative to this PR. |
This commit adds the errno attribute to exceptions, so code can retrieve errno codes from an OSError using exc.errno. The implementation here simply lets `errno` (and the existing `value`) attributes work on any exception instance (they both alias args[0]). This is for efficiency and to keep code size down. The pros and cons of this are: Pros: - more compatible with CPython, less difference to document and learn - OSError().errno will correctly return None, whereas the current way of doing it via OSError().args[0] will raise an IndexError - it reduces code size on most bare-metal ports (because they already have the errno qstr) - for Python code that uses exc.errno the generated bytecode is 2 bytes smaller and more efficient to execute (compared with exc.args[0]); so bytecode loaded to RAM saves 2 bytes RAM for each use of this attribute, and bytecode that is frozen saves 2 bytes flash/ROM for each use - it's easier/shorter to type, and saves 2 bytes of space in .py files that use it (for each use) Cons: - increases code size by 4-8 bytes on minimal ports that don't already have the `errno` qstr - all exceptions now have .errno and .value attributes (a cpydiff test is added to address this) See also micropython#2407. Signed-off-by: Damien George <damien@micropython.org>
|
Support for |
This commit adds the errno attribute to exceptions, so code can retrieve errno codes from an OSError using exc.errno. The implementation here simply lets `errno` (and the existing `value`) attributes work on any exception instance (they both alias args[0]). This is for efficiency and to keep code size down. The pros and cons of this are: Pros: - more compatible with CPython, less difference to document and learn - OSError().errno will correctly return None, whereas the current way of doing it via OSError().args[0] will raise an IndexError - it reduces code size on most bare-metal ports (because they already have the errno qstr) - for Python code that uses exc.errno the generated bytecode is 2 bytes smaller and more efficient to execute (compared with exc.args[0]); so bytecode loaded to RAM saves 2 bytes RAM for each use of this attribute, and bytecode that is frozen saves 2 bytes flash/ROM for each use - it's easier/shorter to type, and saves 2 bytes of space in .py files that use it (for each use) Cons: - increases code size by 4-8 bytes on minimal ports that don't already have the `errno` qstr - all exceptions now have .errno and .value attributes (a cpydiff test is added to address this) See also micropython#2407. Signed-off-by: Damien George <damien@micropython.org>
I added "errno" as an alias for args[0] for exception objects with baseclass OSError in similar fashion to how StopIteration gets "value".