Repository navigation
Add missing dis features for modulefinder #3846
Description
It is using a few missing features in _dis module.
Maybe adding them will not be enough for modulefinder due to bytecode imcompatibility.But adding dis items will be a good issue to start looking at RustPython project.
To add a new python module, check this article: https://rustpython.github.io/blog/2020/04/04/how-to-contribute-by-cpython-unittest.html
See #3863 for errors
Activity
itsankitkp commentedon Jul 19, 2022
Can I pick up this issue? Also, can you suggest a good place to start?
youknowone commentedon Jul 19, 2022
try to checkout #3863 and check what kind of functions are not implemented.
reading related cpython code is also helpful.
In RustPython,
_dismodule is here: https://github.com/RustPython/RustPython/blob/main/stdlib/src/dis.rsdismodule is here: https://github.com/RustPython/RustPython/blob/main/Lib/dis.pyitsankitkp commentedon Jul 19, 2022
Thanks @youknowone, I will take a look at PR and specified files.
itsankitkp commentedon Jul 19, 2022
My observations:
dis.opmap['LOAD_CONST']
cpython: 100
rustpython: AttributeError: module 'dis' has no attribute 'opmap'
dis.EXTENDED_ARG
cpython: 144
rustpython: AttributeError: module 'dis' has no attribute 'EXTENDED_ARG'
dis._unpack_opargs(100)
cpython: <generator object _unpack_opargs at 0x7f465322fde0>
rustpython: AttributeError: module 'dis' has no attribute '_unpack_opargs'
These are only relevant to #3863
Additionally:
dir(dis) in cpython
['BINARY_OP', 'Bytecode', 'CACHE', 'COMPILER_FLAG_NAMES', 'EXTENDED_ARG', 'FORMAT_VALUE', 'FORMAT_VALUE_CONVERTERS', 'FOR_ITER', 'HAVE_ARGUMENT', 'Instruction', 'JUMP_BACKWARD', 'LOAD_ATTR', 'LOAD_CONST', 'LOAD_GLOBAL', 'MAKE_FUNCTION', 'MAKE_FUNCTION_FLAGS', 'Positions', 'UNKNOWN', '_ExceptionTableEntry', '_INT_BITS', '_INT_OVERFLOW', '_Instruction', '_OPARG_WIDTH', '_OPNAME_WIDTH', '_Unknown', 'all', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'spec', '_all_opmap', '_all_opname', '_cache_format', '_deoptop', '_disassemble_bytes', '_disassemble_recursive', '_disassemble_str', '_empty_slot', '_find_imports', '_find_store_names', '_format_code_info', '_get_code_array', '_get_code_object', '_get_const_info', '_get_const_value', '_get_instructions_bytes', '_get_name_info', '_have_code', '_inline_cache_entries', '_is_backward_jump', '_nb_ops', '_specializations', '_specialized_instructions', '_test', '_try_compile', '_unpack_opargs', 'cmp_op', 'code_info', 'collections', 'deoptmap', 'dis', 'disassemble', 'disco', 'distb', 'findlabels', 'findlinestarts', 'get_instructions', 'hasarg', 'hascompare', 'hasconst', 'hasexc', 'hasfree', 'hasjabs', 'hasjrel', 'haslocal', 'hasname', 'io', 'opmap', 'opname', 'parse_exception_table', 'parse_varint', 'pretty_flags', 'show_code', 'spec_op', 'specialized', 'stack_effect', 'sys', 'types']
2. rustpython:
['COMPILER_FLAG_NAMES', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'spec', '_test', 'dis', 'disassemble']
So how do we approach this problem? Module finder needs _unpack_opargs, EXTENDED_ARG and opmap, if these gets implemented in dis module, requirement of this PR gets fulfilled right? @youknowone
youknowone commentedon Jul 20, 2022
yes, I think so. another small part is being ready to be not disappointed even when adding them doesn't fix the module.
you even don't need to add all of them in single PR.
we value and encourage every progressing step (without regression).
itsankitkp commentedon Jul 20, 2022
Thanks for head up, I will be taking incremental approach to this and will mention any challenges.
itsankitkp commentedon Jul 20, 2022
A quick question, how do we all translate python libraries? I get that c code is translated into equivalent rust code but what should be done for the Python code?
For Python code depending on instruction, rustpython equivalent would need to skip out unimplemented piece but for purely python library (think opcode.py, just hard coded values of opcodes), do we need to translate something or code is practically same as cpython (python) library?
youknowone commentedon Jul 21, 2022
Ideally, we run the same code.
But for opcodes, yes, as you described, it cannot be fully compatible. bytecodes are even not compatible between each cpython versions.
When I looked in modulefinder a little bit, it only uses a few (probably stable) opcodes. I wish we also has that amount of stability, but I don't know we are or not.
I don't expect you to translate python code to python code to run it. If you found it is not working, we will accept we cannot run it for now, and try to fix our compiler or vm later.
But even if it happens, we will get a few new
dismodule features during the journey.itsankitkp commentedon Jul 24, 2022
Okay I got it
[-]Add missing dis featues for modulefinder[/-][+]Add missing dis features for modulefinder[/+]youknowone commentedon Oct 6, 2026
The originally reported problem no longer reproduces on current main. Closing per the recheck in #8982.
— commented by Grok 4.6