Skip to content

gh-119786: Edit InternalDocs/frames.md, Python/vm-state.md, Python/tier2_engine.md #124450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Eclips4 wants to merge 11 commits into
base: main
Choose a base branch
from
Open

gh-119786: Edit InternalDocs/frames.md, Python/vm-state.md, Python/tier2_engine.md #124450

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
60b1689
Move files to InternalDocs
Eclips4 Sep 24, 2024
fbf47e4
Fix grammar
Eclips4 Sep 24, 2024
3aaa70c
Add link to VM in the README
Eclips4 Sep 24, 2024
1a6df69
Apply suggestions from code review
Eclips4 Sep 27, 2024
944d1ab
Remove reference to PEP 695; instead add reference to adaptive.md
Eclips4 Sep 27, 2024
0c2489f
Add reference to exception_handling.md
Eclips4 Sep 27, 2024
c7cdfb5
Move thread state section right after the frame state section
Eclips4 Sep 27, 2024
be38623
Remove Tier2 IR section from vm-state
Eclips4 Sep 27, 2024
26d5559
Address review
Eclips4 Jan 13, 2025
52225eb
Move unwinding to the exception_handling
Eclips4 Jan 13, 2025
ba8f39a
Unwinding is already covered; Reverting
Eclips4 Jan 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Move thread state section right after the frame state section
  • Loading branch information
@Eclips4
Eclips4 committed Sep 27, 2024
commit c7cdfb5f6b071d3818953f7b440289995d9f23ad
21 changes: 11 additions & 10 deletions InternalDocs/vm-state.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,22 @@
- **Tier 2**, also known as the micro-instruction ("uop") interpreter, is a new interpreter with a different instruction format.
Comment thread
Eclips4 marked this conversation as resolved.
Outdated
It was introduced in Python 3.13, and also forms the basis for a JIT using copy-and-patch technology. See [Tier 2](tier2_engine.md) for more information.


# Frame state
Comment thread
Eclips4 marked this conversation as resolved.
Outdated

Almost all interpreter state is nominally stored in the frame structure.
A pointer to the current frame is held in `frame`, for more information about what `frame` contains see [Frames](frames.md):
Comment thread
Eclips4 marked this conversation as resolved.
Outdated
Comment thread
Eclips4 marked this conversation as resolved.
Outdated

Copy link
Copy Markdown
Member

@iritkatriel iritkatriel Sep 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CC @markshannon

I can't comment on lines that haven't changed, so this is for a number of comments on vm-state.md.

L 21: The interpreters share an implementation of what? The frame? Caches the depth - is this stack depth?

L 40: Add a link to exception_handling.md.

L45: Not sure what you mean here: "The implementation of jumps within a single Tier 2 superblock/trace is just that, an implementation."

L51: "within the superblock" is repeated twice.
L52: what cannot be modified?

I think it might be worth moving the contents of the "Thread state and interpreter state" section to the beginning, as a high level overview of the components of the state, and then drill into the parts.

The "Tier 2 IR format" section doesn't seem to belong to the vm-state topic at all.

Copy link
Copy Markdown
Member

@markshannon markshannon Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The interpreters share an implementation of what?

Both tier 1 and tier 2 use the same canonical in-memory representation. Tier 2 might store some values temporarily in registers, but that should be invisible to other code. The reason this is noteworthy is that other VMs, e.g. HotSpot can have different frame layouts for the compiler and interpreter.

# Thread state and interpreter state

Another important piece of VM state is the **thread state**, held in `tstate`.
Comment thread
Eclips4 marked this conversation as resolved.
Outdated
The current frame pointer, `frame`, is always equal to `tstate->current_frame`.
The thread state also holds the exception state (`tstate->exc_info`) and the recursion counters (`tstate->c_recursion_remaining` and `tstate->py_recursion_remaining`).

The thread state is also used to access the **interpreter state** (`tstate->interp`), which is important since the "eval breaker" flags are stored there (`tstate->interp->ceval.eval_breaker`, an "atomic" variable), as well as the "PEP 523 function" (`tstate->interp->eval_frame`).
The interpreter state also holds the optimizer state (`optimizer` and some counters).
Note that the eval breaker may be moved to the thread state soon as part of the multicore (PEP 703) work.

## Fast locals and evaluation stack

The frame contains a single array of object pointers, `localsplus`, which contains both the fast locals and the stack.
Expand Down Expand Up @@ -59,16 +70,6 @@ It will be more complex in the JIT.

(We might also consider deoptimizations as a separate jump type.)

# Thread state and interpreter state

Another important piece of VM state is the **thread state**, held in `tstate`.
The current frame pointer, `frame`, is always equal to `tstate->current_frame`.
The thread state also holds the exception state (`tstate->exc_info`) and the recursion counters (`tstate->c_recursion_remaining` and `tstate->py_recursion_remaining`).

The thread state is also used to access the **interpreter state** (`tstate->interp`), which is important since the "eval breaker" flags are stored there (`tstate->interp->ceval.eval_breaker`, an "atomic" variable), as well as the "PEP 523 function" (`tstate->interp->eval_frame`).
The interpreter state also holds the optimizer state (`optimizer` and some counters).
Note that the eval breaker may be moved to the thread state soon as part of the multicore (PEP 703) work.

# Tier 2 IR format

The tier 2 IR (Internal Representation) format is also the basis for the Tier 2 interpreter (though the two formats may eventually differ). This format is also used as the input to the machine code generator (the JIT compiler).
Expand Down