Skip to content

gh-146306: Optimize float operations by mutating uniquely-referenced operands in place (JIT only) #146307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Fidget-Spinner merged 12 commits into from
Mar 24, 2026
Merged

gh-146306: Optimize float operations by mutating uniquely-referenced operands in place (JIT only) #146307

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
42d12b7
gh-146306: Optimize float arithmetic in JIT by mutating uniquely-refe…
eendebakpt Mar 22, 2026
637803c
Fix JIT stencil compilation on i686-pc-windows-msvc
eendebakpt Mar 22, 2026
8559fb8
Fix i686 Windows JIT: use x87 FPU instead of SSE for stencils
eendebakpt Mar 22, 2026
7bfdedb
Update Python/bytecodes.c
eendebakpt Mar 24, 2026
c8f860f
Use sym_new_null for consumed operand slots in inplace float ops
eendebakpt Mar 24, 2026
fbedeb3
Revert unrelated bytecodes.c comment/whitespace changes
eendebakpt Mar 24, 2026
1efd764
Merge branch 'main' into jit_float_inplace_rebased
eendebakpt Mar 24, 2026
78c11e1
regenerate
eendebakpt Mar 24, 2026
8af7b51
handle case there operand is set to null
eendebakpt Mar 24, 2026
7d63316
reduce duplication in new opcodes
eendebakpt Mar 24, 2026
c61b9af
review comments
eendebakpt Mar 24, 2026
4ff36c1
cleanup unary opcode
eendebakpt Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix JIT stencil compilation on i686-pc-windows-msvc 
Avoid compound assignment (+=, -=, *=) directly on ob_fval in
inplace float ops. On 32-bit Windows, this generates JIT stencils
with _xmm register references that MSVC cannot parse. Instead,
read into a local double, compute, and write back.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
  • Loading branch information
@eendebakpt @claude
eendebakpt and claude committed Mar 22, 2026
commit 637803c885f332a0fd34606c254f05cefe230c3a
29 changes: 18 additions & 11 deletions Python/bytecodes.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -467,7 +467,8 @@ dummy_func(
assert(PyFloat_CheckExact(val_o));
assert(_PyObject_IsUniquelyReferenced(val_o));
STAT_INC(UNARY_NEGATIVE, hit);
((PyFloatObject *)val_o)->ob_fval = -((PyFloatObject *)val_o)->ob_fval;
double dres = -((PyFloatObject *)val_o)->ob_fval;
((PyFloatObject *)val_o)->ob_fval = dres;
res = value;
v = PyStackRef_NULL;
INPUTS_DEAD();
Expand Down Expand Up @@ -788,14 +789,18 @@ dummy_func(
// instead of allocating a new float. Tier 2 only.
// The optimizer sets l to null so the following _POP_TOP_FLOAT
// becomes _POP_TOP_NOP.
// Note: read into a local double and write back to avoid compound
// assignment (+=) on ob_fval, which generates problematic JIT
// stencils on i686-pc-windows-msvc.
tier2 op(_BINARY_OP_ADD_FLOAT_INPLACE, (left, right -- res, l, r)) {
Copy link
Copy Markdown
Member

@markshannon markshannon Mar 23, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This op and its variants all share a lot of common code.
Could you factor out the code into a macro to perform the inplace operation?
Like:

tier2 op(_BINARY_OP_ADD_FLOAT_INPLACE, (left, right -- res, l, r)) {
    res = FLOAT_INPLACE_OP(left, +, right);
    l = PyStackRef_NULL;
    r = right;
    INPUTS_DEAD();
}

tier2 op(_BINARY_OP_MULTIPLY_FLOAT_INPLACE_RIGHT, (left, right -- res, l, r)) {
    res = FLOAT_INPLACE_OP(right, *, left);
    l = left
    r = PyStackRef_NULL;
    INPUTS_DEAD();
}

Copy link
Copy Markdown
Contributor Author

@eendebakpt eendebakpt Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normal C macros are not allowed in the bytecodes.c opcodes (they are not expanded). I have not yet found a way to refactor this nicely.

We could add a new macro to the DSL (like INPUTS_DEAD) for this, but it feels a bit odd to add something to the DSL for this particular case.

Copy link
Copy Markdown
Member

@Fidget-Spinner Fidget-Spinner Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normal C macros are not allowed in the bytecodes.c opcodes

They are. You just need to define them in ceval_macros.h. Would you like me to do this, or would you like to do it?

PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
assert(PyFloat_CheckExact(left_o));
assert(PyFloat_CheckExact(right_o));
assert(_PyObject_IsUniquelyReferenced(left_o));
STAT_INC(BINARY_OP, hit);
((PyFloatObject *)left_o)->ob_fval += ((PyFloatObject *)right_o)->ob_fval;
double dres = ((PyFloatObject *)left_o)->ob_fval + ((PyFloatObject *)right_o)->ob_fval;
((PyFloatObject *)left_o)->ob_fval = dres;
res = left;
l = PyStackRef_NULL;
r = right;
Comment thread
eendebakpt marked this conversation as resolved.
Expand All @@ -809,7 +814,8 @@ dummy_func(
assert(PyFloat_CheckExact(right_o));
assert(_PyObject_IsUniquelyReferenced(left_o));
STAT_INC(BINARY_OP, hit);
((PyFloatObject *)left_o)->ob_fval -= ((PyFloatObject *)right_o)->ob_fval;
double dres = ((PyFloatObject *)left_o)->ob_fval - ((PyFloatObject *)right_o)->ob_fval;
((PyFloatObject *)left_o)->ob_fval = dres;
res = left;
l = PyStackRef_NULL;
r = right;
Expand All @@ -823,23 +829,24 @@ dummy_func(
assert(PyFloat_CheckExact(right_o));
assert(_PyObject_IsUniquelyReferenced(left_o));
STAT_INC(BINARY_OP, hit);
((PyFloatObject *)left_o)->ob_fval *= ((PyFloatObject *)right_o)->ob_fval;
double dres = ((PyFloatObject *)left_o)->ob_fval * ((PyFloatObject *)right_o)->ob_fval;
((PyFloatObject *)left_o)->ob_fval = dres;
res = left;
l = PyStackRef_NULL;
r = right;
INPUTS_DEAD();
}

// Inplace RIGHT variants for commutative ops (add, multiply).
// Mutate the uniquely-referenced right operand instead.
// Inplace RIGHT variants: mutate the uniquely-referenced right operand.
tier2 op(_BINARY_OP_ADD_FLOAT_INPLACE_RIGHT, (left, right -- res, l, r)) {
PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
assert(PyFloat_CheckExact(left_o));
assert(PyFloat_CheckExact(right_o));
assert(_PyObject_IsUniquelyReferenced(right_o));
STAT_INC(BINARY_OP, hit);
((PyFloatObject *)right_o)->ob_fval += ((PyFloatObject *)left_o)->ob_fval;
double dres = ((PyFloatObject *)left_o)->ob_fval + ((PyFloatObject *)right_o)->ob_fval;
((PyFloatObject *)right_o)->ob_fval = dres;
res = right;
l = left;
r = PyStackRef_NULL;
Expand All @@ -853,23 +860,23 @@ dummy_func(
assert(PyFloat_CheckExact(right_o));
assert(_PyObject_IsUniquelyReferenced(right_o));
STAT_INC(BINARY_OP, hit);
((PyFloatObject *)right_o)->ob_fval *= ((PyFloatObject *)left_o)->ob_fval;
double dres = ((PyFloatObject *)left_o)->ob_fval * ((PyFloatObject *)right_o)->ob_fval;
((PyFloatObject *)right_o)->ob_fval = dres;
res = right;
l = left;
r = PyStackRef_NULL;
INPUTS_DEAD();
}

// Inplace RIGHT variant for subtract (non-commutative):
// right->ob_fval = left->ob_fval - right->ob_fval
tier2 op(_BINARY_OP_SUBTRACT_FLOAT_INPLACE_RIGHT, (left, right -- res, l, r)) {
PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
assert(PyFloat_CheckExact(left_o));
assert(PyFloat_CheckExact(right_o));
assert(_PyObject_IsUniquelyReferenced(right_o));
STAT_INC(BINARY_OP, hit);
((PyFloatObject *)right_o)->ob_fval = ((PyFloatObject *)left_o)->ob_fval - ((PyFloatObject *)right_o)->ob_fval;
double dres = ((PyFloatObject *)left_o)->ob_fval - ((PyFloatObject *)right_o)->ob_fval;
((PyFloatObject *)right_o)->ob_fval = dres;
res = right;
l = left;
r = PyStackRef_NULL;
Expand Down
63 changes: 42 additions & 21 deletions Python/executor_cases.c.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading