Skip to content

[mypyc] Add irbuild support for vec types#20724

Merged
JukkaL merged 180 commits into
masterfrom
mypyc-vec
Feb 3, 2026
Merged

[mypyc] Add irbuild support for vec types#20724
JukkaL merged 180 commits into
masterfrom
mypyc-vec

Conversation

@JukkaL

@JukkaL JukkaL commented Feb 2, 2026

Copy link
Copy Markdown
Collaborator

Add basic irbuild support for vec[t]. The runtime representation of vec[t] is a C struct. This is similar to how fixed-length tuples are represented. Multiple different structs are used, depending on the item type (VecI32 for vec[i32] and so on).

The C extension librt.vec that defines the vec type was added in #20653 and #20656. These PRs also explain the implementation in more detail.

Add RType subclass RVec that is used for vecs. We need a new RType class, since primitives types can't be generic and they can't be struct types.

This is based on an old branch, so it mostly uses old-style primitives. I am planning to modernize some of the primitives in follow-up PRs.

This doesn't include codegen support, and irbuild test cases are only included for vec[i64]. I will create follow-up PRs that add the remaining irbuild tests, codegen support and run tests. All these tests are are passing on my local full branch.

Related issue: mypyc/mypyc#840

Comment thread mypyc/analysis/ircheck.py Outdated


def is_pointer_arithmetic(op: IntOp) -> bool:
"""Check if op is add/subtract targeting pointer_rprimtive and integer of the same size."""

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""Check if op is add/subtract targeting pointer_rprimtive and integer of the same size."""
"""Check if op is add/subtract targeting pointer_rprimitive and integer of the same size."""

Comment thread mypyc/ir/rtypes.py Outdated
"object_ptr", is_unboxed=False, is_refcounted=False, ctype="PyObject **"
)

# Similar to object_primitive, but doesn not use automatic reference

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Similar to object_primitive, but doesn not use automatic reference
# Similar to object_primitive, but does not use automatic reference

Comment thread mypyc/ir/rtypes.py Outdated
return RVec(deserialize_type(data["item_type"], ctx))


def vec_depth(t: RVec) -> int:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as RVec.depth()

Comment thread mypyc/ir/rtypes.py
Comment on lines +1325 to +1326
names=["ob_base", "len", "items"],
types=[PyVarObject, int64_rprimitive],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

items is missing a type here and in the other buf objects for primitives.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IR can't represent the items field, since it's a variable-length array. That's why the type is not included. I will add a comment about this.

Comment thread mypyc/ir/rtypes.py Outdated
bool_rprimitive: "VecBoolApi",
}

# These are special type item type contants used in nested vecs to represent

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# These are special type item type contants used in nested vecs to represent
# These are special item type constants used in nested vecs to represent

Comment thread mypyc/irbuild/vec.py Outdated
item_addr = vec_item_ptr(builder, base, index)
result = builder.load_mem(item_addr, vtype.item_type, borrow=can_borrow)
builder.keep_alives.append(base)
return result

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aside from the index check this is very similar to the unsafe function, could it be called here?

Comment thread mypyc/irbuild/vec.py Outdated
Comment on lines +463 to +467
elif vec_depth(vec_type) == 0 and not isinstance(item_type, RUnion):
name = "VecTApi.remove"
else:
coerced_item = convert_to_t_ext_item(builder, coerced_item)
name = "VecNestedApi.remove"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean that for vecs with optional types we use the API for nested vecs? i thought it would be vecT. same pattern in vec_slice but not in other functions.

Comment on lines +331 to +336
r0 = VecI64Api.alloc(0, 0)
r1 = r0
r2 = 0
x = r2
L1:
r3 = r2 < 5 :: signed

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it probably won't matter much, but if we can tell that there will be 5 elements, we should be able to pre-allocate the vector.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good idea, I will add a TODO comment about this.

Comment on lines +469 to +475
r0 = VecI64Api.alloc(0, 0)
r1 = r0
r2 = 0
r3 = r2 >> 1
___tmp_5 = r3
L1:
r4 = int_lt r2, 14

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same in this case

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add a TODO comment.

Comment thread mypyc/irbuild/vec.py Outdated
Comment on lines +524 to +527
elif vec_type.depth() == 0 and not isinstance(item_type, RUnion):
name = "VecTApi.slice"
else:
name = "VecNestedApi.slice"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one more case where the handling for vecs with optional items should be updated, i think.

@JukkaL JukkaL merged commit 9da6043 into master Feb 3, 2026
16 checks passed
@JukkaL JukkaL deleted the mypyc-vec branch February 3, 2026 16:26
JukkaL added a commit that referenced this pull request Feb 4, 2026
…0732)

Follow-up to #20724. This wasn't included in the original PR to keep it
smaller. This includes irbuild tests for item types other than `i64`.

Related issue: mypyc/mypyc#840
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants