Skip to content

gh-123881: make compiler add the .generic_base base class without constructing AST nodes #123883

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iritkatriel merged 15 commits into from
Sep 10, 2024
Merged

gh-123881: make compiler add the .generic_base base class without constructing AST nodes #123883

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
031c697
add action helper for ClassDef
iritkatriel Sep 9, 2024
6547576
add .generic_base to bases in AST so compiler doesn't need to do it
iritkatriel Sep 9, 2024
412fe99
update tests
iritkatriel Sep 9, 2024
781fc79
add news
iritkatriel Sep 9, 2024
8200b77
add whatsnew entry
iritkatriel Sep 9, 2024
d75532a
Revert "add whatsnew entry"
iritkatriel Sep 9, 2024
22228eb
Revert "add news"
iritkatriel Sep 9, 2024
4831765
Revert "update tests"
iritkatriel Sep 9, 2024
5218173
Revert "add .generic_base to bases in AST so compiler doesn't need to…
iritkatriel Sep 9, 2024
0e52c0f
Revert "add action helper for ClassDef"
iritkatriel Sep 9, 2024
341f8a5
keep it in the compiler
iritkatriel Sep 9, 2024
e92c432
handle stararg case
iritkatriel Sep 10, 2024
40046c1
Merge remote-tracking branch 'upstream/main' into generic
iritkatriel Sep 10, 2024
4611b92
the compiler struct no longer needs the arena
iritkatriel Sep 10, 2024
c6c3cff
Merge branch 'main' into generic
iritkatriel Sep 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add .generic_base to bases in AST so compiler doesn't need to do it
  • Loading branch information
@iritkatriel
iritkatriel committed Sep 9, 2024
commit 654757698f47007609a8c0cbc3285c1d41d7a4c0
1 change: 0 additions & 1 deletion Include/internal/pycore_compile.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,6 @@ int _PyCompile_IsNestedScope(struct _PyCompiler *c);
int _PyCompile_IsInInlinedComp(struct _PyCompiler *c);
int _PyCompile_ScopeType(struct _PyCompiler *c);
int _PyCompile_OptimizationLevel(struct _PyCompiler *c);
PyArena *_PyCompile_Arena(struct _PyCompiler *c);
int _PyCompile_LookupArg(struct _PyCompiler *c, PyCodeObject *co, PyObject *name);
PyObject *_PyCompile_Qualname(struct _PyCompiler *c);
_PyCompile_CodeUnitMetadata *_PyCompile_Metadata(struct _PyCompiler *c);
Expand Down
24 changes: 24 additions & 0 deletions Parser/action_helpers.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -758,6 +758,30 @@ _PyPegen_class_def(expr_ty a, arguments_ty b, asdl_stmt_seq *c, asdl_type_param_
asdl_stmt_seq *body = c;
asdl_expr_seq *decorator_list = NULL;
asdl_type_param_seq *type_params = t;

int is_generic = asdl_seq_LEN(type_params) > 0;
if (is_generic) {
/* Add */
asdl_expr_seq *orig_bases = bases;
Py_ssize_t orig_len = asdl_seq_LEN(orig_bases);
bases = _Py_asdl_expr_seq_new(orig_len + 1, arena);
if (bases == NULL) {
return NULL;
}
for (Py_ssize_t i = 0; i < orig_len; i++) {
asdl_seq_SET(bases, i, asdl_seq_GET(orig_bases, i));
}
_Py_DECLARE_STR(generic_base, ".generic_base");
expr_ty name_node = _PyAST_Name(
&_Py_STR(generic_base), Load,
lineno, col_offset, end_lineno, end_col_offset, arena
);
if (name_node == NULL) {
return NULL;
}
asdl_seq_SET(bases, orig_len, name_node);
}

return _PyAST_ClassDef(name, bases, keywords, body, decorator_list, type_params,
lineno, col_offset, end_lineno, end_col_offset, arena);
}
Expand Down
22 changes: 1 addition & 21 deletions Python/codegen.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ typedef struct _PyCompiler compiler;
#define SCOPE_TYPE(C) _PyCompile_ScopeType(C)
#define QUALNAME(C) _PyCompile_Qualname(C)
#define METADATA(C) _PyCompile_Metadata(C)
#define ARENA(C) _PyCompile_Arena(C)

typedef _PyInstruction instruction;
typedef _PyInstructionSequence instr_sequence;
Expand Down Expand Up @@ -1549,27 +1548,8 @@ codegen_class(compiler *c, stmt_ty s)
ADDOP_I_IN_SCOPE(c, loc, CALL_INTRINSIC_1, INTRINSIC_SUBSCRIPT_GENERIC);
RETURN_IF_ERROR_IN_SCOPE(c, codegen_nameop(c, loc, &_Py_STR(generic_base), Store));

Py_ssize_t original_len = asdl_seq_LEN(s->v.ClassDef.bases);
asdl_expr_seq *bases = _Py_asdl_expr_seq_new(
original_len + 1, ARENA(c));
if (bases == NULL) {
_PyCompile_ExitScope(c);
return ERROR;
}
for (Py_ssize_t i = 0; i < original_len; i++) {
asdl_seq_SET(bases, i, asdl_seq_GET(s->v.ClassDef.bases, i));
}
expr_ty name_node = _PyAST_Name(
&_Py_STR(generic_base), Load,
loc.lineno, loc.col_offset, loc.end_lineno, loc.end_col_offset, ARENA(c)
);
if (name_node == NULL) {
_PyCompile_ExitScope(c);
return ERROR;
}
asdl_seq_SET(bases, original_len, name_node);
RETURN_IF_ERROR_IN_SCOPE(c, codegen_call_helper(c, loc, 2,
bases,
s->v.ClassDef.bases,
Copy link
Copy Markdown
Member

@JelleZijlstra JelleZijlstra Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's important to you to remove the code creating a fake AST node here, you could instead add an extra argument to codegen_call_helper that is usually NULL and maps to an extra argument to add at the end of the list. I think I did that initially while working on PEP 695, but it seemed worse than creating a new virtual asdl_seq as the current code does. (Possibly you were involved in discussing that in person, don't remember exactly.)

But I'd like to keep the extra base class as an implementation detail in the compiler, not something that gets exposed in the AST layer.

Copy link
Copy Markdown
Member Author

@iritkatriel iritkatriel Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll try something like that, should be quite simple.
It's possible I was involved in the discussion at the time, I don't remember the details.

I think it would be better if the compiler didn't need to know how to create AST nodes. (I take your point about it not modifying the AST, I'll correct the wording in the issue)

Copy link
Copy Markdown
Member

@JelleZijlstra JelleZijlstra Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found back the discussion from last time: #103764 (comment).

s->v.ClassDef.keywords));

PyCodeObject *co = _PyCompile_OptimizeAndAssemble(c, 0);
Expand Down
6 changes: 0 additions & 6 deletions Python/compile.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1244,12 +1244,6 @@ _PyCompile_Metadata(compiler *c)
return &c->u->u_metadata;
}

PyArena *
_PyCompile_Arena(compiler *c)
{
return c->c_arena;
}

#ifndef NDEBUG
int
_PyCompile_IsTopLevelAwait(compiler *c)
Expand Down