Skip to content

GH-100240: Generic freelist, applied to ints #101453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
iritkatriel wants to merge 19 commits into from
Closed

GH-100240: Generic freelist, applied to ints #101453

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
e953c57
freelist for ints
iritkatriel Jan 30, 2023
cf99ee4
moved things around a little
iritkatriel Jan 31, 2023
8a6e6a3
remove WITH_FREELISTS in longobject.c
iritkatriel Jan 31, 2023
fe65f49
FreeToFull leaves the list empty
iritkatriel Jan 31, 2023
a33061f
reuse _PyFreeList_Clear
iritkatriel Feb 3, 2023
6b312e0
fix free bug and remove ifdef in a few places
iritkatriel Feb 3, 2023
1d81198
use _PyLong_Free in specialized bytecodes
iritkatriel Feb 3, 2023
5609e30
use _PyLong_Free in _Py_DECREF_INT
iritkatriel Feb 3, 2023
b8b1879
stats for generic freelist
iritkatriel Feb 5, 2023
460d12c
print new stats
iritkatriel Feb 6, 2023
2e2a861
fix index mapping
iritkatriel Feb 6, 2023
42ee27f
tweak the stats
iritkatriel Feb 7, 2023
a9e76ad
shift instead of div
iritkatriel Feb 8, 2023
1134727
Merge remote-tracking branch 'upstream/main' into int-freelist
iritkatriel Feb 8, 2023
78592b9
int --> Py_ssize_t
iritkatriel Feb 8, 2023
044743e
disable the freelist after clearing it in interpreter_clear
iritkatriel Feb 8, 2023
827b81b
add _PyFreeList_Disable
iritkatriel Feb 9, 2023
51c6c3c
rename INDEX --> SIZE_CLASS
iritkatriel Feb 10, 2023
cfb886d
stats per size class
iritkatriel Feb 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix free bug and remove ifdef in a few places
  • Loading branch information
@iritkatriel
iritkatriel committed Feb 3, 2023
commit 6b312e0da1acdedd0f104392578a16c249d09345
17 changes: 4 additions & 13 deletions Include/internal/pycore_interp.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,9 +182,7 @@ struct _is {
struct _Py_context_state context;
struct _Py_exc_state exc_state;

#if WITH_FREELISTS
_PyFreeList freelists[INTERP_NUM_FREELISTS];
#endif
struct ast_state ast;
struct types_state types;
struct callable_cache callable_cache;
Expand Down Expand Up @@ -237,30 +235,23 @@ PyAPI_FUNC(int) _PyInterpreterState_IDInitref(PyInterpreterState *);
PyAPI_FUNC(int) _PyInterpreterState_IDIncref(PyInterpreterState *);
PyAPI_FUNC(void) _PyInterpreterState_IDDecref(PyInterpreterState *);

#define SIZE_TO_FREELIST_INDEX(size) ((size-4)/2)
#define FREELIST_INDEX_TO_SIZE(idx) (2*(idx) + 4)
#define FREELIST_QUANTUM (2*sizeof(void*))
#define SIZE_TO_FREELIST_INDEX(size) ((size-4)/FREELIST_QUANTUM)
Copy link
Copy Markdown
Member

@markshannon markshannon Feb 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this formula?

You want to allocate all sizes from (n-1)*FREELIST_QUANTUM + 1 to n*FREELIST_QUANTUM (inclusive) from the same freelist.
I would expect the formula to be (size + FREELIST_QUANTUM -1)/FREELIST_QUANTUM, or (size-1)/FREELIST_QUANTUM.

And because C division rounds towards 0, not -infinity, you want to use >>LOG_BASE_2_OF_FREELIST_QUANTUM, not /FREELIST_QUANTUM.

Copy link
Copy Markdown
Member Author

@iritkatriel iritkatriel Feb 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't changed this yet. I'm trying to get it to work with just ints.

Copy link
Copy Markdown
Member Author

@iritkatriel iritkatriel Feb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There already are some macros in pycore_obmalloc.h that seem to do something like this:

/* 
 * Alignment of addresses returned to the user. 8-bytes alignment works
 * on most current architectures (with 32-bit or 64-bit address buses).
 * The alignment value is also used for grouping small requests in size
 * classes spaced ALIGNMENT bytes apart.
 *
 * You shouldn't change this unless you know what you are doing.
 */

#if SIZEOF_VOID_P > 4
#define ALIGNMENT              16               /* must be 2^N */
#define ALIGNMENT_SHIFT         4
#else
#define ALIGNMENT               8               /* must be 2^N */
#define ALIGNMENT_SHIFT         3
#endif
   
/* Return the number of bytes in size class I, as a uint. */
#define INDEX2SIZE(I) (((pymem_uint)(I) + 1) << ALIGNMENT_SHIFT)

#define FREELIST_INDEX_TO_SIZE(idx) (FREELIST_QUANTUM*(idx) + 4)
Comment thread
iritkatriel marked this conversation as resolved.
Outdated

static inline PyObject*
_PyInterpreterState_FreelistAlloc(PyInterpreterState *interp, Py_ssize_t size) {
#if WITH_FREELISTS
int index = SIZE_TO_FREELIST_INDEX(size);
assert(index >= 0 && index < INTERP_NUM_FREELISTS);
return _PyFreeList_Alloc(&interp->freelists[index]);
#else
return PyObject_Malloc(size);
#endif
}

static inline void
_PyInterpreterState_FreelistFree(PyInterpreterState * interp, PyObject *op, Py_ssize_t size) {
#if WITH_FREELISTS
/* todo: assert the size is correct? */
Comment thread
iritkatriel marked this conversation as resolved.
int index = SIZE_TO_FREELIST_INDEX(size);
assert(index >= 0 && index < INTERP_NUM_FREELISTS);
_PyFreeList_Alloc(&interp->freelists[index]);
#else
Py_TYPE(op)->tp_free((PyObject *)op);
#endif
_PyFreeList_Free(&interp->freelists[index], op);
}


Expand Down