-
-
Notifications
You must be signed in to change notification settings - Fork 34.5k
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. #1748
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ericsnowcurrently
merged 78 commits into
python:master
from
ericsnowcurrently:low-level-interpreters-module
Jan 30, 2018
Merged
Changes from 1 commit
Commits
Show all changes
78 commits
Select commit
Hold shift + click to select a range
be93a28
Add the _interpreters module to the stdlib.
ericsnowcurrently 6f7a120
Add create() and destroy().
ericsnowcurrently 941efdd
Finish nearly all the create/destroy tests.
ericsnowcurrently 95e7342
Add run_string().
ericsnowcurrently 92aeaac
Get tricky tests working.
ericsnowcurrently b6e2f3a
Add a test for a still running interpreter when main exits.
ericsnowcurrently c15e79b
Add run_string_unrestricted().
ericsnowcurrently 8d809c7
Exit out of the child process.
ericsnowcurrently 55bb53c
Resolve several TODOs.
ericsnowcurrently cad0a91
Set up the execution namespace before switching threads.
ericsnowcurrently 1395c58
Run in a copy of __main__.
ericsnowcurrently e0e8176
Close stdin and stdout after the proc finishes.
ericsnowcurrently 0eb4acc
Clean up a test.
ericsnowcurrently 3562084
Chain exceptions during cleanup.
ericsnowcurrently 8e7e724
Finish the module docs.
ericsnowcurrently f8173fb
Fix docs.
ericsnowcurrently 84cea38
Fix includes.
ericsnowcurrently 48a1a38
Add _interpreters.is_shareable().
ericsnowcurrently bbdadb0
Add _PyObject_CheckShareable().
ericsnowcurrently 9a75fd2
Add _PyCrossInterpreterData.
ericsnowcurrently f38b138
Use the shared data in run() safely.
ericsnowcurrently 0055e80
Do not use a copy of the __main__ ns.
ericsnowcurrently 9030634
Never return the execution namespace.
ericsnowcurrently df53c6c
Group sharing-related code.
ericsnowcurrently b27597f
Fix a refcount.
ericsnowcurrently b69a643
Add get_current() and enumerate().
ericsnowcurrently dc21da6
Add is_running().
ericsnowcurrently a4dad08
Add get_main().
ericsnowcurrently bc818e5
Fix an INCREF placement.
ericsnowcurrently d80ffdb
Fix error propagation.
ericsnowcurrently f4b8819
enumerate -> list_all.
ericsnowcurrently c1b2a79
Correctly handle a failed _shared_exception allocation.
ericsnowcurrently 406d1ec
Add _interpreters.RunFailedError.
ericsnowcurrently 6a53db3
Return a simple RunFailedError.
ericsnowcurrently f1e91b0
(mostly) Fix the unit tests.
ericsnowcurrently e7c581a
Add tests for shareable objects.
ericsnowcurrently 24095ab
Add None as shareable.
ericsnowcurrently 7ebae2f
Add a registry for shareable types.
ericsnowcurrently 6d562bb
Fix (and clean up) the cross-interpreter-data code.
ericsnowcurrently 237f743
Add basic channel code.
ericsnowcurrently 3b0ca21
Stub out the other channel-related module functions.
ericsnowcurrently 064f710
Drop _cidclass.classname.
ericsnowcurrently 703ac97
Drop some outdated TODO comments.
ericsnowcurrently e36e4d4
Add a mutex for the registry.
ericsnowcurrently b973cc4
Add _PyChannelState.mutex.
ericsnowcurrently 6b9e457
Add a mutex for operating on the global list of channels.
ericsnowcurrently 12efb00
Clean up the locking a little.
ericsnowcurrently 310d0ac
Implement _interpreters.channel_list_all().
ericsnowcurrently 8a24220
Implement _interpreters.channel_close().
ericsnowcurrently 7c8ede9
Add channel-related exceptions.
ericsnowcurrently 47d75de
Add _interpreters.ChannelID.
ericsnowcurrently 153bbec
Solidify the "close" story.
ericsnowcurrently 79d31cd
Drop channel_list_interpreters().
ericsnowcurrently 3aad28b
Register ChannelID as a cross-intepreter type.
ericsnowcurrently db72825
_interpreters -> _xxsubinterpreters.
ericsnowcurrently 6a0c0ff
Fix an un-initialized variable.
ericsnowcurrently 9d50730
NULL-terminate kwlist for PyArg_ParseTupleAndKeywords().
ericsnowcurrently b31d130
Drop docs for now.
ericsnowcurrently 91e7c8b
Do not check for NULL from PyThreadState_Get().
ericsnowcurrently 7d2cfea
Format "else" statements properly.
ericsnowcurrently 73dad33
Always use braces with if statements.
ericsnowcurrently ee7141c
Check for a NULL return from PyMem_NEW().
ericsnowcurrently 8ec5137
Return the error.
ericsnowcurrently 65d822e
Always check the result of PyDict_SetItemString().
ericsnowcurrently 2df5aff
Add typedefs.
ericsnowcurrently 9bb05cf
_shared_exception -> _sharedexception
ericsnowcurrently 633a3d3
Free memory in the error case.
ericsnowcurrently 74e7241
Whitelist instead of blacklist for richcompare ops.
ericsnowcurrently bb8619a
Decref in the error case.
ericsnowcurrently d785573
Add a Misc/NEWS entry.
ericsnowcurrently f856035
Do not try to print a bytes object.
ericsnowcurrently ba94571
Drop some outdated comments.
ericsnowcurrently 9065001
Wrap long lines.
ericsnowcurrently 2ebc68c
Fix some compiler warnings.
ericsnowcurrently 48291af
Always use PyMem_NEW().
ericsnowcurrently 39df95e
Do not try to allocate too much space.
ericsnowcurrently b07e928
Add Windows build support.
ericsnowcurrently 1bfed0f
Drop Windows build support (for now).
ericsnowcurrently File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next
Next commit
Add the _interpreters module to the stdlib.
- Loading branch information
commit be93a283167187478ba215748f1ebb0d31dcfd14
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| :mod:`_interpreters` --- Low-level interpreters API | ||
| =================================================== | ||
|
|
||
| .. module:: _interpreters | ||
| :synopsis: Low-level interpreters API. | ||
|
|
||
| .. versionadded:: 3,7 | ||
|
|
||
| :ref:`_sub-interpreter-support` | ||
|
|
||
| threading | ||
|
|
||
| -------------- | ||
|
|
||
| This module provides low-level primitives for working with multiple | ||
| Python interpreters in the same process. | ||
|
|
||
| .. XXX The :mod:`interpreters` module provides an easier to use and | ||
| higher-level API built on top of this module. | ||
|
|
||
| This module is optional. It is provided by Python implementations which | ||
| support multiple interpreters. | ||
|
|
||
| .. XXX For systems lacking the :mod:`_interpreters` module, the | ||
| :mod:`_dummy_interpreters` module is available. It duplicates this | ||
| module's interface and can be used as a drop-in replacement. | ||
|
|
||
| It defines the following functions: | ||
|
|
||
|
|
||
| .. XXX TBD | ||
|
|
||
|
|
||
| **Caveats:** | ||
|
|
||
| * ... | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| import unittest | ||
| from test import support | ||
| interpreters = support.import_module('_interpreters') | ||
|
|
||
|
|
||
| class InterpretersTests(unittest.TestCase): | ||
| pass | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
|
|
||
| /* interpreters module */ | ||
| /* low-level access to interpreter primitives */ | ||
|
|
||
| #include "Python.h" | ||
|
|
||
|
|
||
| static PyMethodDef module_functions[] = { | ||
| {NULL, NULL} /* sentinel */ | ||
| }; | ||
|
|
||
|
|
||
| /* initialization function */ | ||
|
|
||
| PyDoc_STRVAR(module_doc, | ||
| "This module provides primitive operations to manage Python interpreters.\n\ | ||
| The 'interpreters' module provides a more convenient interface."); | ||
|
|
||
| static struct PyModuleDef interpretersmodule = { | ||
| PyModuleDef_HEAD_INIT, | ||
| "_interpreters", | ||
| module_doc, | ||
| -1, | ||
| module_functions, | ||
| NULL, | ||
| NULL, | ||
| NULL, | ||
| NULL | ||
| }; | ||
|
|
||
|
|
||
| PyMODINIT_FUNC | ||
| PyInit__interpreters(void) | ||
| { | ||
| PyObject *module; | ||
|
|
||
| module = PyModule_Create(&interpretersmodule); | ||
| if (module == NULL) | ||
| return NULL; | ||
|
|
||
|
|
||
| return module; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There should be a warning about the module being private, and/or the API being subject to change without notice, IMHO.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. For the most part I was following the precedent of the _thread module. However, in this case the module is relatively unstable still and there is no "interpreters" module yet. So a warning makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the not-yet-approved PEP, I think it would be a good idea to use a name that makes it clear that this really is intended as an API for the CPython test suite, rather than for general use.
That would mean going with something like
_xxsubinterpretersfor now.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: one consequence of that is that redistributors that remove the test suite would likely remove this module as well, and I think that's a desirable outcome: we want this as a learning tool for the further development of PEP 554 as a proposal for 3.8 (and to find and fix bugs in the subinterpreter support in 3.7), not as an inadvertent commitment to a particular API.