Skip to content

Commit ae5b621

Browse files
msimacekmsimacek
committed
Merge branch 'master' into msimacek/GR-48809_python_3.11
2 parents e447059 + 9f18046 commit ae5b621

File tree

866 files changed

+61687
-22046
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

866 files changed

+61687
-22046
lines changed

‎.github/workflows/build-linux-aarch64-wheels.yml‎

Lines changed: 953 additions & 0 deletions
Large diffs are not rendered by default.

‎.github/workflows/build-linux-amd64-wheels.yml‎

Lines changed: 1123 additions & 0 deletions
Large diffs are not rendered by default.

‎.github/workflows/build-macos-aarch64-wheels.yml‎

Lines changed: 838 additions & 0 deletions
Large diffs are not rendered by default.

‎.github/workflows/build-macos-amd64-wheels.yml‎

Lines changed: 838 additions & 0 deletions
Large diffs are not rendered by default.

‎.github/workflows/build-repository.yml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: build-repository
2+
'on': workflow_dispatch
3+
jobs:
4+
build-repo:
5+
runs-on: ubuntu-latest
6+
steps:
7+
- name: Checkout
8+
uses: actions/checkout@v4
9+
- name: Download artifacts for linux-amd64
10+
uses: dawidd6/action-download-artifact@268677152d06ba59fcec7a7f0b5d961b6ccd7e1e
11+
continue-on-error: true
12+
with:
13+
workflow: build-linux-amd64-wheels.yml
14+
workflow_conclusion: ''
15+
if_no_artifact_found: warn
16+
allow_forks: 'false'
17+
- name: Download artifacts for linux-aarch64
18+
uses: dawidd6/action-download-artifact@268677152d06ba59fcec7a7f0b5d961b6ccd7e1e
19+
continue-on-error: true
20+
with:
21+
workflow: build-linux-aarch64-wheels.yml
22+
workflow_conclusion: ''
23+
if_no_artifact_found: warn
24+
allow_forks: 'false'
25+
- name: Download artifacts for macos-amd64
26+
uses: dawidd6/action-download-artifact@268677152d06ba59fcec7a7f0b5d961b6ccd7e1e
27+
continue-on-error: true
28+
with:
29+
workflow: build-macos-amd64-wheels.yml
30+
workflow_conclusion: ''
31+
if_no_artifact_found: warn
32+
allow_forks: 'false'
33+
- name: Download artifacts for macos-aarch64
34+
uses: dawidd6/action-download-artifact@268677152d06ba59fcec7a7f0b5d961b6ccd7e1e
35+
continue-on-error: true
36+
with:
37+
workflow: build-macos-aarch64-wheels.yml
38+
workflow_conclusion: ''
39+
if_no_artifact_found: warn
40+
allow_forks: 'false'
41+
- name: Download artifacts for windows-amd64
42+
uses: dawidd6/action-download-artifact@268677152d06ba59fcec7a7f0b5d961b6ccd7e1e
43+
continue-on-error: true
44+
with:
45+
workflow: build-windows-amd64-wheels.yml
46+
workflow_conclusion: ''
47+
if_no_artifact_found: warn
48+
allow_forks: 'false'
49+
- name: Set up Python
50+
uses: actions/setup-python@v4
51+
with:
52+
python-version: '3.10'
53+
- name: Create repository
54+
run: python ${GITHUB_WORKSPACE}/scripts/wheelbuilder/generate_repository.py
55+
- name: Store repository
56+
uses: umutozd/upload-artifact@5c459179e7745e2c730c50b10a6459da0b6f25db
57+
with:
58+
name: repository
59+
path: repository.zip
60+
if-no-files-found: error

‎.github/workflows/build-windows-amd64-wheels.yml‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
name: build-windows-amd64-wheels
2+
'on':
3+
workflow_dispatch:
4+
inputs:
5+
name:
6+
type: string
7+
description: Pkg to build (empty for all)
8+
required: false
9+
graalpy:
10+
type: string
11+
description: GraalPy download url prefix (empty for default)
12+
required: false
13+
jobs:
14+
numpy-windows-amd64:
15+
runs-on:
16+
- self-hosted
17+
- windows
18+
- X64
19+
if: ${{ !cancelled() && (inputs.name == '' || inputs.name == 'numpy') }}
20+
steps:
21+
- uses: ilammy/msvc-dev-cmd@v1
22+
- name: Install dependencies
23+
run: |
24+
Invoke-WebRequest https://kumisystems.dl.sourceforge.net/project/gnuwin32/patch/2.5.9-7/patch-2.5.9-7-bin.zip -OutFile patch.zip
25+
Expand-Archive patch.zip -DestinationPath ../patch -Force
26+
- name: Checkout
27+
uses: actions/checkout@v3
28+
- name: Setup custom GraalPy
29+
if: inputs.graalpy != ''
30+
run: |-
31+
$ProgressPreference = 'SilentlyContinue'
32+
Invoke-WebRequest ${{ inputs.graalpy }}-windows-amd64.zip -OutFile graalpy-windows-amd64.zip
33+
Expand-Archive graalpy-windows-amd64.zip
34+
mv graalpy-windows-amd64/* graalpy
35+
graalpy/bin/graalpy.exe -s -m ensurepip
36+
graalpy/bin/graalpy.exe -m pip install wheel
37+
- name: Setup GraalPy
38+
uses: actions/setup-python@main
39+
if: inputs.graalpy == ''
40+
with:
41+
python-version: graalpy23.1
42+
- name: Setup local GraalPy venv
43+
if: inputs.graalpy == ''
44+
run: python -m venv graalpy
45+
- name: Build wheel
46+
run: |-
47+
$env:PIP_FIND_LINKS=$PWD
48+
$env:PATH+=";$PWD\graalpy\bin;$PWD\graalpy\Scripts;$PWD\..\patch\bin"
49+
graalpy/bin/graalpy -m pip wheel --find-links $PWD numpy
50+
graalpy/bin/graalpy -m pip wheel --find-links $PWD numpy==1.21.6
51+
graalpy/bin/graalpy -m pip wheel --find-links $PWD numpy==1.22.4
52+
graalpy/bin/graalpy -m pip wheel --find-links $PWD numpy==1.23.1
53+
graalpy/bin/graalpy -m pip wheel --find-links $PWD numpy==1.23.4
54+
- name: Store wheels
55+
uses: umutozd/upload-artifact@5c459179e7745e2c730c50b10a6459da0b6f25db
56+
with:
57+
name: numpy-windows-amd64
58+
path: numpy*.whl
59+
if-no-files-found: error
60+
PyYAML-windows-amd64:
61+
runs-on:
62+
- self-hosted
63+
- windows
64+
- X64
65+
needs: numpy-windows-amd64
66+
if: ${{ !cancelled() && (inputs.name == '' || inputs.name == 'PyYAML') }}
67+
steps:
68+
- uses: ilammy/msvc-dev-cmd@v1
69+
- name: Install dependencies
70+
run: |
71+
Invoke-WebRequest https://kumisystems.dl.sourceforge.net/project/gnuwin32/patch/2.5.9-7/patch-2.5.9-7-bin.zip -OutFile patch.zip
72+
Expand-Archive patch.zip -DestinationPath ../patch -Force
73+
- name: Checkout
74+
uses: actions/checkout@v3
75+
- name: Setup custom GraalPy
76+
if: inputs.graalpy != ''
77+
run: |-
78+
$ProgressPreference = 'SilentlyContinue'
79+
Invoke-WebRequest ${{ inputs.graalpy }}-windows-amd64.zip -OutFile graalpy-windows-amd64.zip
80+
Expand-Archive graalpy-windows-amd64.zip
81+
mv graalpy-windows-amd64/* graalpy
82+
graalpy/bin/graalpy.exe -s -m ensurepip
83+
graalpy/bin/graalpy.exe -m pip install wheel
84+
- name: Setup GraalPy
85+
uses: actions/setup-python@main
86+
if: inputs.graalpy == ''
87+
with:
88+
python-version: graalpy23.1
89+
- name: Setup local GraalPy venv
90+
if: inputs.graalpy == ''
91+
run: python -m venv graalpy
92+
- name: Download artifacts from numpy
93+
uses: actions/download-artifact@v3
94+
continue-on-error: true
95+
with:
96+
name: numpy-windows-amd64
97+
- name: Build wheel
98+
run: |-
99+
$env:PIP_FIND_LINKS=$PWD
100+
$env:PATH+=";$PWD\graalpy\bin;$PWD\graalpy\Scripts;$PWD\..\patch\bin"
101+
graalpy/bin/graalpy -m pip wheel --find-links $PWD PyYAML
102+
- name: Store wheels
103+
uses: umutozd/upload-artifact@5c459179e7745e2c730c50b10a6459da0b6f25db
104+
with:
105+
name: PyYAML-windows-amd64
106+
path: PyYAML*.whl
107+
if-no-files-found: error
108+
ujson-windows-amd64:
109+
runs-on:
110+
- self-hosted
111+
- windows
112+
- X64
113+
needs: PyYAML-windows-amd64
114+
if: ${{ !cancelled() && (inputs.name == '' || inputs.name == 'ujson') }}
115+
steps:
116+
- uses: ilammy/msvc-dev-cmd@v1
117+
- name: Install dependencies
118+
run: |
119+
Invoke-WebRequest https://kumisystems.dl.sourceforge.net/project/gnuwin32/patch/2.5.9-7/patch-2.5.9-7-bin.zip -OutFile patch.zip
120+
Expand-Archive patch.zip -DestinationPath ../patch -Force
121+
- name: Checkout
122+
uses: actions/checkout@v3
123+
- name: Setup custom GraalPy
124+
if: inputs.graalpy != ''
125+
run: |-
126+
$ProgressPreference = 'SilentlyContinue'
127+
Invoke-WebRequest ${{ inputs.graalpy }}-windows-amd64.zip -OutFile graalpy-windows-amd64.zip
128+
Expand-Archive graalpy-windows-amd64.zip
129+
mv graalpy-windows-amd64/* graalpy
130+
graalpy/bin/graalpy.exe -s -m ensurepip
131+
graalpy/bin/graalpy.exe -m pip install wheel
132+
- name: Setup GraalPy
133+
uses: actions/setup-python@main
134+
if: inputs.graalpy == ''
135+
with:
136+
python-version: graalpy23.1
137+
- name: Setup local GraalPy venv
138+
if: inputs.graalpy == ''
139+
run: python -m venv graalpy
140+
- name: Download artifacts from PyYAML
141+
uses: actions/download-artifact@v3
142+
continue-on-error: true
143+
with:
144+
name: PyYAML-windows-amd64
145+
- name: Build wheel
146+
run: |-
147+
$env:PIP_FIND_LINKS=$PWD
148+
$env:PATH+=";$PWD\graalpy\bin;$PWD\graalpy\Scripts;$PWD\..\patch\bin"
149+
graalpy/bin/graalpy -m pip wheel --find-links $PWD ujson
150+
- name: Store wheels
151+
uses: umutozd/upload-artifact@5c459179e7745e2c730c50b10a6459da0b6f25db
152+
with:
153+
name: ujson-windows-amd64
154+
path: ujson*.whl
155+
if-no-files-found: error

‎.gitignore‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,11 @@ GRAALPYTHON.dist
2222
GRAALPYTHON_UNIT_TESTS.dist
2323
mx.graalpython/eclipse-launches
2424
*.json
25+
!jbang-catalog.json
2526
!**/resources/*.json
2627
!**/META-INF/**/*.json
27-
!graalpython/lib-graalpython/modules/standalone/shared/native-image-resources.json
28-
!graalpython/lib-graalpython/modules/standalone/shared/native-image-proxy-configuration.json
28+
!graalpython/lib-graalpython/modules/standalone/native-image-resources.json
29+
!graalpython/graalpy-archetype/src/main/resources/archetype-resources/native-image-proxy-configuration.json
2930
language
3031
*.bc
3132
*.iml
@@ -85,3 +86,7 @@ graalpython.src.zip.files/
8586
graalpython/com.oracle.graal.python.test/src/tests/patched_package/build/
8687
graalpython/com.oracle.graal.python.test/src/tests/patched_package/src/patched_package.egg-info
8788
graalpython/com.oracle.graal.python.test.integration/target
89+
graalpython/graalpy-maven-plugin/target
90+
graalpython/graalpy-archetype-polyglot-app/target
91+
graalpython/graalpy-archetype-polyglot-app/target
92+
pom-mx.xml

‎CHANGELOG.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,28 @@
33
This changelog summarizes major changes between GraalVM versions of the Python
44
language runtime. The main focus is on user-observable behavior of the engine.
55

6+
## Version 24.1.0
7+
* We now provide intrinsified `_pickle` module also in the community version.
8+
9+
## Version 24.0.0
10+
* We now provide a collection of recipes in the form of GitHub Actions to build popular native extensions on GraalPy. These provide a reproducible way for the community to build native extensions for GraalPy with the correct dependencies. See scripts/wheelbuilder/README.md for details.
11+
* Foreign big integers are now supported and work with all `Numeric` operators.
12+
* Interop `null` values are now treated as *identical*, not only *equal* to Python's `None`. This means that something like `java.type("java.lang.Object[]")(1)[0] is None` will now return `True`.
13+
* Update to Python 3.10.13. This inlines the security and bugfixes from 3.10.8 to 3.10.13.
14+
* Include the GraalPy C API revision in the ABI tag for wheels. This avoids accidentally using incompatible binaries when using snapshots.
15+
* Honor the `allowHostSocketAccess` configuration in embeddings. This means sockets can now be disabled independently of other IO.
16+
* Avoid eager initialization of the Sulong LLVM runtime. This reduces footprint in the default configuration where C extensions are run natively.
17+
* Expand support for the following modules: llvmlite, pydantic-core, catboost, ray, tensorflow, tensorflow-io, readme-renderer, safetensors, keras, pybind11, protbuf, grpcio, PyO3, cryptography, bcrypt, cramjam, libcst, orjson, rpds_py.
18+
* Support installing some packages with native extensions on Windows. Simple packages like `ujson` or `kiwisolver` will now work when installed from a venv inside a Visual Studio command prompt.
19+
* Raise `KeyboardInterrupt` exception when the process is interrupted. Enabled only when using the launcher.
20+
* Add option `python.InitialLocale` to change the default locale. If not set, then Java Locale#getDefault is used.
21+
* `multiprocessing` module now uses the `spawn` method (creates new processes) by default. The formerly default method that uses threads and multiple Truffle contexts can be selected using `multiprocessing.set_start_method('graalpy')`.
22+
* `polyglot` module: add API to redefine Truffle interop messages for external / user defined types. For more details see [The Truffle Interoperability Extension API](docs/user/Interoperability.md).
23+
*Adding integration with jBang (https://www.jbang.dev/)
24+
** running example via `jbang hello@oracle/graalpython` or `jbang hello@oracle/graalpython "print(1*4)"`
25+
** creating new script via: `jbang init --template=graalpy@oracle/graalpython myscript.java`
26+
** creating new script with local maven repo for testing: `jbang init --template=graalpy_local_repo@oracle/graalpython -Dpath_to_local_repo=/absolute/path/to/local/maven/repository myscript.java'
27+
628
## Version 23.1.0
729
* Oracle GraalPy distributions (previously known as GraalPy Enterprise) are now available under the [GFTC license](https://www.oracle.com/downloads/licenses/graal-free-license.html). The community builds published on Github have been renamed to `graalpy-community-<version>-<os>-<arch>.tar.gz`.
830
* Add support for the sqlite3 module. This allows many packages like `coverage` or `Flask-SQLAlchemy` to work on top of this embedded database.

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ We welcome issue reports of all kinds and are working hard to close our compatib
1212

1313
The easiest option to try GraalPy is [pyenv](https://github.com/pyenv/pyenv/), the Python version manager.
1414
It allows you to easily install different GraalPy releases.
15-
To install version 22.3.0, for example, just run `pyenv install graalpy-22.3.0`.
15+
To install version 23.1.0, for example, just run `pyenv install graalpy-community-23.1.0`.
1616

1717
Another option is to use [Conda-Forge](https://conda-forge.org/).
1818
To get an environment with the latest version of GraalPy, use the following command:

‎bisect-benchmark.ini‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,14 @@ build_command = mx --dy /compiler build
2929
# "-XX ..." list is not part of the command, that's already part of the output
3030
benchmark_command = mx --dy /compiler benchmark micro:try-except-simple
3131

32+
33+
# If the benchmark command runs multiple benchmarks, then the results file
34+
# will contain all their results and the tool would not know which one to
35+
# choose. In particular, this is the case of some of the pyperformance
36+
# benchmarks. Use the following option to filter the results by benchmark name:
37+
#
38+
# benchmark_name = scimark_lu
39+
3240
# Once the good and bad commits are identified, the tool will run all the
3341
# following benchmark commands on both the good and bad revisions. One can
3442
# run the benchmarks again with some extra logging, IGV dumping, ...
@@ -61,7 +69,8 @@ good = 1234cafebabe
6169
enterprise = true
6270

6371
# Which metric should be used for comparisons, corresponds to "metric.name" in
64-
# the benchmark data.
72+
# the benchmark data. If there are multiple entries of given metric, then the
73+
# last one is used.
6574
# A special value "WORKS" can be used to bisect a benchmark that fails to
6675
# execute to find when it started to fail.
6776
benchmark_metric = time

0 commit comments

Comments
 (0)