Skip to content

Commit 37c4426

Browse files
BadSingletonBadSingleton
authored
Merge pull request #15 from Unity-Technologies/last-ditch-serialization
Last line of defense, best effort serialization Also updates the base version of pythonnet.
2 parents 9d0bd9c + 0cc7069 commit 37c4426

File tree

329 files changed

+20108
-16000
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

329 files changed

+20108
-16000
lines changed

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,6 @@ Check all those that are applicable and complete.
1616

1717
- [ ] Make sure to include one or more tests for your change
1818
- [ ] If an enhancement PR, please create docs and at best an example
19+
- [ ] Ensure you have signed the [.NET Foundation CLA](https://cla.dotnetfoundation.org/pythonnet/pythonnet)
1920
- [ ] Add yourself to [`AUTHORS`](../blob/master/AUTHORS.md)
2021
- [ ] Updated the [`CHANGELOG`](../blob/master/CHANGELOG.md)

‎.github/workflows/ARM.yml‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: Main (ARM)
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
pull_request:
8+
9+
jobs:
10+
build-test-arm:
11+
name: Build and Test ARM64
12+
runs-on: [self-hosted, linux, ARM64]
13+
timeout-minutes: 15
14+
15+
steps:
16+
- name: Checkout code
17+
uses: actions/checkout@v2
18+
19+
- name: Setup .NET
20+
uses: actions/setup-dotnet@v1
21+
with:
22+
dotnet-version: '6.0.x'
23+
24+
- name: Clean previous install
25+
run: |
26+
pip uninstall -y pythonnet
27+
28+
- name: Install dependencies
29+
run: |
30+
pip install --upgrade -r requirements.txt
31+
pip install pytest numpy # for tests
32+
33+
- name: Build and Install
34+
run: |
35+
pip install -v .
36+
37+
- name: Set Python DLL path (non Windows)
38+
run: |
39+
echo PYTHONNET_PYDLL=$(python -m find_libpython) >> $GITHUB_ENV
40+
41+
- name: Embedding tests
42+
run: dotnet test --logger "console;verbosity=detailed" src/embed_tests/
43+
44+
- name: Python Tests (Mono)
45+
run: python -m pytest --runtime mono
46+
47+
- name: Python Tests (.NET Core)
48+
run: python -m pytest --runtime coreclr
49+
50+
- name: Python tests run from .NET
51+
run: dotnet test src/python_tests_runner/
52+
53+
#- name: Perf tests
54+
# run: |
55+
# pip install --force --no-deps --target src/perf_tests/baseline/ pythonnet==2.5.2
56+
# dotnet test --configuration Release --logger "console;verbosity=detailed" src/perf_tests/

‎.github/workflows/main.yml‎

Lines changed: 48 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,28 @@
1-
name: GitHub Actions
1+
name: Main
22

3-
on: [ pull_request, push ]
3+
on:
4+
push:
5+
branches:
6+
- master
7+
pull_request:
48

59
jobs:
610
build-test:
711
name: Build and Test
812
runs-on: ${{ matrix.os }}-latest
9-
timeout-minutes: 5
13+
timeout-minutes: 15
1014

1115
strategy:
1216
fail-fast: false
1317
matrix:
1418
os: [windows, ubuntu, macos]
15-
python: [3.6, 3.7, 3.8, 3.9]
16-
platform: [x64]
17-
shutdown_mode: [Normal, Soft]
18-
19-
env:
20-
PYTHONNET_SHUTDOWN_MODE: ${{ matrix.SHUTDOWN_MODE }}
19+
python: ["3.7", "3.8", "3.9", "3.10"]
20+
platform: [x64, x86]
21+
exclude:
22+
- os: ubuntu
23+
platform: x86
24+
- os: macos
25+
platform: x86
2126

2227
steps:
2328
- name: Set Environment on macOS
@@ -31,6 +36,8 @@ jobs:
3136

3237
- name: Setup .NET
3338
uses: actions/setup-dotnet@v1
39+
with:
40+
dotnet-version: '6.0.x'
3441

3542
- name: Set up Python ${{ matrix.python }}
3643
uses: actions/setup-python@v2
@@ -41,22 +48,47 @@ jobs:
4148
- name: Install dependencies
4249
run: |
4350
pip install --upgrade -r requirements.txt
51+
pip install numpy # for tests
4452
4553
- name: Build and Install
4654
run: |
47-
python setup.py configure
4855
pip install -v .
4956
50-
- name: Python Tests
51-
run: pytest
57+
- name: Set Python DLL path (non Windows)
58+
if: ${{ matrix.os != 'windows' }}
59+
run: |
60+
echo PYTHONNET_PYDLL=$(python -m find_libpython) >> $GITHUB_ENV
61+
62+
- name: Set Python DLL path (Windows)
63+
if: ${{ matrix.os == 'windows' }}
64+
run: |
65+
Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append -InputObject "PYTHONNET_PYDLL=$(python -m find_libpython)"
5266
5367
- name: Embedding tests
54-
run: dotnet test --runtime any-${{ matrix.platform }} src/embed_tests/
55-
if: ${{ matrix.os != 'macos' }} # Not working right now, doesn't find libpython
68+
run: dotnet test --runtime any-${{ matrix.platform }} --logger "console;verbosity=detailed" src/embed_tests/
69+
env:
70+
MONO_THREADS_SUSPEND: preemptive # https://github.com/mono/mono/issues/21466
71+
72+
- name: Python Tests (Mono)
73+
if: ${{ matrix.os != 'windows' }}
74+
run: pytest --runtime mono
75+
76+
# TODO: Run these tests on Windows x86
77+
- name: Python Tests (.NET Core)
78+
if: ${{ matrix.platform == 'x64' }}
79+
run: pytest --runtime coreclr
80+
81+
- name: Python Tests (.NET Framework)
82+
if: ${{ matrix.os == 'windows' }}
83+
run: pytest --runtime netfx
5684

5785
- name: Python tests run from .NET
5886
run: dotnet test --runtime any-${{ matrix.platform }} src/python_tests_runner/
59-
if: ${{ matrix.os == 'windows' }} # Not working for others right now
6087

61-
# TODO: Run perf tests
88+
- name: Perf tests
89+
if: ${{ (matrix.python == '3.8') && (matrix.platform == 'x64') }}
90+
run: |
91+
pip install --force --no-deps --target src/perf_tests/baseline/ pythonnet==2.5.2
92+
dotnet test --configuration Release --runtime any-${{ matrix.platform }} --logger "console;verbosity=detailed" src/perf_tests/
93+
6294
# TODO: Run mono tests on Windows?

‎.github/workflows/nuget-preview.yml‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
name: NuGet Preview Release
2+
3+
on:
4+
schedule:
5+
- cron: "5 4 3 */1 *" # once a month, at 4:05 on 3rd
6+
workflow_dispatch:
7+
8+
jobs:
9+
release:
10+
name: Release Preview
11+
runs-on: ubuntu-latest
12+
environment: NuGet
13+
timeout-minutes: 10
14+
15+
env:
16+
PYTHONNET_SHUTDOWN_MODE: Normal
17+
18+
steps:
19+
- name: Get Date
20+
run: |
21+
echo "DATE_VER=$(date "+%Y-%m-%d")" >> $GITHUB_ENV
22+
23+
- name: Checkout code
24+
uses: actions/checkout@v2
25+
26+
- name: Setup .NET
27+
uses: actions/setup-dotnet@v1
28+
with:
29+
dotnet-version: '6.0.x'
30+
31+
- name: Set up Python 3.8
32+
uses: actions/setup-python@v2
33+
with:
34+
python-version: 3.8
35+
architecture: x64
36+
37+
- name: Install dependencies
38+
run: |
39+
pip install --upgrade -r requirements.txt
40+
pip install numpy # for tests
41+
42+
- name: Build and Install
43+
run: |
44+
pip install -v .
45+
46+
- name: Set Python DLL path (non Windows)
47+
if: ${{ matrix.os != 'windows' }}
48+
run: |
49+
echo PYTHONNET_PYDLL=$(python -m find_libpython) >> $GITHUB_ENV
50+
51+
- name: Python Tests
52+
run: pytest
53+
54+
- name: Embedding tests
55+
run: dotnet test --runtime any-ubuntu src/embed_tests/
56+
57+
- name: Pack
58+
run: dotnet pack --configuration Release --version-suffix preview${{env.DATE_VER}} --output "Release-Preview"
59+
60+
- name: Publish NuGet
61+
run: |
62+
dotnet nuget push --source https://api.nuget.org/v3/index.json --api-key ${{ secrets.NUGET_MONTHLY }} Release-Preview/*.nupkg
63+
dotnet nuget push --skip-duplicate --source https://api.nuget.org/v3/index.json --api-key ${{ secrets.NUGET_MONTHLY }} Release-Preview/*.snupkg

‎.gitignore‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
/src/runtime/interopNative.cs
2-
3-
# Configuration data
4-
configured.props
2+
/src/perf_tests/baseline/
53

64
# General binaries and Build results
75
*.dll

‎AUTHORS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@
5555
- Meinrad Recheis ([@henon](https://github.com/henon))
5656
- Mohamed Koubaa ([@koubaa](https://github.com/koubaa))
5757
- Patrick Stewart ([@patstew](https://github.com/patstew))
58+
- Peter Kese ([@pkese](https://github.com/pkese))
5859
- Raphael Nestler ([@rnestler](https://github.com/rnestler))
5960
- Rickard Holmberg ([@rickardraysearch](https://github.com/rickardraysearch))
6061
- Sam Winstanley ([@swinstanley](https://github.com/swinstanley))
@@ -83,3 +84,4 @@
8384
- ([@DanBarzilian](https://github.com/DanBarzilian))
8485
- ([@alxnull](https://github.com/alxnull))
8586
- ([@gpetrou](https://github.com/gpetrou))
87+
- Ehsan Iran-Nejad ([@eirannejad](https://github.com/eirannejad))

‎CHANGELOG.md‎

Lines changed: 96 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -12,47 +12,124 @@ This document follows the conventions laid out in [Keep a CHANGELOG][].
1212
- Ability to instantiate new .NET arrays using `Array[T](dim1, dim2, ...)` syntax
1313
- Python operator method will call C# operator method for supported binary and unary operators ([#1324][p1324]).
1414
- Add GetPythonThreadID and Interrupt methods in PythonEngine
15+
- Ability to implement delegates with `ref` and `out` parameters in Python, by returning the modified parameter values in a tuple. ([#1355][i1355])
16+
- Ability to override .NET methods that have `out` or `ref` in Python by returning the modified parameter values in a tuple. ([#1481][i1481])
17+
- `PyType` - a wrapper for Python type objects, that also permits creating new heap types from `TypeSpec`
18+
- Improved exception handling:
19+
* exceptions can now be converted with codecs
20+
* `InnerException` and `__cause__` are propagated properly
21+
- `__name__` and `__signature__` to reflected .NET methods
22+
- .NET collection types now implement standard Python collection interfaces from `collections.abc`.
23+
See [Mixins/collections.py](src/runtime/Mixins/collections.py).
24+
- you can cast objects to generic .NET interfaces without specifying generic arguments as long as there is no ambiguity.
25+
- .NET arrays implement Python buffer protocol
26+
- Python integer interoperability with `System.Numerics.BigInteger`
27+
- Python.NET will correctly resolve .NET methods, that accept `PyList`, `PyInt`,
28+
and other `PyObject` derived types when called from Python.
29+
- .NET classes, that have `__call__` method are callable from Python
30+
- `PyIterable` type, that wraps any iterable object in Python
31+
- `PythonEngine` properties for supported Python versions: `MinSupportedVersion`, `MaxSupportedVersion`, and `IsSupportedVersion`
32+
- The runtime that is loaded on `import clr` can now be configured via environment variables
33+
1534

1635
### Changed
17-
- Drop support for Python 2, 3.4, and 3.5
36+
- Drop support for Python 2, 3.4, 3.5, and 3.6
1837
- `wchar_t` size aka `Runtime.UCS` is now determined at runtime
1938
- `clr.AddReference` may now throw errors besides `FileNotFoundException`, that provide more
2039
details about the cause of the failure
2140
- `clr.AddReference` no longer adds ".dll" implicitly
2241
- `PyIter(PyObject)` constructor replaced with static `PyIter.GetIter(PyObject)` method
42+
- Python runtime can no longer be shut down if the Python error indicator is set, as it would have unpredictable behavior
2343
- BREAKING: Return values from .NET methods that return an interface are now automatically
2444
wrapped in that interface. This is a breaking change for users that rely on being
2545
able to access members that are part of the implementation class, but not the
26-
interface. Use the new __implementation__ or __raw_implementation__ properties to
46+
interface. Use the new `__implementation__` or `__raw_implementation__` properties to
2747
if you need to "downcast" to the implementation class.
48+
- BREAKING: `==` and `!=` operators on `PyObject` instances now use Python comparison
49+
(previously was equivalent to `object.ReferenceEquals(,)`)
2850
- BREAKING: Parameters marked with `ParameterAttributes.Out` are no longer returned in addition
2951
to the regular method return value (unless they are passed with `ref` or `out` keyword).
3052
- BREAKING: Drop support for the long-deprecated CLR.* prefix.
3153
- `PyObject` now implements `IEnumerable<PyObject>` in addition to `IEnumerable`
3254
- floating point values passed from Python are no longer silently truncated
3355
when .NET expects an integer [#1342][i1342]
56+
- More specific error messages for method argument mismatch
57+
- members of `PyObject` inherited from `System.Object and `DynamicObject` now autoacquire GIL
58+
- BREAKING: when inheriting from .NET types in Python if you override `__init__` you
59+
must explicitly call base constructor using `super().__init__(.....)`. Not doing so will lead
60+
to undefined behavior.
61+
- BREAKING: most `PyScope` methods will never return `null`. Instead, `PyObject` `None` will be returned.
62+
- BREAKING: `PyScope` was renamed to `PyModule`
63+
- BREAKING: Methods with `ref` or `out` parameters and void return type return a tuple of only the `ref` and `out` parameters.
64+
- BREAKING: to call Python from .NET `Runtime.PythonDLL` property must be set to Python DLL name
65+
or the DLL must be loaded in advance. This must be done before calling any other Python.NET functions.
66+
- BREAKING: `PyObject.Length()` now raises a `PythonException` when object does not support a concept of length.
67+
- BREAKING: disabled implicit conversion from C# enums to Python `int` and back.
68+
One must now either use enum members (e.g. `MyEnum.Option`), or use enum constructor
69+
(e.g. `MyEnum(42)` or `MyEnum(42, True)` when `MyEnum` does not have a member with value 42).
70+
- Sign Runtime DLL with a strong name
71+
- Implement loading through `clr_loader` instead of the included `ClrModule`, enables
72+
support for .NET Core
73+
- BREAKING: .NET and Python exceptions are preserved when crossing Python/.NET boundary
74+
- BREAKING: custom encoders are no longer called for instances of `System.Type`
75+
- `PythonException.Restore` no longer clears `PythonException` instance.
76+
- Replaced the old `__import__` hook hack with a PEP302-style Meta Path Loader
77+
- BREAKING: Names of .NET types (e.g. `str(__class__)`) changed to better support generic types
78+
- BREAKING: overload resolution will no longer prefer basic types. Instead, first matching overload will
79+
be chosen.
80+
- BREAKING: acquiring GIL using `Py.GIL` no longer forces `PythonEngine` to initialize
81+
- BREAKING: `Exec` and `Eval` from `PythonEngine` no longer accept raw pointers.
82+
- BREAKING: .NET collections and arrays are no longer automatically converted to
83+
Python collections. Instead, they implement standard Python
84+
collection interfaces from `collections.abc`.
85+
See [Mixins/collections.py](src/runtime/Mixins/collections.py).
86+
- BREAKING: When trying to convert Python `int` to `System.Object`, result will
87+
be of type `PyInt` instead of `System.Int32` due to possible loss of information.
88+
Python `float` will continue to be converted to `System.Double`.
89+
- BREAKING: Python.NET will no longer implicitly convert types like `numpy.float64`, that implement `__float__` to
90+
`System.Single` and `System.Double`. An explicit conversion is required on Python or .NET side.
91+
- BREAKING: `PyObject.GetHashCode` can fail.
92+
- BREAKING: Python.NET will no longer implicitly convert any Python object to `System.Boolean`.
93+
- BREAKING: `PyObject.GetAttr(name, default)` now only ignores `AttributeError` (previously ignored all exceptions).
94+
- BREAKING: `PyObject` no longer implements `IEnumerable<PyObject>`.
95+
Instead, `PyIterable` does that.
96+
- BREAKING: `IPyObjectDecoder.CanDecode` `objectType` parameter type changed from `PyObject` to `PyType`
3497

3598
### Fixed
3699

37-
- Fix incorrect dereference of wrapper object in `tp_repr`, which may result in a program crash
38-
- Fix incorrect dereference in params array handling
39-
- Fixes issue with function resolution when calling overloaded function with keyword arguments from python ([#1097][i1097])
40-
- Fix `object[]` parameters taking precedence when should not in overload resolution
41-
- Fixed a bug where all .NET class instances were considered Iterable
42-
- Fix incorrect choice of method to invoke when using keyword arguments.
43-
- Fix non-delegate types incorrectly appearing as callable.
44-
- Indexers can now be used with interface objects
45-
- Fixed a bug where indexers could not be used if they were inherited
46-
- Made it possible to use `__len__` also on `ICollection<>` interface objects
47-
- Fixed issue when calling PythonException.Format where another exception would be raise for unnormalized exceptions
48-
- Made it possible to call `ToString`, `GetHashCode`, and `GetType` on inteface objects
49-
- Fixed objects returned by enumerating `PyObject` being disposed too soon
50-
- Incorrectly using a non-generic type with type parameters now produces a helpful Python error instead of throwing NullReferenceException
100+
- Fix incorrect dereference of wrapper object in `tp_repr`, which may result in a program crash
101+
- Fixed parameterless .NET constructor being silently called when a matching constructor overload is not found ([#238][i238])
102+
- Fix incorrect dereference in params array handling
103+
- Fixes issue with function resolution when calling overloaded function with keyword arguments from python ([#1097][i1097])
104+
- Fix `object[]` parameters taking precedence when should not in overload resolution
105+
- Fixed a bug where all .NET class instances were considered Iterable
106+
- Fix incorrect choice of method to invoke when using keyword arguments.
107+
- Fix non-delegate types incorrectly appearing as callable.
108+
- Indexers can now be used with interface objects
109+
- Fixed a bug where indexers could not be used if they were inherited
110+
- Made it possible to use `__len__` also on `ICollection<>` interface objects
111+
- Fixed issue when calling PythonException.Format where another exception would be raise for unnormalized exceptions
112+
- Made it possible to call `ToString`, `GetHashCode`, and `GetType` on inteface objects
113+
- Fixed objects returned by enumerating `PyObject` being disposed too soon
114+
- Incorrectly using a non-generic type with type parameters now produces a helpful Python error instead of throwing NullReferenceException
51115
- `import` may now raise errors with more detail than "No module named X"
116+
- Exception stacktraces on `PythonException.StackTrace` are now properly formatted
117+
- Providing an invalid type parameter to a generic type or method produces a helpful Python error
118+
- Empty parameter names (as can be generated from F#) do not cause crashes
119+
- Unicode strings with surrogates were truncated when converting from Python
120+
- `Reload` mode now supports generic methods (previously Python would stop seeing them after reload)
121+
- Temporarily fixed issue resolving method overload when method signature has `out` parameters ([#1672](i1672))
122+
- Decimal default parameters are now correctly taken into account
52123

53124
### Removed
54125

126+
- `ShutdownMode` has been removed. The only shutdown mode supported now is an equivalent of `ShutdownMode.Reload`.
127+
There is no need to specify it.
55128
- implicit assembly loading (you have to explicitly `clr.AddReference` before doing import)
129+
- messages in `PythonException` no longer start with exception type
130+
- `PyScopeManager`, `PyScopeException`, `PyScope` (use `PyModule` instead)
131+
- support for .NET Framework 4.0-4.6; Mono before 5.4. Python.NET now requires .NET Standard 2.0
132+
(see [the matrix](https://docs.microsoft.com/en-us/dotnet/standard/net-standard#net-implementation-support))
56133

57134
## [2.5.0][] - 2020-06-14
58135

@@ -813,3 +890,6 @@ This version improves performance on benchmarks significantly compared to 2.3.
813890
[p534]: https://github.com/pythonnet/pythonnet/pull/534
814891
[i449]: https://github.com/pythonnet/pythonnet/issues/449
815892
[i1342]: https://github.com/pythonnet/pythonnet/issues/1342
893+
[i238]: https://github.com/pythonnet/pythonnet/issues/238
894+
[i1481]: https://github.com/pythonnet/pythonnet/issues/1481
895+
[i1672]: https://github.com/pythonnet/pythonnet/pull/1672

0 commit comments

Comments
 (0)