Skip to content

Dotnetcore System.UInt32 conversion issues #950

Closed
Closed
Dotnetcore System.UInt32 conversion issues#950

Description

@jmlidbetter
jmlidbetter
opened on Aug 30, 2019

Environment

  • Pythonnet version: 2.4.0
  • Python version: 3.6.5
  • Operating System: Centos 7 64bit

Details

  • Creating UInt32 in python with a value greater than 0x7FFFFFFF in dotnetcore will cause an overflow error. This is due (seemingly) to the fact that .NET core clr will cast the uint to a signed long (64 bits in size on my system) before calling the PyLong_FromUsignedLong function. Using GDB and breaking before the errornous overflow we can examine the created PyLongObject:

The PyLongObject given to C# by the python interpretter that can be converted to a uint using PyLong_AsUnsignedLong

image

Using PyLong_FromUnsignedLong on the resulting uint gives this:

image

I have raised an issue on the coreclr project: Issue 26444

FYI changing the function definition from

internal static extern IntPtr PyLong_FromUnsignedLong(uint value);

to

internal static extern IntPtr PyLong_FromUnsignedLong(ulong value);

fixes the issue.

Activity

  1. filmor commented on Sep 11, 2019

    @filmor
    filmor
    on Sep 11, 2019
    Member

    Well, this is problematic. We want to be platform-independent, but C's unsigned long that is used in Python is 64bit on most 64bit Unices and 32bit on Windows, so the uint parameter is indeed wrong.

  2. Cronan commented on Sep 11, 2019

    @Cronan
    Cronan
    on Sep 11, 2019 via email
    Contributor

  3. filmor commented on Sep 18, 2019

    @filmor
    filmor
    on Sep 18, 2019
    Member

    As far as I can tell, Python is completely oblivious to this difference as it "lives" on the C platform, so all compiled modules will have the same idea of what long means. We can pretty reliably decide this by looking at the platform we are running on in .NET, though. We are already providing internally booleans for IsWindows and Is32Bit here https://github.com/pythonnet/pythonnet/blob/master/src/runtime/runtime.cs#L106-L109.

  4. jmlidbetter commented on Sep 24, 2019

    @jmlidbetter
    jmlidbetter
    on Sep 24, 2019
    ContributorAuthor

    As far as I can tell, Python is completely oblivious to this difference as it "lives" on the C platform, so all compiled modules will have the same idea of what long means. We can pretty reliably decide this by looking at the platform we are running on in .NET, though. We are already providing internally booleans for IsWindows and Is32Bit here https://github.com/pythonnet/pythonnet/blob/master/src/runtime/runtime.cs#L106-L109.

    Would this method get caught out if, for whatever reason, someone is using CPython compiled with 64bit longs on Win64 or CPython compiled with 32bit longs on 64bit Linux?

  5. jmlidbetter commented on Sep 24, 2019

    @jmlidbetter
    jmlidbetter
    on Sep 24, 2019
    ContributorAuthor

    What we really need is some hook or proxy into sizeof(long).

  6. Cronan commented on Sep 24, 2019

    @Cronan
    Cronan
    on Sep 24, 2019
    Contributor

    As per a discussion with Joe, It seems the correct way to do this would be with a call to struct.calcsize('P') which can be done in runtime.cs, in InitializePlatformData(). This gives us the bitness of the interpreter architecture, rather than the platform architecture.

  7. filmor commented on Sep 26, 2019

    @filmor
    filmor
    on Sep 26, 2019
    Member

    As far as I can tell, Python is completely oblivious to this difference as it "lives" on the C platform, so all compiled modules will have the same idea of what long means. We can pretty reliably decide this by looking at the platform we are running on in .NET, though. We are already providing internally booleans for IsWindows and Is32Bit here https://github.com/pythonnet/pythonnet/blob/master/src/runtime/runtime.cs#L106-L109.

    Would this method get caught out if, for whatever reason, someone is using CPython compiled with 64bit longs on Win64 or CPython compiled with 32bit longs on 64bit Linux?

    If you managed to compile a program that messed up, it will not be able to reliably interact with anything else on the same system. This is an ABI thing, it's not something a user can just decide.

    /edit: Note, that this is a completely different question than asking for a 32bit vs 64bit program on a given system. All operating systems that I know of have separate variants of all relevant libraries compiled in the respective fashion, but sizeof(long) is consistent in each of those environments individually.

  8. Cronan commented on Sep 26, 2019

    @Cronan
    Cronan
    on Sep 26, 2019 via email
    Contributor

  9. filmor commented on Sep 26, 2019

    @filmor
    filmor
    on Sep 26, 2019
    Member

    Yes, and the correct P/Invoke signature can be decided by looking at the IsWindows and Is32Bit values. You wouldn't be able to boot up a .NET VM (or load a libpython in case you're embedding) with non-matching bitness, so drawing those values from .NET will lead to the same result with much less work.

  10. jmlidbetter commented on Oct 2, 2019

    @jmlidbetter
    jmlidbetter
    on Oct 2, 2019
    ContributorAuthor

    Fixed in PR #961

  11. jmlidbetter
    closed this as completedon Oct 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions