Skip to content

bpo-41100: ctypes fixes for arm64 Mac OS #21249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lawrence-danna-apple wants to merge 10 commits into from
Closed

bpo-41100: ctypes fixes for arm64 Mac OS #21249

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
ff883bc
use system libffi for Mac OS 10.15 and up
lawrence-danna-apple Jun 30, 2020
36b35bc
ctypes: use the correct ABI for variadic functions
lawrence-danna-apple Jun 30, 2020
bacf128
ctypes: probe libffi for ffi_closure_alloc and ffi_prep_cif_var
lawrence-danna-apple Jul 1, 2020
971ef00
📜🤖 Added by blurb_it.
blurb-it[bot] Jul 1, 2020
da30587
fix spelling in Doc/library/ctypes.rst
lawrence-danna-apple Jul 13, 2020
80f852b
use system libffi for older releases of Mac OS
lawrence-danna-apple Jul 15, 2020
aeac14b
remove FUNCFLAG_VARIADIC
lawrence-danna-apple Jul 15, 2020
62fe20a
fixup! use system libffi for older releases of Mac OS
lawrence-danna-apple Jul 16, 2020
e160a59
fixup! use system libffi for older releases of Mac OS
lawrence-danna-apple Jul 16, 2020
bca75fd
Fix build failure on Mac OS 10.10 / Xcode 6, remove libffi_osx
lawrence-danna-apple Jul 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
use system libffi for Mac OS 10.15 and up 
This updates setup.py to default to the system libffi on Mac OS 10.15 and forward.   It also updates
detect_ctypes to prefer finding libffi in the Xcode SDK, rather than /usr/include
  • Loading branch information
@lawrence-danna-apple
lawrence-danna-apple committed Jul 1, 2020
commit ff883bc5d7d8968f37cf971ac63071a4b85c8190
35 changes: 27 additions & 8 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,13 @@ def macosx_sdk_specified():
macosx_sdk_root()
return MACOS_SDK_SPECIFIED

def is_macosx_at_least(vers):
if MACOS:
dep_target = sysconfig.get_config_var('MACOSX_DEPLOYMENT_TARGET')
if dep_target:
return tuple(map(int, dep_target.split('.'))) >= vers
return False


def is_macosx_sdk_path(path):
"""
Expand Down Expand Up @@ -2136,7 +2143,12 @@ def configure_ctypes(self, ext):

def detect_ctypes(self):
# Thomas Heller's _ctypes module
self.use_system_libffi = False

if not sysconfig.get_config_var("LIBFFI_INCLUDEDIR") and is_macosx_at_least((10,15)):
self.use_system_libffi = True
else:
self.use_system_libffi = '--with-system-ffi' in sysconfig.get_config_var("CONFIG_ARGS")

Copy link
Copy Markdown
Contributor

@ronaldoussoren ronaldoussoren Jul 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to just drop "use_system_libffi" and unconditionally use the system headers.

Copy link
Copy Markdown
Contributor Author

@lawrence-danna-apple lawrence-danna-apple Jul 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

include_dirs = []
extra_compile_args = ['-DPy_BUILD_CORE_MODULE']
extra_link_args = []
Expand Down Expand Up @@ -2183,24 +2195,31 @@ def detect_ctypes(self):
sources=['_ctypes/_ctypes_test.c'],
libraries=['m']))

ffi_inc = [sysconfig.get_config_var("LIBFFI_INCLUDEDIR")]
ffi_lib = None

ffi_inc_dirs = self.inc_dirs.copy()
if MACOS:
if '--with-system-ffi' not in sysconfig.get_config_var("CONFIG_ARGS"):
if not self.use_system_libffi:
return
# OS X 10.5 comes with libffi.dylib; the include files are
# in /usr/include/ffi
ffi_inc_dirs.append('/usr/include/ffi')
ffi_in_sdk = os.path.join(macosx_sdk_root(), "usr/include/ffi")
Comment on lines +2153 to +2154
Copy link
Copy Markdown
Contributor

@jmroot jmroot Sep 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please bear in mind that Python may be built against an external libffi on macOS that is not the OS-provided version. In MacPorts we build against our port of libffi 3.3 on all OS versions.

Copy link
Copy Markdown
Contributor Author

@lawrence-danna-apple lawrence-danna-apple Sep 20, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment by @ronaldoussoren who prefers to unconditionally use the system's libffi. I updated this PR accordingly. Is there any reason not to use the system libffi anymore? I tested going back to 10.10 i think for the other PR adding availability checks. What's the need to support alternative libffi these days?

Copy link
Copy Markdown
Contributor

@jmroot jmroot Sep 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reasons as any other library: the OS often ships an older version which naturally lacks the features and bug fixes of later versions.

if os.path.exists(ffi_in_sdk):
ffi_inc = [ffi_in_sdk]
ffi_lib = 'ffi'
sources.remove('_ctypes/malloc_closure.c')
else:
# OS X 10.5 comes with libffi.dylib; the include files are
# in /usr/include/ffi
ffi_inc_dirs.append('/usr/include/ffi')

ffi_inc = [sysconfig.get_config_var("LIBFFI_INCLUDEDIR")]
if not ffi_inc or ffi_inc[0] == '':
ffi_inc = find_file('ffi.h', [], ffi_inc_dirs)
if ffi_inc is not None:
ffi_h = ffi_inc[0] + '/ffi.h'
if not os.path.exists(ffi_h):
ffi_inc = None
print('Header file {} does not exist'.format(ffi_h))
ffi_lib = None
if ffi_inc is not None:
if ffi_lib is None and ffi_inc is not None:
for lib_name in ('ffi', 'ffi_pic'):
if (self.compiler.find_library_file(self.lib_dirs, lib_name)):
ffi_lib = lib_name
Expand Down