I'm a core developer with the CPython project and I am researching a warning we're getting with recent version of the system compiler.
First some background:
The CPython interpreter can be extended using extension modules. Those modules are shared libraries with a known entry point based on the extension name and are loaded using dlopen(3). Currently extension modules are not linked to libpython (or a python framework), but use -undefined dynamic_lookups to look up symbols at runtime.
The major advantage for us for this is that compiled extension modules can be used with multiple different python distributions (our own installer, homebrew, anaconda, ...), which makes it possible to redistribute compiled extensions on the PyPI index. A minor advantage is that the same build procedure can be used with all types of CPython builds (statically linked, shared library, framework). A disadvantage is that some build errors are only detected at runtime (e.g. typo's in symbol names).
The issue:
With recent versions of Xcode (and the commmand line tools) we get link errors when linking extension modules:
ld: warning: -undefined dynamic_lookup may not work with chained fixups
We've been unable to come up with a good fix for this so far.
What we've looked into:
-
Use
-bundle_loader .../pythonwhile linkingThis works for static builds, but not for shared library builds even when reexporting libpython symbols in the executable.
-
Link extension modules to libpython using
@rpathThis won't work with static builds as there is no shared library to link to.
Is there a way to avoid this problem without disabling chained fixups (for which there is no documented link option)?
We're tracking this issue in #97524 in CPython's GitHub repository.