cmake fails to generate executable file under macOS Monterey (v12.6.1), but manages under macOS v10.15.7

I should probably change the title to: "runtime error ‘symbol not found in flat namespace’ on macOS 12 but not macOS 10.15, with both Xcode and CMake builds"

What puzzles me is the involvement of flat namespace at all. The project was created as a new project on macOS12 - and the GSL dylib does not use the flat namespace either (was checked with otool). The gsl library (v 2.71.) was installed with homebrew (v.3.6.18). The program runs fine when launched under Xcode itself, but if I try to run the built executable from Xcode on the command line, that fails with the same error as for the CMake executable. So Xcode itself is, somehow, magically able to get this linker issue to resolve.

Let’s start at the beginning. Was your main executable built with the -force_flat_namespace flag? You can check this using otool:

% otool -hv MyTool
MyTool:
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64   X86_64        ALL  0x00     EXECUTE    18       1544 DYLDLINK FORCE_FLAT PIE

Note the FORCE_FLAT flag.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi Quinn,

thanks for your reply. No, the -force_flat_namespace flag does not seem to be set:

denguesim:
Mach header
   magic cputype cpusubtype caps  filetype ncmds sizeofcmds   flags
MH_MAGIC_64  X86_64    ALL 0x00   EXECUTE  18    1640  NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE

Best, Anna

OK.

Which library is importing the _cblas_caxpy symbol? If you run nm -m on that library, what do you see for that symbol?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi Quinn,

that must be the gsl library (that is the only one I use in this project).

When I type nm -m /usr/local/Cellar/gsl/2.7.1/lib/libgsl.27.dylib

I receive 7297 lines of output - I send you the output via mail (it was too big for the forum thread).

Thanks, Anna

I receive 7297 lines of output

Right. I usually handle that using grep:

% nm -m /path/to/some.dylib | grep some_symbol

I looked at the file you sent me and the entry for _cblas_caxpy looks like this:

    (undefined) external _cblas_caxpy (dynamically looked up)

Note the absence of a library indicator on that line. In a two-level namespace library this would look something like this:

% nm -m libWaffle.dylib
…
    (undefined) external _fprintf (from libSystem)
…

indicating that fprintf is being imported from libSystem.

What does the following print?

% otool -L /usr/local/Cellar/gsl/2.7.1/lib/libgsl.27.dylib

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Right. I usually handle that using grep

Ah right - I was not aware that we are only interested in the _cblas_caxpy entry - my apologies.

When I type otool -L /usr/local/Cellar/gsl/2.7.1/lib/libgsl.27.dylib I receive:

	/usr/local/opt/gsl/lib/libgsl.27.dylib (compatibility version 28.0.0, current version 28.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)

Thanks for pointing out the absences/differences in library indicators - that's very helpful to me for following the troubleshooting.

Best, Anna

I think what’s going on here is that libgsl is used cblas_caxpy [1] but is not importing the library that defines it. Normally, with the two-level namespace, this would be a link-time error but one of the many drawbacks of the flat namespace is that it’s terrible at diagnosing such issues.

I used Xcode to build a test library that calls cblas_caxpy and this is what I see:

% nm -m libQGSL.dylib | grep cblas_caxpy
    (undefined) external _cblas_caxpy (from Accelerate)
% otool -L libQGSL.dylib 
libQGSL.dylib:
	/usr/local/lib/libQGSL.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)

This library, which Xcode built using the two-level space as is the default, imports the symbol from Accelerate and imports Accelerate as a whole to resolve that symbol.

So, you may be able to work around this problem by explicitly linking Accelerate to your main executable. That’ll bring Accelerate into your process and that might be sufficient for its symbols to be available to resolve this flat namespace link request.

There’s a lot of uncertainly in the above because I never use the flat namespace. And, honestly, that’s my advice here: Stop using the flat namespace. It has many disadvantages, including much worse performance [2].

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] This is the C name for the function. At the linker level it gets a leading underscore for historical reasons. If you’re curious as to why…

https://stackoverflow.com/a/5908708

[2] At load time. Once the program is loading and running, performance should be roughly the same.

Hi Quinn,

thanks for your explanation. I have a very general question now: How do I avoid using the flat namespace altogether? I am also a bit confused about the flat namespace: Didn't we check that -force_flat_namespace was not set? Is the flat namespace still used then?

Best, Anna

How do I avoid using the flat namespace altogether?

Avoid using libraries that are built that way.

My understanding is that libgsl was built by Homebrew. If so, you should consult its support resources to see if there’s a way to get it to build the library two-level.

Hmmm, searching the ’net it seems that this might be something specific to libgsl rather than Homebrew.

[Funnily, searching for “Homebrew flat namespace” turned up hints and tips about preventing your beer from going flat (-: ]

I am also a bit confused about the flat namespace: Didn't we check that -force_flat_namespace was not set?

Yeah, I was off down the wrong path there. -force_flat_namespace forces the executable to run in a flat namespace regardless of how the underlying libraries were built. There isn’t a flag that does the reverse.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi Quinn,

Hmmm, searching the ’net it seems that this might be something specific to libgsl rather than Homebrew.

Thanks for pointing this out. Building upon that, I was able to solve my issue by applying what you suggested in a previous post:

So, you may be able to work around this problem by explicitly linking Accelerate to your main executable. That’ll bring Accelerate into your process and that might be sufficient for its symbols to be available to resolve this flat namespace link request.

Indeed, linking Accelerates explicitly to my XCode Project resolved the libgsl flat namespace issue.

Many thanks for the great support and for taking the time to write out detailed explanations along the way - I learned a lot !

[I even know now what to google in case my beer is going flat ;) ]

Best, Anna