How to build projects once?

Hi,

I have been tasked with porting our C++ codebase from Windows/Linux to MacOS.

Our setup uses Premake to generate platform specific project files. On Windows it creates a Visual Studio solution and project files, on Linux it creates makefiles, and on MacOS it generates an Xcode workspace and project files.

Our project hierarchy looks something like this:

Solution/Workspace S

  • Project A (shared library)

  • Project B (shared library)

  • Project C (shared library)

  • Projects D to R (About a dozen or so - all shared libraries)

  • Project V (application)

  • Project W (application)

  • Project X (application)

  • Project Y (application)

  • Project Z (application)

Projects B and C depend on A.

Projects D to R are dependent on some or all of the projects A, B, and C.

Each application is dependent on projects A, B, and C, and some of the intermediate projects D to R. (The exact dependencies vary according to the application.) Additionally, some applications can load shared libraries dynamically according to their state/settings. I.e., not all shared library dependencies are known at build time.

Initially all building typically happens on the command line. Once our build script has run Premake to generate the project files, it proceeds to build everything. Further builds can happen within Visual Studio/Xcode as part of the normal workflow. However, building must happen on the command line for DevOps CI/Release builds.

On Windows, for example, MSBuild lets me build the entire solution for a given configuration (Debug/Release/etc) in a single command. This builds projects A, B, C in order, followed by the intermediate projects (D to R), followed by the applications (V to Z). Each project is built once.

Same deal for Linux. Make builds each project in turn according to the dependencies with a single command. Each project is build once.

On MacOS, however, xcodebuild doesn't let me just build the entire workspace for a given configuration.

I can specify a workspace, but then I have to also specify a scheme. Or I can just specify a specific project.

However, when I build a specific project, Xcodebuild always builds all dependencies.

So my build script at the moment explicitly builds projects D to R, followed by applications V to Z.

This, however, leads to the ridiculous situation of Xcode building projects in the following order:

  • D, C, B, A

  • E, C, A

  • F, C, B, A

  • G, B, A

  • etc...

  • V, F, G, H, C, B, A

  • W, G, I, C, B, A

  • X, K, C, B, A

  • etc...

With the obvious effect of our MacOS builds taking 20+ times longer than their Windows/Linux counterparts.

So, my main questions are:

  • Is it possible to just build every project in the workspace once?
  • Or if that can't be done, is it possible to not build project dependencies? I.e., can I just build the single project I specify?
  • If neither of those are possible, how can I optimise my builds so that they don't take half a day to complete?

I should point out, removing the project dependencies isn't an option. If project A is genuinely out of date, it should be built if I build one of the applications. Xcode should be smart enough to realise that it doesn't need to build everything every time.

Our setup uses Premake to generate platform specific project files. On Windows it creates a Visual Studio solution and project files, on Linux it creates makefiles, and on MacOS it generates an Xcode workspace and project files.

I suggest starting out by creating new Xcode projects directly in Xcode, and then creating a mock project that models the complexity between the different apps and modules, without the existing source code as part of that mix. This accomplishes two things:

  • Let's you learn how to configure your project relationships manually, so you get a feel of how Xcode expects you to do it out of the box, and then fine tune things to your needs
  • Removes your third-party tooling from the picture, which might have made assumptions that gets in the way of your goal

Based on what you provided, I will provide a synopsis here of the top things to look at, but I also encourage you to spend time with the Build system documentation, in case you're new to Apple's platforms and Xcode (and if so, welcome!).

Use a workspace and schemes

Since you have a lot of related projects and modules, group all of them in a workspace, as you have done. This lets all of the build products live as part of the workspace, so they can be shared.

Define your dependent relationships

For targets that are shared across Xcode projects inside the workspace, you will need to rely on implicit relationships to define the build graph. Let's take this example, slightly modified from your example so that we're precise about the configuration:

Solution/Workspace S
	Project A (shared library), containing a library target named A, (but also another named A')
	Project V (application), , containing an application target named V

In target for V, when you configure the build phases, look for the Link Libraries section, and add A to this list. This is an implicit dependency, because when Xcode builds V, it can infer that you depend on linking to A, though you didn't explicitly tell Xcode that you depend on A. When working in a workspace, references between targets defined in different projects contained by the workspace need to be implicit.

For an example of an explicit dependency, let's say A depends on another library in the same project named A'. You can form an explicit dependency that A depends on A' by adding A` to the Target Dependencies build phase. This is available in this circumstance because the dependency is defined within the same Xcode project. You'd also want to add A' to the Link Libraries build phase of A so that it's correctly linked, but that extra step of defining it in the target dependency build phase as an explicit dependency is information that Xcode can use when planning the build.

If project A is genuinely out of date, it should be built if I build one of the applications. Xcode should be smart enough to realise that it doesn't need to build everything every time.

Once you have the dependencies set up, that should be the case for incremental builds. Depending on the scope of change in a library like A when building V, Xcode can choose from not building at all (because A didn't change), to building some small parts of it, to rebuilding everything (for things like big changes).

Limit scripts in the build

Xcode can run scripts during the build, but before reaching to script something, you should look to leverage Xcode's build in features first. For example, if you need to move some files around in the build output, perhaps a Copy Files build phase is order, instead of a script.

When you do need to script something, pay close attention to making sure that you declare inputs and outputs to the script. For example, if a script generates a file as an output, make sure that file is listed in the output file list. This will let Xcode skip this build phase entirely if the file already exists in the build directory. More info in the documentation

By limiting scripts when possible, and then declaring the input and outputs, that can save a lot of time in incremental builds.

Other things to consider

I'd also suggest comparing the build settings for one of these new targets you created in Xcode for your mock project with the ones for your real project. While you may need to change some settings to suit your project's exact needs, I'd be wary of significant amounts of changes away from Xcode's default settings if that's how your real project is configured. Some of them can dramatically affect build times, like the level of optimization in a build, and Xcode's defaults here for Debug and Release configurations are often fine as is.

I'd also suggest switching to a durable set of Xcode projects that persist and are maintained over time.

— Ed Ford,  DTS Engineer

How to build projects once?
 
 
Q