Xcode Cloude + SonarQube analysis + Code Coverage is working... but...

Hello guys, I was able to get code coverage and sonar-scanner working with Xcode Cloud.

  1. First in, in post-clone script I install sonar-scanner tool like this

brew install sonar-scanner

  1. Then, in post-xcodebuild script, I do three things: (a) get current app version, (b)

get app version, (c) run xcodebuild again with forced code coverage reporting, (d) find coverage data and convert it to the format that Sonarqube understands, and finally (e) run sonar-scanner, which uploads results to Sonarqube dashboard.

  cd $CI_WORKSPACE
  
  # declare variables
  SCHEME=[REMOVED]
  PRODUCT_NAME=[REMOVED]
  WORKSPACE_NAME=${PRODUCT_NAME}.xcworkspace
  APP_VERSION=$(sed -n '/MARKETING_VERSION/{s/MARKETING_VERSION = //;s/;//;s/^[[:space:]]*//;p;q;}' ./${PRODUCT_NAME}.xcodeproj/project.pbxproj)

  # clean, build and test project
  xcodebuild \
    -workspace ${WORKSPACE_NAME} \
    -destination 'platform=iOS Simulator,name=iPad (10th generation),OS=latest' \
    -scheme ${SCHEME} \
    -derivedDataPath DerivedData/ \
    -enableCodeCoverage YES \
    -resultBundlePath DerivedData/Logs/Test/ResultBundle.xcresult \
    clean build test

  # find profdata and binary
  PROFDATA=$(find . -name "Coverage.profdata")
  BINARY=$(find . -path "*${PRODUCT_NAME}.app/${PRODUCT_NAME}")

  # check if we have profdata file
  if [[ -z $PROFDATA ]]; then
  	echo "ERROR: Unable to find Coverage.profdata. Be sure to execute tests before running this script."
  	exit 1
  fi
  
  # extract coverage data from project using xcode native tool
  xcrun --run llvm-cov show -instr-profile=${PROFDATA} ${BINARY} > sonarqube-coverage.report

  # run sonar scanner and upload coverage data with the current app version
  sonar-scanner \
    -Dsonar.projectVersion=${APP_VERSION}

It all works fine but my team and I think that this is a workaround that shouldn't work like this, because technically xcodebuild command is executed two times, first by Xcode Cloud, then by my script, which takes a lot of time and feels hacky.

Ideally, Xcode Cloud's Build and Test actions should generate code coverage (the option is already enabled in our project) and give us access to profile data in DerivedData folder which I can access with environment variable CI_DERIVED_DATA_PATH. However, there is none.

My question is, do I do everything correctly? Is there a way to improve this flow? Do I miss on how to correctly get code coverage data from Xcode Cloud?

Post not yet marked as solved Up vote post of kovallux77 Down vote post of kovallux77
1.5k views

Replies

HI, were you able to use the xcode cloud code coverage data?

@kovallux77 @alejandro-vasquez-spatial After a lengthy journey, found a semi-stupid solution

No need to run xcodebuild twice. After each Test workflow, you can access the CI_RESULT_BUNDLE_PATH which will be the xcresult bundle folder(https://developer.apple.com/documentation/xcode/environment-variable-reference).

ci_post_xcodebuild.sh will be executed twice: first time it's for build-without-testing, and second time is test-without-building. Latter is our focus, so in ci_post_xcodebuild.sh, run this github script to convert XCResult to a general XML file: xccov-to-sonarqube-generic.sh XCResult > AAAAA.xml (you have to put xccov-to-sonarqube-generic.sh in ci_scripts otherwise you won't find it in other folder. there's too many catches in Xcode cloud, damn)

So now you got XML in Xcode cloud, now run cat AAAAA.xml and this will print it into logs.

After Xcode cloud is done, from your machine(don't need to be a mac), call appstoreconnect API to get your build/action/artifact and get 'downloadUrl' of it(https://developer.apple.com/documentation/appstoreconnectapi/read_xcode_cloud_artifact_information). Make to do this within maybe 30 mins, otherwise link won't be able to download without Xcode. And there's 6 or 7 zip files attached to that particular test action, what you need is something "logs" and "test-without-building'. You'll find the ci_post_xcodebuild.log inside this zip.

Use sed or awk to scrap all timestamps from it, (I believe you'll remove the first 30 characters) because a log file is the same XML but with timestamp. Now it's back to a coverage XML again. call sonar-scanner to upload it

TL;DR: at ci_post_xcodebuild.sh run xccov-to-sonarqube-generic.sh to convert XCResult to a XML, run print it out all to logs, download logs zip file through Appstore connect API, scrap all timestamps from logs, make it a XML file again. Then run your "sonar-scanner" to upload it

From Xcode cloud, the things you can download is very limited, not arbitrary product at all. Your best bet is print anything your want into logs and grab it later.

Hope this helps

Hi @3fasdf23fasdf thank you for sharing this solution. Compared to @kovallux77 approach it is more efficient in terms of saving an additional build, but hosting an additional service for every single build, for every pull request and merge to main doesn't feel like a viable solution, and a hosted service like Xcode Cloud should really offer a hook for post test handling to keep infrastructure work out of the developers way.

My hope was to use the secondary ci_post_xcodebuild.sh to check for a$CI_DERIVED_DATA_PATH/Logs/Test/*.xcresult, and if it exists invoke the sonar-scanner command once and apply that script to do the conversion. Do you think this would be viable?