Build a Reliable Xcode Static Analysis Baseline on a Cloud Mac

Once a team integrates xcodebuild analyze into a cloud Mac workflow, the most common outcome is not an immediate clean bill of health, but dozens of historical diagnostics appearing at once. If the total warning count becomes the failure threshold, the pipeline may remain red indefinitely. If the logs are merely stored as artifacts, nobody consistently reviews them. A more practical approach is to establish a reviewed baseline, then hold each merge accountable only for newly introduced issues.

Pin the analysis inputs first

Static analysis depends on project settings, compilation conditions, and the source files actually included in the build. Before running the analysis, pin the Xcode selection, workspace, Scheme, configuration, and target platform. Also ensure that CI uses the same dependency lockfiles as the archive job. Do not rely on a developer’s personal Scheme or automatically select the “latest” toolchain in a script.

Use a dedicated directory for the analysis job so it does not modify DerivedData concurrently with build or test jobs:

set -o pipefail

ROOT="$PWD"
OUT="$ROOT/ci-artifacts/analyze"
DERIVED="$ROOT/.derived/analyze"
RESULT="$OUT/AppAnalyze.xcresult"

rm -rf "$OUT" "$DERIVED"
mkdir -p "$OUT" "$DERIVED"

xcodebuild analyze \
  -workspace App.xcworkspace \
  -scheme App \
  -configuration Debug \
  -destination 'generic/platform=iOS' \
  -derivedDataPath "$DERIVED" \
  -resultBundlePath "$RESULT" \
  CODE_SIGNING_ALLOWED=NO \
  | tee "$OUT/xcodebuild.log"

CODE_SIGNING_ALLOWED=NO is appropriate only for analysis targets that can complete without signing. If a project script explicitly reads signing-related build settings, fix the script’s input boundaries first instead of blindly adding more override parameters.

A baseline is not a list of issues that can be ignored. It is a snapshot of known facts during a migration. New issues should still fail immediately, while existing issues need assigned owners and a remediation plan.

Retain complete results instead of counting warnings

Running grep -c warning: is convenient, but it is not suitable as a long-term quality gate. Paths, line numbers, and log formats can change, and the same diagnostic may be printed more than once. CI should retain the complete .xcresult bundle and raw log, then generate a diagnostic inventory from the structured results for comparison.

The available xcresulttool subcommands can vary between Xcode versions. Record the help output first, and revalidate the parsing script whenever the node image is updated:

xcrun xcresulttool help > "$OUT/xcresulttool-help.txt"
xcrun xcresulttool get \
  --legacy \
  --path "$RESULT" \
  --format json > "$OUT/xcresult.json"

If the current toolchain does not accept --legacy, adjust the command according to that version’s help output, and include the parser change in the same merge request as the toolchain upgrade. Do not fall back to counting log entries and allow the job to continue when the script fails, or the quality gate will silently stop working.

Each diagnostic should retain the following fields:

Build a reviewable baseline with stable fingerprints

A diagnostic fingerprint should not include absolute paths, DerivedData paths, or a line number by itself. A more reliable combination is “issue type + repository-relative path + symbol name + normalized message.” The line number can be retained for display, but it should not be part of the primary key. Otherwise, adding one line at the top of a file would cause every existing issue to be classified as new.

Avoid over-normalization

It is reasonable to remove working-directory prefixes, temporary UUIDs, and repeated whitespace, but do not remove variable names, call names, or resource types. If two distinct defects in the same file collapse into one fingerprint, a later issue may be hidden by the existing baseline.

Use a sorted JSON Lines file for the baseline, with one record per line, and commit it to version control:

{"fingerprint":"sha256:…","type":"AnalyzeWarning","path":"Sources/Cache.swift","symbol":"load()","message":"Potential leak of an object"}

When creating the baseline for the first time, code owners should review every entry. At a minimum, they must confirm that the diagnostic comes from the current main branch, cannot be fixed immediately, has a corresponding tracking item, and is not the result of a parsing failure or duplicate record.

Compare only set differences in merge requests

After each job generates current.jsonl, compare its fingerprint set with baseline.jsonl. current - baseline represents newly introduced issues and should fail the job. baseline - current represents issues that have disappeared and should prompt maintainers to remove the corresponding baseline entries rather than retain obsolete records.

Quality-gate output should be concise and actionable. For every new issue, print its type, relative path, line number, symbol, and message, along with the artifact location of the complete xcresult. Show only a summary in the console so thousands of lines of analysis output do not obscure the actual differences.

Distinguish three failure modes

The analysis script should distinguish at least these states:

1. xcodebuild analyze itself failed, for example because a dependency is missing or the Scheme is unavailable;
2. Result parsing failed, for example because a toolchain upgrade changed the JSON structure;
3. Analysis succeeded, but found new diagnostics outside the baseline.

The first two are infrastructure or configuration errors and must not be reported as “zero new defects.” The job may pass only when both analysis and parsing succeed and the set difference is empty.

Keep shrinking the baseline

If the baseline remains unchanged for too long, it will eventually become another unmaintained ignore list. Assign owners by module and fix nearby diagnostics whenever related files are modified, or plan to remove a small number of high-risk items during each iteration. After an issue is fixed, its baseline record must also be deleted. The next run will verify whether the issue has truly disappeared.

Use the following sequence for routine checks:

• Confirm that the toolchain, Scheme, configuration, and target platform have not drifted;
• Confirm that the analysis directory is isolated from other jobs;
• Confirm that the xcresult, log, and normalized inventory were all generated;
• Spot-check fingerprints for absolute paths or temporary identifiers;
• Block merges on new diagnostics and flag disappeared diagnostics so the baseline can shrink;
• When upgrading the toolchain, rebuild the baseline on a separate branch and review the differences first.

When running this type of job on OakVM, the key is not to choose a more aggressive threshold, but to keep the toolchain, scripts, and result formats traceable on the same physical node. As long as parsing failures cannot be mistaken for success and baseline changes require review, static analysis can evolve from a long, easily ignored log into a reliable incremental quality gate.

Frequently asked questions