HomeFeaturesDocumentationContactDownload
Browse the docs

Quality gates in CI

Use the sitecmd CLI to fail a build when site health regresses. Examples for GitHub Actions, GitLab CI, CircleCI, and more.

A quality gate is a check that fails your build if your site is in worse shape than you'll accept. SiteCMD's CLI gives you three exit-code-driven gates: a free local Web Scan score gate, a free local Code Scan severity gate, and a connected Code Scan gate that compares a checkout with the site's shared baseline.

The local sitecmd scan gate runs live-site checks. Local sitecmd audit runs the full Code Scan against the checkout. sitecmd gate runs the same source audit and asks the connected service which findings are new against its shared baseline. Default CLI binaries omit browser analysis; supported desktop platforms, hosted scans, and optional CLI source builds with a browser provide that separate coverage.

This page is the recipe collection. For the underlying CLI behavior, see CLI reference.

The basic gate

test -f .sitecmd/config.json || sitecmd init https://staging.example.com --yes --no-deep-link
sitecmd scan --url https://staging.example.com --fail-under 85

The first command initializes a fresh project only when no config exists; init itself refuses an existing .sitecmd/ directory. The explicit --url keeps the gate pointed at staging even if a committed config names another environment. scan runs the checks. If the score is below 85, the command exits with code 1 and the CI job fails. If the score is 85 or above, it exits 0 and the job passes.

Most teams start here and iterate.

Free source-code gate

Audit the checkout directly and fail when Code Scan finds the selected severity or higher:

sitecmd audit . --format github --fail-on high

This requires no account, license, baseline, or secret. Exit code 0 passes, 1 means a finding met the severity floor, and 2 means the audit itself could not run. Other output formats are summary, json, markdown, review, and sarif.

Connected shared-baseline gate

For a site using the connected service, sitecmd gate audits the current checkout with Code Scan and asks the service which findings are new against the shared baseline:

export SITECMD_CI_TOKEN='<the site-scoped CI token>'
export SITECMD_CONNECTION_PASSPHRASE='<the export passphrase>'
sitecmd gate --connection-export ./connection.json --fail-on high

Create the encrypted connection export and CI token from Settings, then Connected, in the desktop app. The export contains no bearer credential. Store the token and passphrase in the CI system's secret store; the encrypted export can travel with the workflow by whatever file or artifact mechanism your team uses.

The gate exits 0 when the checkout passes, 1 when new findings at or above the selected severity block the merge, and 2 when the gate could not run. Its candidate is evaluated and discarded, so a pull request gate never changes the baseline. Add --strict when detector or corpus drift should also block instead of producing an uncertain comparison.

This is distinct from sitecmd connected --submit, which binds code evidence to a deployment after it exists. See the CLI reference for connected submission, GitHub OIDC attestation, and every flag.

Picking a threshold

Some defaults that work for most projects:

  • Marketing site, low-stakes: --fail-under 75. Catches bad regressions, doesn't block on polish.
  • Production product site: --fail-under 85. Reasonable bar for a launched site you care about.
  • High-stakes (financial, health, anything regulated): --fail-under 90 and add --type security --fail-under 95 as a second gate.

You can also pin a category-specific gate. sitecmd scan --type security --fail-under 95 only checks security category, and only fails if security drops below 95.

Multi-gate strategy

Run the same scan multiple times with different filters when you want different bars for different categories:

sitecmd scan --type security --fail-under 95
sitecmd scan --type accessibility --fail-under 80
sitecmd scan --fail-under 75

Each line is its own gate. Any one failing fails the build.

GitHub Actions

name: Site health

on:
  push:
    branches: [main]
  pull_request:

jobs:
  sitecmd:
    permissions:
      contents: read
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1

      - name: Install SiteCMD CLI
        uses: brambleworks/SiteCMD-Local/.github/actions/setup-sitecmd@<full-release-commit-sha>
        with:
          version: "<version>"

      - name: Audit source
        run: sitecmd audit . --format github --fail-on high

      - name: Scan staging
        run: |
          test -f .sitecmd/config.json || sitecmd init https://staging.example.com --yes --no-deep-link
          sitecmd scan --url https://staging.example.com --fail-under 85

Replace the two placeholders with the version and full commit behind the signed SiteCMD release tag. Do not use main: the setup action verifies that exact release archive with SiteCMD's updater signing key before it runs the binary. The CLI's exit code is what GitHub Actions reads. No extra adapter is needed.

GitLab CI

sitecmd:
  image: ubuntu:24.04
  variables:
    SITECMD_VERSION: "<version>"
    SITECMD_SOURCE_COMMIT: "<full-release-commit-sha>"
  before_script:
    - apt-get update && apt-get install -y curl ca-certificates minisign
    - mkdir -p .sitecmd-setup "$CI_PROJECT_DIR/.sitecmd-bin"
    - curl -fsSLo .sitecmd-setup/install.sh "https://raw.githubusercontent.com/brambleworks/SiteCMD-Local/$SITECMD_SOURCE_COMMIT/.github/actions/setup-sitecmd/install.sh"
    - curl -fsSLo .sitecmd-setup/updater-public-key.pub "https://raw.githubusercontent.com/brambleworks/SiteCMD-Local/$SITECMD_SOURCE_COMMIT/.github/actions/setup-sitecmd/updater-public-key.pub"
    - chmod +x .sitecmd-setup/install.sh
    - SITECMD_INSTALL_DIR="$CI_PROJECT_DIR/.sitecmd-bin" .sitecmd-setup/install.sh
    - export PATH="$CI_PROJECT_DIR/.sitecmd-bin:$PATH"
  script:
    - sitecmd audit . --format github --fail-on high
    - test -f .sitecmd/config.json || sitecmd init https://staging.example.com --yes --no-deep-link
    - sitecmd scan --url https://staging.example.com --fail-under 85

Use the version and full source commit from the same release. The installer and trust root are fetched from that immutable commit, then verify the release archive before installing it.

CircleCI

jobs:
  sitecmd:
    docker:
      - image: cimg/base:current
    steps:
      - checkout
      - run:
          name: Scan
          command: |
            export SITECMD_VERSION="<version>"
            export SITECMD_SOURCE_COMMIT="<full-release-commit-sha>"
            sudo apt-get update
            sudo apt-get install -y minisign
            mkdir -p .sitecmd-setup "$HOME/.local/bin"
            curl -fsSLo .sitecmd-setup/install.sh "https://raw.githubusercontent.com/brambleworks/SiteCMD-Local/$SITECMD_SOURCE_COMMIT/.github/actions/setup-sitecmd/install.sh"
            curl -fsSLo .sitecmd-setup/updater-public-key.pub "https://raw.githubusercontent.com/brambleworks/SiteCMD-Local/$SITECMD_SOURCE_COMMIT/.github/actions/setup-sitecmd/updater-public-key.pub"
            chmod +x .sitecmd-setup/install.sh
            SITECMD_INSTALL_DIR="$HOME/.local/bin" .sitecmd-setup/install.sh
            export PATH="$HOME/.local/bin:$PATH"
            sitecmd audit . --format github --fail-on high
            test -f .sitecmd/config.json || sitecmd init https://staging.example.com --yes --no-deep-link
            sitecmd scan --url https://staging.example.com --fail-under 85

Pre-deploy vs. post-deploy

Two common shapes:

  • Pre-deploy gate: scan staging before promoting to production. Fails the deploy if staging regressed.
  • Post-deploy verification: scan production right after deploy. Surfaces regressions immediately, even if they only manifest with real production config (CDN headers, env vars, third-party scripts).

You typically want both, run independently.

Audit checkout
  -> Deploy staging
  -> Scan staging URL
  -> Promote to production
  -> Scan production URL

Express these dependencies with your CI system's job dependencies. In GitHub Actions, use needs between jobs and put commands inside each job's steps, as in the complete job above.

Severity gates (no score involved)

A score floor tolerates a bad finding if the rest of the site is healthy. When some findings are never acceptable regardless of the overall score, gate on severity instead:

sitecmd scan --fail-on high

This exits 1 when any failing issue is at or above the chosen floor (critical, high, medium, or low). It combines with --fail-under; the build fails when either gate trips.

Comparing scans (regression-only gates)

If you want to fail on new regressions rather than absolute thresholds, use --diff:

sitecmd scan --diff

This compares against .sitecmd/last-scan.json from the previous run and exits 1 when a new critical issue appears, even with no --fail-under set (add --fail-under if you also want an absolute floor). Useful when you're improving a site over time and don't want to block on the legacy baseline.

You'll need to persist .sitecmd/last-scan.json between CI runs for --diff to work. Most CI tools support cache or artifact-passing for this.

Performance

The CLI's live-site checks measure performance without a browser: response time and TTFB, compression, caching headers, render-blocking resources, and image hygiene. They run fast enough for every PR scan.

Browser performance measurements and the axe-core accessibility pass are absent from the default CLI binary. Run them on a supported desktop platform or through a connected site's hosted scan. A CLI built from source with the optional browser feature can also run browser checks where Chromium is available. See CLI reference for coverage.

Output for humans

By default, sitecmd scan prints a readable summary to stdout. CI logs are perfectly fine for the day-to-day. If you want structured output for a script or a custom report:

sitecmd scan --json > results.json

Then parse with jq or pipe into whatever you want.

Output for pull request comments

Save the full fix output as an artifact and keep one short PR comment linking to it. This avoids sending the same prompts to an agent each time it reads the discussion. A scan must have run earlier in the job because fix reads the exported .sitecmd/ results. No license environment variables are required:

sitecmd fix --all > fixes.md

For a GitHub Actions job running trusted pull requests from the same repository, add these job settings. Use the same concurrency group for every job that writes this summary, so overlapping runs cannot both create the first comment:

permissions:
  contents: read
  pull-requests: write
concurrency:
  group: sitecmd-fix-summary-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: false

Append these steps after the Web Scan. Replace the action placeholder with a reviewed full commit of actions/github-script:

- name: Generate fix prompts
  id: sitecmd_fix_prompts
  if: >-
    ${{ !cancelled() && github.event_name == 'pull_request' &&
        github.event.pull_request.head.repo.full_name == github.repository &&
        hashFiles('.sitecmd/last-scan.json') != '' }}
  run: sitecmd fix --all > fixes.md

- name: Upload fix prompts
  id: sitecmd_fix_artifact
  if: ${{ !cancelled() && steps.sitecmd_fix_prompts.outcome == 'success' }}
  uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
  with:
    name: sitecmd-fix-prompts-${{ github.run_id }}-${{ github.run_attempt }}
    path: fixes.md
    if-no-files-found: error

- name: Update fix summary on PR
  if: ${{ !cancelled() && steps.sitecmd_fix_artifact.outcome == 'success' }}
  uses: actions/github-script@<full-github-script-commit-sha>
  env:
    SITECMD_FIX_ARTIFACT_URL: ${{ steps.sitecmd_fix_artifact.outputs.artifact-url }}
  with:
    script: |
      const fs = require("fs");
      const marker = "<!-- sitecmd-fix-summary -->";
      const artifactUrl = process.env.SITECMD_FIX_ARTIFACT_URL;
      if (!artifactUrl || artifactUrl.length > 1024) {
        throw new Error("Missing or oversized fix artifact URL");
      }
      const sizeKiB = Math.ceil(fs.statSync("fixes.md").size / 1024);
      const body = [
        marker,
        "### SiteCMD fix guidance",
        `Full fix output (${sizeKiB} KiB): [Download artifact](${artifactUrl}).`
      ].join("\n\n");
      const comments = await github.paginate(github.rest.issues.listComments, {
        ...context.repo, issue_number: context.issue.number, per_page: 100
      });
      const existing = comments.find(comment =>
        comment.user?.login === "github-actions[bot]" &&
        comment.user?.type === "Bot" && comment.body?.startsWith(marker)
      );
      if (existing) {
        if (existing.body !== body) {
          await github.rest.issues.updateComment({
            ...context.repo, comment_id: existing.id, body
          });
        }
      } else {
        await github.rest.issues.createComment({
          ...context.repo, issue_number: context.issue.number, body
        });
      }

The comment stays under 1,500 characters regardless of the prompt file's size. Repeating the same summary and artifact link makes no comment write; a new artifact link updates the existing bot comment. The complete prompts remain in fixes.md for a reviewer or agent to open when needed. Artifact downloads require GitHub sign-in and remain available until the artifact expires or is deleted, as described in the upload-artifact documentation.

The explicit conditions let these steps run after a failed quality gate when its scan artifact exists, and prevent commenting if the fix artifact upload fails. Fork pull requests should use the artifact-only flow below without granting a write token to untrusted code. Check your repository's publication policy before sharing artifacts; the generated text can contain evidence from the scan.

Fix prompts are part of the complete free local workbench. The command reads only the scan artifact in the CI checkout and writes to standard output.

When the gate fails

A failed gate identifies the threshold that failed. The CI log shows the summary, and the scan JSON carries the detailed findings. Common pattern:

  1. CI fails the build with the score.
  2. Developer opens the SiteCMD desktop app (if they have it), or downloads results.json from the CI artifacts.
  3. Triage from there.

You can also wire the post-failure step to upload the scan JSON as an artifact:

- name: Upload scan results
  if: failure()
  uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
  with:
    name: sitecmd-scan
    path: .sitecmd/last-scan.json

What CI gates can't do

  • A local Web Scan cannot verify an unshipped fix. It runs against a URL, so if a pull request has not reached staging yet, the command is scanning the previous version of the site. The connected checkout gate can evaluate source findings before deployment, but not live behavior that does not exist at a URL yet.
  • Local gates do not have shared integration history. A fresh CI checkout has no local analytics, deploy, or uptime history to correlate. Connected CI adds the hosted baseline through a separate site-scoped credential; it does not copy all desktop integration data into the runner.
  • They can't replace a real review. A passing gate means the site cleared the threshold, not that it's perfect. Use gates as a floor, not a ceiling.