CI/CD integration

Three steps in your pipeline

Grammage runs in a Docker container, so it takes its place in your pipeline like any other step, and nobody has to think about it afterwards since it starts by itself on every merge request.

Install it in GitLab CI

The ci/grammage.gitlab-ci.yml template carries three ready-made jobs, which you take in with an include and then tune with a few variables. The first audits the preview of the merge request, the second reads the source code of the repository, and the third, .grammage-certify, has production verified further down this page.

include:
    - project: solyzon/products/grammage/grammage
      ref: prod
      file: ci/grammage.gitlab-ci.yml

grammage:
    extends: .grammage
    needs: [preview]
    variables:
        GRAMMAGE_CATEGORY: auto
        GRAMMAGE_FAIL_UNDER: C

grammage-source:
    extends: .grammage-source

needs: [preview] brings in PREVIEW_URL, the address of the preview published by the job that deploys it. To audit another address, GRAMMAGE_URL replaces it, and your licence arrives through the masked variable GRAMMAGE_LICENSE, which you set once for the whole group.

Without waiting for the preview

The job can also build your site and start it itself, which avoids waiting for the preview and its password. GRAMMAGE_SERVE holds the command that builds then starts the site, and the job waits for it to answer before placing a small proxy in front of it, which compresses responses in zstd or gzip as your production server would. Without this proxy, a Node server that compresses nothing would make the site look much heavier than it really is online.

grammage:
    extends: .grammage
    variables:
        GRAMMAGE_SERVE: pnpm install --frozen-lockfile && pnpm exec astro build && HOST=127.0.0.1 PORT=4321 node dist/server/entry.mjs
        GRAMMAGE_SITE: '1'
        GRAMMAGE_FAIL_UNDER: A

If the site never answers, the job fails and shows the log of the command, so that you see right away what got stuck.

The whole site and the duration of the job

With GRAMMAGE_SITE, the job no longer measures one page but the whole site, three pages per template and fifty at most. A pass takes about thirty seconds on average on a real site, twenty-four passes in a little over twelve minutes in our September 2026 tests, so the duration of the job follows directly the number of pages, the number of passes and the number of pages measured at the same time. Translations of the same page are measured only once, and the fast mode also skips the return visit, which saves 10 to 15% per page without changing the score.

The calculation is therefore simple, about thirty seconds times the number of pages times the number of passes, divided by GRAMMAGE_PARALLEL, plus the discovery of the site. These are estimates drawn from that average, and a very heavy page or a slow server makes them longer.

grammage:
    extends: .grammage
    needs: [preview]
    variables:
        GRAMMAGE_SITE: '1'
        GRAMMAGE_MODE: fast
        GRAMMAGE_PARALLEL: '3'
        GRAMMAGE_MAX_PAGES: '10'

The runner

The image starts from Node 24 and only ships a windowless Chromium, to stay light on runners. Each page measured at the same time opens its own Chromium, so GRAMMAGE_PARALLEL must never exceed the number of cores the runner gives to the job, and it is capped at 8 anyway. Beyond that, the browsers fight over the processor, which changes neither the weight, nor the requests, nor the score, but distorts the computing time shown in the report. For a measurement you publish, the full mode with few pages at the same time therefore remains the most reliable, and it is better kept for a scheduled job.

What the merge request shows

The job stores its results where GitLab already knows how to display them, so you read the audit without leaving the merge request, and each finding points to the file of the repository to fix, an image public/images/hero.png served under an address transformed by the framework for example.

Block or warn

The job exits with 1 when the threshold or the budget is not met, and with 2 when a page could not be measured. By default, it warns without blocking on a missed threshold, whereas an impossible measurement does make the pipeline fail, since nothing is known about the site any more. For a failed audit to really prevent a release to production, you only need to make it mandatory and make the deployment wait for it.

grammage:
    allow_failure: false

deploy:
    needs: [grammage]

Restricted pages

A preview closed by an HTTP password is audited with GRAMMAGE_HTTP_AUTH. A page behind a login form is audited with the session of a test account without special rights, which you record once on your machine with npx playwright codegen --save-storage=session.json, then paste into a masked file-type variable, GRAMMAGE_STORAGE_STATE. The session eventually expires, and a page that then redirects to the login is measured as such, so you redo it the same way as soon as the audited pages are no longer the right ones.

Only the audit receives the session. The attestation only covers public pages, because Grammage then measures them again without any session to check your report.

Attest production

The .grammage-certify job runs right after the deployment. It measures the public address, signs the report with the key carried by your licence and sends it to the Grammage API, then waits for the verdict, twenty minutes at most, while Solyzon measures the entry page and two randomly chosen pages again. The domain must be part of your licence, and the licence must include the badge.

grammage:certify:
    extends: .grammage-certify
    needs: [deploy]
    rules:
        - if: $CI_COMMIT_BRANCH == "prod"
    variables:
        GRAMMAGE_URL: https://example.com/
        GRAMMAGE_SITE: '1'
        GRAMMAGE_CATEGORY: auto

When the Solyzon figures confirm your report, the job shows the end date of the attestation and the badge is updated. When a page weighs more than 20% more than declared, or when a chosen page could not be measured again, the job fails and says why, and the badge keeps the previous attestation as long as it is valid. Without GRAMMAGE_FAIL_UNDER or a budget, the site is verified whatever its score, which makes it possible to display an honest badge even when the site is still heavy. The detail of the verification is on the attestation page.

The variables of the template

The audit and attestation jobs read the same variables. The verification always covers a page or the whole site, never a list, since the attestation is valid for a page or for a domain, so GRAMMAGE_URLS_FILE and GRAMMAGE_SERVE are only used for the audit.

The command line

Everything the jobs do goes through the grammage command, which you run the same way on a machine, in the Docker image. grammage audit measures, grammage certify measures then records the result, grammage source reads the code of the repository, and grammage license verify tells what your licence covers, which is read from GRAMMAGE_LICENSE. Finally grammage proxy serves a site started in the CI by compressing it in zstd or gzip, as a real server would, so that the measurement sees the same bytes as in production, and this is in fact what the GitLab template does with GRAMMAGE_SERVE.

grammage proxy http://127.0.0.1:4321 --port 8080

Other CI platforms

The ready-to-include template is written for GitLab CI, which also displays the reports in the merge request. Elsewhere, Grammage remains a Docker image and a command, so the job is rewritten in a few lines, and the reports are collected in the output folder.

• GitLab CIready-to-include template
• GitHub Actions
• Bitbucket Pipelines
• Jenkins
• Docker

jobs:
    grammage:
        runs-on: ubuntu-latest
        container:
            image: registry.solyzon.net/solyzon/products/grammage/grammage:<your-tag>
            credentials:
                username: ${{ secrets.GRAMMAGE_USER }}
                password: ${{ secrets.GRAMMAGE_LICENSE }}
        env:
            GRAMMAGE_LICENSE: ${{ secrets.GRAMMAGE_LICENSE }}
        steps:
            - run: grammage audit --site https://example.com/ --mode fast --format json,html,agent --out grammage

docker login registry.solyzon.net
docker run --rm -e GRAMMAGE_LICENSE -v "$PWD/grammage:/out" \
    registry.solyzon.net/solyzon/products/grammage/grammage:<your-tag> \
    grammage audit --site https://example.com/ --format json,html --out /out

The image and the licence

The image is pulled from registry.solyzon.net, with your user name and your licence as the password, which goes into GitLab through the masked variable DOCKER_AUTH_CONFIG of your project. The registry only serves you the tag in your name and the versions your licence covers, and this tag moves forward by itself with every new version, never backwards, so a project that follows it receives the updates without touching anything.

{ "auths": { "registry.solyzon.net": { "auth": "<base64 of user:licence>" } } }

Without a valid licence, the image still audits one page at a time, in JSON and HTML, which only excludes the whole site, the GitLab reports and the badge.