OpenAPI diff in CI that never blocks the build

Why start non-blocking

• Some breaking changes are intended. A check that fails the build makes the author fight the tool instead of telling the consumers.
• The first weeks are noisy. You will find rules you want to ignore and specs that do not parse. Better to learn that from a report than from a blocked release.
• It is easier to approve. A step that cannot stop a pipeline is a small change to review. Platform and security teams say yes faster.
• You can tighten it later. Turning a report into a gate is one line. Going the other way after people have learned to override it is harder.

Three ways a “non-blocking” check still blocks

A step that calls a hosted service has a fourth way: the service is down or slow, and the step waits or fails. Any such step should time out and exit cleanly. That is the first thing to check before you add one.

GitHub Actions

The oasdiff GitHub Action without fail-on reports and passes. continue-on-error covers the case where oasdiff cannot run:

name: api-spec-check
on:
  pull_request:
    paths: [ "openapi.yaml" ]
permissions:
  contents: read
jobs:
  spec-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - run: git fetch --depth=1 origin ${{ github.base_ref }}
        continue-on-error: true
      - uses: oasdiff/oasdiff-action/breaking@v0
        continue-on-error: true
        with:
          base: 'origin/${{ github.base_ref }}:openapi.yaml'
          revision: 'HEAD:openapi.yaml'
          review: false

1. No fail-on

   Breaking changes appear as annotations on the Files changed tab and in the run summary. The job stays green.

2. continue-on-error on two steps

   If the base branch cannot be fetched or the spec does not parse, the step fails but the job passes. In later steps, steps.<id>.outcome is failure while conclusion is success, if you want to report that somewhere.

3. review: false

   By default the action uploads both specs to oasdiff.com, encrypted in CI, for a side-by-side review link. With false, no spec leaves the runner. Drop the line if you want the link.

4. Minimal permissions

   contents: read only. The action needs pull-requests: write just to comment the review link, which this setup does not post.

GitLab CI

oasdiff publishes a Docker image with git included, so the job needs no install step. The merge request’s base commit comes from a predefined variable:

api-spec-diff:
  stage: test
  image:
    name: tufin/oasdiff:v1.37.0
    entrypoint: [""]
  variables:
    GIT_DEPTH: "0"
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        - openapi.yaml
  script:
    - oasdiff breaking "$CI_MERGE_REQUEST_DIFF_BASE_SHA:openapi.yaml" HEAD:openapi.yaml --format junit > oasdiff-junit.xml
    - oasdiff breaking "$CI_MERGE_REQUEST_DIFF_BASE_SHA:openapi.yaml" HEAD:openapi.yaml --fail-on ERR --color never
  allow_failure: true
  artifacts:
    when: always
    reports:
      junit: oasdiff-junit.xml

1. entrypoint: [""]

   The image’s entrypoint is the oasdiff binary. GitLab needs a shell to run script, so the entrypoint is cleared.

2. GIT_DEPTH: "0" and the base SHA

   CI_MERGE_REQUEST_DIFF_BASE_SHA is the commit the merge request diff starts from. A full clone makes sure that commit is there. Comparing with it, not with the tip of the target branch, keeps other merged work out of the diff. In merged-results pipelines, compare with $CI_MERGE_REQUEST_TARGET_BRANCH_SHA instead.

3. JUnit report

   The first command writes each breaking change as a failed test case. GitLab shows them in the merge request’s test report, so reviewers do not have to open the job log.

4. --fail-on ERR with allow_failure: true

   An error-level change makes the job fail, and allow_failure turns that into an orange warning while the pipeline still passes. The same happens if oasdiff cannot run. Reviewers see something is off; nobody is stopped.

For a plain green job, drop the second command and keep allow_failure: true for the cases where oasdiff cannot run. On other CI systems the rule is the same: run the diff without a threshold, and tell the CI to carry on if the step itself fails.

What a security review will ask

Any step that reads your API spec in CI gets the same questions. Have the answers ready:

Rolling it out

1. Week one: report only

   Add the job without a threshold on one API. Nobody is blocked, and you see what it finds on real pull requests.

2. Tune the noise

   Fix specs that do not parse. Put agreed breaking changes in an ignore file, each with the pull request that accepted it.

3. Add a visible signal

   On GitLab, the --fail-on ERR line with allow_failure gives an orange job. On GitHub, the annotations are the signal.

4. Decide about a gate, if ever

   If a breaking change should stop a merge until someone approves it, make that a separate, explicit decision and mark the check required.

Where Graphwake fits

A diff in the pull request tells the author and the reviewers. It does not tell the teams that call the endpoint. Graphwake’s CI step sends the spec on every pull request; Graphwake finds the breaking changes and alerts the teams subscribed to the changed endpoints. The step follows the rules above by design.

• Never fails a build

  The CI step never fails or blocks a build, whatever it finds.

• Exits cleanly when we are down

  If Graphwake can’t be reached, the step logs it and exits cleanly. The build carries on.

• Reads the spec, not your code

  The step sends your OpenAPI spec. Graphwake never reads your code and needs no repository access.

• One line in the log

  On a pull request it prints graphwake: spec sent (pull request #1284). Pull request specs are stored as drafts and do not replace the live spec.

It runs in GitHub Actions, GitLab CI, Azure DevOps, Bitbucket Pipelines and Jenkins, next to an oasdiff job if you keep one. If you would rather not touch CI at all, give Graphwake your spec’s URL and it checks it every 30 minutes.

Next steps

• Every input of the oasdiff Action, and what to do with its output: the oasdiff GitHub Action, step by step.
• What each breaking change looks like in a spec: OpenAPI breaking changes.
• Getting the alert to the teams that use the endpoint: API breaking change alerts.

Questions

Sources and last verified

• github.com/oasdiff/oasdiff-action
• github.com/oasdiff/oasdiff/blob/main/docs/GIT-REVISION.md
• github.com/oasdiff/oasdiff/blob/main/Dockerfile
• docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax
• docs.github.com/en/actions/reference/workflows-and-actions/contexts
• docs.gitlab.com/ci/yaml/
• docs.gitlab.com/ci/variables/predefined_variables/
• docs.gitlab.com/ci/runners/configure_runners/
• oasdiff and its name belong to its authors. Graphwake is not affiliated with the oasdiff project.

Sources checked on 2026-11-09.

Related

• GuideAPI breaking change alerts: how to tell the right teams
• GuideThe oasdiff GitHub Action, step by step
• Compareoasdiff vs Graphwake
• CompareSpecmatic vs Graphwake