sbt-zipx · v0.11.0

Verify

Verify is several Once jobs with no needs between them, so GitHub runs them in parallel: test, fmt, workflow-check, and advisories. Publish does not wait on them in-workflow (tag runs already do not need test); PR merge is the required-checks story.

On sbt 2, plain sbt test can skip suites it thinks are unaffected. zipx defaults to testFull so CI actually runs every suite. Path-based job skipping (only run jobs this PR touched) lives on Affected, and only in Graph mode.

zipxVerify := ZipxVerify.Strict   // all On; the default
zipxVerify := ZipxVerify.Strict.copy(
  fmt = VerifyOpt.Skip("hotfix: scalafmt 3.x busts this branch, revert Monday"),
  advisories = VerifyOpt.Skip("hotfix: GHSA-xxxx in checkout, Action bump in follow-up"),
)

Skip(reason) still emits the job: it prints zipx: skipping <gate>: <reason> and exits 0. The check name stays on the PR. Empty or whitespace reason is a zipx: generate error. Replace-by-name still works for a totally custom fmt command; Skip is for turning the gate off out loud.

Test task and optional clean

zipxTestTask    := zipxTasks.of(testFull)  // Aggregate root; Graph/Layer per-module (plugin default)
zipxVerifyClean := VerifyClean.CleanFull   // None (default) | Clean | CleanFull

For a one-off LocalDir / action-cache bust without making clean permanent, leave zipxVerifyClean at None (default) and add the GitHub PR label clean. Verify then runs cleanFull; <task> only on that PR. The version-updates and pin-updates companions always add that label. Override the label name with zipxVerifyCleanLabel, or set it to None to disable.

zipxVerifyCleanLabel := Some("clean")  // default

{
  given PlanConfig = config.copy(verifyClean = VerifyClean.CleanFull)
  DocsRender.jobs("test")(Capability.test) + "\n---\n" +
    DocsRender.job("test-schema")(Capability.testGraph)
}
test:
  name: test
  runs-on: ubuntu-latest
  if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: test
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: test
      run: sbt 'cleanFull; test'
---
test-schema:
  name: test schema
  runs-on: ubuntu-latest
  needs:
    - affected
  if: (!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch') && (!cancelled() && (contains(fromJson(needs.affected.outputs.modules), 'schema') || contains(fromJson(needs.affected.outputs.modules), 'all')))
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: test-schema
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: test
      run: sbt 'cleanFull; schema/test'
{
  given PlanConfig = config.copy(verifyCleanLabel = PlanConfig.verifyCleanLabel("clean"))
  DocsRender.jobs("test")(Capability.test)
}
test:
  name: test
  runs-on: ubuntu-latest
  if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: test
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: test
      run: |
        if [ "$ZIPX_VERIFY_CLEAN_FULL" = "true" ]; then
          sbt 'cleanFull; test'
        else
          sbt 'test'
        fi
      env:
        ZIPX_VERIFY_CLEAN_FULL: ${{ github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'clean') }}

Coverage (and why `test` is the wrong task here)

On sbt 2.0 plain test is testQuick. It skips tests it deems unaffected and prints No tests to run, which reads like success. zipx's Verify default is testFull so CI cannot skip suites. Under coverage that skip is a silent wrong answer: the hand-rolled alias everyone writes,

addCommandAlias("testWithCoverage", "cleanFull; coverage; compile; test; coverageAggregate; coverageReport; coverageOff")

measures whichever tests sbt happened to run, satisfies coverageMinimum on near-zero data, and goes green. So zipx builds the command instead of taking one:

zipxCapabilities += Coverage.once()   // coverage; testFull; coverageAggregate  (one session)
zipxCapabilities += Coverage.graph()  // one job per module, each measuring that module's own zipxTestTask

Coverage.graph() reads each module's zipxTestTask and substitutes testFull where it is still the default test. A module that set the task itself is left alone, because an explicit choice outranks zipx's. Pass _.testTask for literal inheritance, default and all:

zipxCapabilities += Coverage.graph(task = _.testTask)

Two things the alias has that the capability deliberately does not. No trailing coverageOff: the sbt session ends with the job, and that command exists because a developer's shell outlives the command. And no cleanFull: use zipxVerifyClean or the clean PR label above if you want one, so the choice is in one place.

coverage is a session-wide toggle, which is why enable / test / report are one sbt '…; …; …' invocation rather than three steps.

Prefer once: coverageAggregate is a root task over every module's measurement data, so splitting it across jobs means downloading and merging artifacts to get one number back. graph buys affected-gating (it is an ordinary Graph Verify capability, so everything on the Affected page applies) at the cost of per-module minimums instead of a build-wide one.

The report is uploaded with the already-pinned actions/upload-artifact, per module under graph so N jobs do not collide on one artifact name. if-no-files-found: error, on purpose: a run that measured nothing produces no report, and that should be a red job rather than an empty upload. Turn it off with uploadReport = false.

DocsRender.jobs("coverage")(Coverage.once())
coverage:
  name: coverage
  runs-on: ubuntu-latest
  if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: coverage
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: coverage
      run: sbt 'coverage; testFull; coverageAggregate'
    - name: Upload coverage report
      uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
      with:
        name: coverage-report
        path: "**/scoverage-report/**"
        if-no-files-found: error
DocsRender.job("coverage-schema")(Coverage.graph())
coverage-schema:
  name: coverage schema
  runs-on: ubuntu-latest
  needs:
    - affected
  if: (!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch') && (!cancelled() && (contains(fromJson(needs.affected.outputs.modules), 'schema') || contains(fromJson(needs.affected.outputs.modules), 'all')))
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: coverage-schema
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: coverage
      run: sbt 'coverage; schema/testFull; schema/coverageReport'
    - name: Upload coverage report
      uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
      with:
        name: coverage-report-schema
        path: "**/scoverage-report/**"
        if-no-files-found: error

Affected-only PRs (Graph only)

zipxAffectedOnPR (default true) emits an affected setup job only when a Graph Verify capability is present. Aggregate and Layer always invoke their full stage command (they do not skip GitHub jobs). That is not the same as "always recompile everything": Zinc and the cross-run task cache (restored by zipx at the epoch, or via remote cache) still skip unaffected compile work, even on a cold JVM. Verify's default is testFull, so suites still run unless the whole test task is a cache hit. See Execution modes ("Two kinds of affected") and the Affected page for the fail-open handoff, who is gated, and zipxAffectedPublish, which extends the same narrowing to Graph Publish jobs as a separate opt-in.

zipxAffectedOnPR := true   // default with Graph Verify
or Graph
{
  given PlanConfig = config.copy(affected = AffectedMode.AffectedOnPR)
  DocsRender.body(Capability.test) + "\n---\n" + DocsRender.body(Capability.testGraph)
}
name: CI
"on":
  push:
    branches:
      - main
  pull_request: null
concurrency:
  group: CI-${{ github.ref }}
  cancel-in-progress: ${{ !startsWith(github.ref, 'refs/tags/') }}
jobs:
  test:
    name: test
    runs-on: ubuntu-latest
    if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
        with:
          fetch-depth: "0"
          fetch-tags: "true"
      - name: zipx sbt setup
        uses: ./.github/actions/zipx-sbt-setup
        with:
          java-version: "21"
          runner-os: ubuntu-latest
          cache-key-suffix: test
          node-version: ""
          sbt-disk-cache: "false"
          local-cache: "true"
          cache-epoch: "0.1.0-ci"
      - name: test
        run: sbt 'test'
---
name: CI
"on":
  push:
    branches:
      - main
  pull_request: null
concurrency:
  group: CI-${{ github.ref }}
  cancel-in-progress: ${{ !startsWith(github.ref, 'refs/tags/') }}
jobs:
  affected:
    name: affected
    runs-on: ubuntu-latest
    if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
    outputs:
      modules: ${{ steps.compute.outputs.modules }}
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
        with:
          fetch-depth: "0"
          fetch-tags: "true"
      - name: zipx sbt setup
        uses: ./.github/actions/zipx-sbt-setup
        with:
          java-version: "21"
          runner-os: ubuntu-latest
          cache-key-suffix: affected
          node-version: ""
          sbt-disk-cache: "false"
          local-cache: "false"
          cache-epoch: "0.1.0-ci"
      - name: Compute affected modules
        id: compute
        run: |
          if [ "${{ github.event_name }}" = "pull_request" ]; then
            BASE="${{ github.event.pull_request.base.sha }}"
            sbt -batch --error "zipxAffectedModules $BASE"
            modules=$(cat target/zipx-affected.json)
          else
            modules='["all"]'
          fi
          echo "modules=$modules" >> "$GITHUB_OUTPUT"
  test-schema:
    name: test schema
    runs-on: ubuntu-latest
    needs:
      - affected
    if: (!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch') && (!cancelled() && (contains(fromJson(needs.affected.outputs.modules), 'schema') || contains(fromJson(needs.affected.outputs.modules), 'all')))
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
        with:
          fetch-depth: "0"
          fetch-tags: "true"
      - name: zipx sbt setup
        uses: ./.github/actions/zipx-sbt-setup
        with:
          java-version: "21"
          runner-os: ubuntu-latest
          cache-key-suffix: test-schema
          node-version: ""
          sbt-disk-cache: "false"
          local-cache: "true"
          cache-epoch: "0.1.0-ci"
      - name: test
        run: sbt 'schema/test'
  test-api:
    name: test api
    runs-on: ubuntu-latest
    needs:
      - affected
      - test-schema
    if: (!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch') && (!cancelled() && (contains(fromJson(needs.affected.outputs.modules), 'api') || contains(fromJson(needs.affected.outputs.modules), 'all')) && needs.test-schema.result != 'failure')
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
        with:
          fetch-depth: "0"
          fetch-tags: "true"
      - name: zipx sbt setup
        uses: ./.github/actions/zipx-sbt-setup
        with:
          java-version: "21"
          runner-os: ubuntu-latest
          cache-key-suffix: test-api
          node-version: ""
          sbt-disk-cache: "false"
          local-cache: "true"
          cache-epoch: "0.1.0-ci"
      - name: test
        run: sbt 'api/test'
  test-service:
    name: test service
    runs-on: ubuntu-latest
    needs:
      - affected
      - test-api
    if: (!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch') && (!cancelled() && (contains(fromJson(needs.affected.outputs.modules), 'service') || contains(fromJson(needs.affected.outputs.modules), 'all')) && needs.test-api.result != 'failure')
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
        with:
          fetch-depth: "0"
          fetch-tags: "true"
      - name: zipx sbt setup
        uses: ./.github/actions/zipx-sbt-setup
        with:
          java-version: "21"
          runner-os: ubuntu-latest
          cache-key-suffix: test-service
          node-version: ""
          sbt-disk-cache: "false"
          local-cache: "true"
          cache-epoch: "0.1.0-ci"
      - name: test
        run: sbt 'service/test'

Changed files → owning module (longest base-dir prefix) → reverse-dependency closure. A .sbt change or anything under project/ forces a full build. On push/tag everything builds unless zipxAffectedOnPush is enabled. If the diff cannot run, zipx emits ["all"] (fail open) so a bad base ref never reports a green, untested PR.

Skip Verify after merge / on tags

By default (zipxSkipMergedPrPush := true), a push to main that lands a merged PR does not re-run Verify. Direct pushes still Verify. Tag pushes never run Verify (release tags only need Publish / Deploy). Graph Publish and Deploy still run after a merge; the affected job they read stays up and emits ["all"] on that non-PR push, so they do not fromJson an empty output.

verify-gate looks up the landed SHA on GET /commits/{sha}/pulls. That index can lag a few seconds after a squash (new SHA at merge). The gate retries with 1/2/4/8/16s sleeps rather than treating an empty first answer as a direct push and running Verify again.

With LocalDir, that skip would otherwise leave main without an actions/cache save (PR caches are branch-scoped; later PRs only warm from the default branch). So by default zipx also emits a minimal cache-rehydrate job that runs only when verify-gate skips Verify: same checkout / JDK / LocalDir cache path, then compile (override with zipxCacheRehydrateTask). No full test, no verifyClean. Set zipxCacheRehydrateOnMerge := false to opt out; remote backends never emit it.

To also warm non-sbt blobs that live under target/ (e.g. Playwright browsers), opt into rehydrate-only extraSteps. Prefer build-wide zipxEnv for vars needed on Verify and rehydrate; use zipxCacheRehydrateEnv only for merge-only overlays. Neither is copied from Verify capability extraSteps / env; assign the same setup function when you want step parity:

val browserSetup = Steps.built("browsers")(
  Step.run(Script(Exec("npm", Word.lit("ci")))).named("Install browsers")
)

zipxSkipMergedPrPush := true  // default
zipxCacheRehydrateOnMerge := true  // default; LocalDir only
zipxCacheRehydrateTask := zipxTasks.of(compile)  // default
zipxEnv := Map(
  "PLAYWRIGHT_BROWSERS_PATH" -> EnvValue.typed(Expr.github("workspace") ++ Expr.lit("/target/ms-playwright")),
)
zipxCacheRehydrateExtraSteps := browserSetup  // after cache restore, before compile

EnvValue.typed takes any Expr, so ++ concatenates a context reference with literal text instead of spelling the ${{ … }} out. EnvValue.expr still accepts a raw string, and zipxWorkflowGenerate warns when one is used.

browserSetup is a Steps bundle, not a lambda: it carries a name, composes with ++, gates with .when(...), and reports escape-hatch use so zipxWorkflowGenerate can warn and name it. The step body is a typed Script, so nothing here is a hand-written shell string. See Shell and steps for the whole DSL.

{
  given PlanConfig = config.copy(skipMergedPrPush = true)
  DocsRender.jobs("verify-gate", "cache-rehydrate", "test")(Capability.test)
}
verify-gate:
  name: verify-gate
  runs-on: ubuntu-latest
  if: github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')
  permissions:
    contents: read
    pull-requests: read
  outputs:
    run: ${{ steps.check.outputs.run }}
  steps:
    - name: Skip Verify after merged PR push
      id: check
      run: |
        # Commits landed by merging/squashing a PR are associated with that PR via the API.
        # The commit-to-PR index can lag after squash; retry rather than fail-open into a second Verify.
        prs=0
        for attempt in 1 2 3 4 5 6; do
          if [ "$prs" -eq 0 ]; then
            if [ "$attempt" -eq 2 ]; then
              sleep 1
            elif [ "$attempt" -eq 3 ]; then
              sleep 2
            elif [ "$attempt" -eq 4 ]; then
              sleep 4
            elif [ "$attempt" -eq 5 ]; then
              sleep 8
            elif [ "$attempt" -eq 6 ]; then
              sleep 16
            fi
            prs=$(gh api "repos/${{ github.repository }}/commits/${{ github.sha }}/pulls" \
              --jq "[.[] | select(.merged_at != null and .base.ref == \"${{ github.ref_name }}\")] | length")
          fi
        done
        if [ "$prs" -gt 0 ]; then
          echo "Merged PR push, skipping redundant Verify (already ran on the PR)"
          echo "run=false" >> "$GITHUB_OUTPUT"
        else
          echo "run=true" >> "$GITHUB_OUTPUT"
        fi
      env:
        GH_TOKEN: ${{ github.token }}
cache-rehydrate:
  name: cache-rehydrate
  runs-on: ubuntu-latest
  needs:
    - verify-gate
  if: needs.verify-gate.result == 'success' && needs.verify-gate.outputs.run == 'false'
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: cache-rehydrate
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: cache-rehydrate
      run: sbt 'compile'
test:
  name: test
  runs-on: ubuntu-latest
  needs:
    - verify-gate
  if: "!cancelled() && !startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch' && ((needs.verify-gate.result != 'success') || (needs.verify-gate.outputs.run == 'true'))"
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: test
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: test
      run: sbt 'test'
{
  given PlanConfig = config.copy(
    skipMergedPrPush = true,
    env = Map(
      "PLAYWRIGHT_BROWSERS_PATH" ->
        EnvValue.typed(Expr.github("workspace") ++ Expr.lit("/target/ms-playwright"))
    ),
    cacheRehydrateExtraSteps = Steps.built("browsers")(
      Step.run(Script(Exec("npm", Word.lit("ci")))).named("Install browsers")
    ),
  )
  DocsRender.jobs("cache-rehydrate", "test")(Capability.test)
}
cache-rehydrate:
  name: cache-rehydrate
  runs-on: ubuntu-latest
  needs:
    - verify-gate
  if: needs.verify-gate.result == 'success' && needs.verify-gate.outputs.run == 'false'
  env:
    PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/target/ms-playwright
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: cache-rehydrate
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: Install browsers
      run: npm ci
    - name: cache-rehydrate
      run: sbt 'compile'
test:
  name: test
  runs-on: ubuntu-latest
  needs:
    - verify-gate
  if: "!cancelled() && !startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch' && ((needs.verify-gate.result != 'success') || (needs.verify-gate.outputs.run == 'true'))"
  env:
    PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/target/ms-playwright
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: zipx sbt setup
      uses: ./.github/actions/zipx-sbt-setup
      with:
        java-version: "21"
        runner-os: ubuntu-latest
        cache-key-suffix: test
        node-version: ""
        sbt-disk-cache: "false"
        local-cache: "true"
        cache-epoch: "0.1.0-ci"
    - name: test
      run: sbt 'test'