sbt-zipx · v0.1.3

Caching

sbt 2.x caches task results across JVM runs (content-addressed, machine-wide; remote backends also share declared outputs). That is why Aggregate stays cheap on a cold CI runner: zipx restores the cache before sbt test, keyed by a commit-stable epoch (zipxCacheEpoch, default CacheEpoch.GitTags()). Every push within a PR reuses prior hits; cutting a release tag rolls the epoch without regenerating ci.yml. Remote backends make the same story stronger across machines, including developer laptops when CI hydrates a shared store (see Remote cache for teams). This pairs with sbt-dynver-ci.

yesno

Miss path (amber → red) pays compile/test and writes digests; hit path (green) reuses them. Both land in the same backend: LocalDir via actions/cache, or a remote gRPC store. The restore key is the commit-stable epoch (zipxCacheEpoch), so PR pushes share hits and a release tag rolls a fresh namespace.

zipx wires cache into generated jobs (same planner as topology). It is not a standalone acceleration appliance: the goal is CI-from-graph plus content-addressed reuse, not a second product to configure beside hand-maintained YAML.

Epoch strategies

zipxCacheEpoch := CacheEpoch.GitTags()                 // default: resolve from git tags on the runner
zipxCacheEpoch := CacheEpoch.GitTags(tagMatch = "v*")  // same, explicit match glob
zipxCacheEpoch := CacheEpoch.Fixed(version.value)      // bake at generate time (old behaviour)
zipxCacheEpoch := CacheEpoch.Script(myEpochShell)      // custom shell; must write epoch= and release=

GitTags (default): a Resolve cache epoch step runs after checkout (fetch-depth: 0, fetch-tags: true). On a v* tag ref, epoch = release = tag without v. Otherwise the latest matching tag becomes release and epoch is ${release}-ci. If local tags lag origin (or none match), the step emits an Actions ::warning titled zipx cache epoch so shallow/missing tags are obvious in the run summary.

Fixed: embeds a literal into the workflow at zipxWorkflowGenerate (useful for scripted tests or unusual versioning). Prefer GitTags so post-tag PRs warm from the release cache without a regenerate commit.

Script: supply your own shell; write epoch= and release= to $GITHUB_OUTPUT. Restore-keys use both outputs.

Backends

zipxCache := CacheBackend.LocalDir
zipxCache := CacheBackend.BazelRemoteSidecar(RemoteCacheProof.image, RemoteCacheProof.port)
zipxCache := CacheBackend.ManagedRemote("grpcs://cache.buildbuddy.io", "BUILDBUDDY_KEY")

  • LocalDir: persist local cache dirs and target/ with actions/cache. Primary key is OS + JDK + epoch + run id + job id; restore-keys prefer the same run, then the epoch, then the prior release epoch (Fixed: strip -ci / -SNAPSHOT; GitTags/Script: steps.*.outputs.release) so the first post-tag PR can warm from the tag build, then any older OS+JDK sbt cache. No infrastructure. GitHub scopes cache entries to the branch that saved them; other PRs restore from the default branch. With zipxSkipMergedPrPush, Verify does not run on the merge push, so by default a minimal cache-rehydrate job recreates a main-scoped save (see Verify). You cannot copy a PR cache onto main via the API.

  • BazelRemoteSidecar: pinned buchgr/bazel-remote-cache as a job service; shared across the run via Bazel gRPC. Proof pins live in RemoteCacheProof (docs, planner tests, and RemoteCacheItSpec share them).

  • ManagedRemote: point sbt at BuildBuddy / EngFlow / NativeLink; auth header from a named repository secret. This is the path for CI-hydrated caches that developers reuse (see Remote cache for teams).

The remote-cache transport is bundled with zipx. For remote backends zipx also sets Global / cacheVersion from (JDK, OS) so heterogeneous runners cannot poison the shared cache. Remote backends omit actions/cache (the gRPC store is the persistence); LocalDir uses epoch-keyed actions/cache instead.

{
  val local = DocsRender.job("test")(Capability.test)(using
    libGraph,
    config.copy(cache = CacheBackend.LocalDir),
  )
  val sidecar = DocsRender.job("test")(Capability.test)(using
    libGraph,
    config.copy(cache = RemoteCacheProof.sidecar),
  )
  val remote = DocsRender.job("test")(Capability.test)(using
    libGraph,
    config.copy(cache = CacheBackend.ManagedRemote("grpcs://cache.example", "CACHE_KEY")),
  )
  local + "\n---\n" + sidecar + "\n---\n" + remote
}
test:
  name: test
  runs-on: ubuntu-latest
  if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: Setup JDK 21
      uses: actions/setup-java@03ad4de0992f5dab5e18fcb136590ce7c4a0ac95
      with:
        distribution: temurin
        java-version: "21"
    - uses: sbt/setup-sbt@d059c39de700f4cc5cb64f9f56577315e44a984e
      with:
        disk-cache: "false"
    - name: Cache sbt
      uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9
      with:
        path: |
          ~/.sbt
          ~/.cache/sbt
          ~/.cache/coursier
          target
        key: ubuntu-latest-jdk21-sbt-0.1.0-ci-${{ github.run_id }}-test
        restore-keys: |
          ubuntu-latest-jdk21-sbt-0.1.0-ci-${{ github.run_id }}-
          ubuntu-latest-jdk21-sbt-0.1.0-ci-
          ubuntu-latest-jdk21-sbt-0.1.0-
          ubuntu-latest-jdk21-sbt-
    - name: test
      run: sbt 'test'
---
test:
  name: test
  runs-on: ubuntu-latest
  if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
  services:
    bazel-remote:
      image: buchgr/bazel-remote-cache:v2.6.1
      ports:
        - "9092:9092"
      options: --max_size=1
  env:
    ZIPX_REMOTE_CACHE: grpc://localhost:9092
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: Setup JDK 21
      uses: actions/setup-java@03ad4de0992f5dab5e18fcb136590ce7c4a0ac95
      with:
        distribution: temurin
        java-version: "21"
    - uses: sbt/setup-sbt@d059c39de700f4cc5cb64f9f56577315e44a984e
    - name: test
      run: sbt 'test'
---
test:
  name: test
  runs-on: ubuntu-latest
  if: "!startsWith(github.ref, 'refs/tags/') && github.event_name != 'workflow_dispatch'"
  env:
    ZIPX_REMOTE_CACHE: grpcs://cache.example
    ZIPX_REMOTE_CACHE_HEADER: ${{ secrets.CACHE_KEY }}
  steps:
    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
      with:
        fetch-depth: "0"
        fetch-tags: "true"
    - name: Setup JDK 21
      uses: actions/setup-java@03ad4de0992f5dab5e18fcb136590ce7c4a0ac95
      with:
        distribution: temurin
        java-version: "21"
    - uses: sbt/setup-sbt@d059c39de700f4cc5cb64f9f56577315e44a984e
    - name: test
      run: sbt 'test'

Action pins

Generated workflows use commit-SHA pins (not floating @v4 tags), with # vX.Y.Z comments for readability.

Prefer .github/zipx/action-pins.yml (Dependabot-friendly) over pasting SHAs into build.sbt. Full guide: Action pins (resolve order, Dependabot, zipxActionsPull, sync workflow, jar defaults).