Reusing trusted CI results without weakening coverage

Orgabot's CI runtime does not run the same verification twice when it can prove the result would be the same. A mission's local run, the pull request head, and the commit that lands on main after a squash merge often build exactly the same files. When they do, the job runs once and the other runs reuse its signed result.

Reuse only ever happens when equivalence is proven. Any doubt means the job runs. A job that runs when it did not need to costs a few minutes; a job that is skipped when it should have run is a gate reporting a pass for code nobody tested.

What must be equal

Every job that can be reused gets a reuse key. The key is a hash of everything that could change its result:

• The source. The git tree of the commit under test, so two commits with identical contents are the same work. A job that declares paths hashes only the files it maps, plus shared scripts, configuration and lockfiles, plus any script its commands run, plus a fingerprint of every file no job maps. Scripts are followed transitively: a package manager script such as npm run setup is resolved through the package.json it reads, and every script that runs is read for the files it calls and the modules it imports in turn. A shell glob such as bash checks/*.sh is expanded against the tree and every matching script is followed. A Python module run by name, such as python3 -m checks.gate, is resolved to its file in the tree (and a package's __main__.py) and followed the same way. An interpreter or package manager named by a path or version, such as /usr/bin/python3, python3.12, .venv/bin/python or /usr/local/bin/npm, is read as that program, so /usr/bin/python3 -m checks.gate is followed exactly like python3 -m checks.gate. A command run through a package manager or runner, such as npm exec -- node tools/gate.js, pnpm exec ..., yarn node ... or npx node ..., is followed as a command of its own, including everything after --. When Orgabot cannot establish what a command runs (an unreadable script, an unparseable package.json, a flag such as --workspaces that selects an unknown set of packages, a cd into a directory it cannot resolve, a script named by a variable such as bash $SETUP, a glob that runs a script but matches nothing, a bracket or brace expansion such as checks/{a,b}.sh, a named script that is not in the tree, a tool such as make that picks its own files, an interpreter option that runs code Orgabot does not see, such as python3 -c, node -e, node --require, node --test or bash -c (also when wrapped, as in npm exec -- node -e ...), a runner option such as npx -p that names what runs, a -m module whose package is in the tree but whose module is not, a path to an interpreter-like program Orgabot cannot name, such as /usr/bin/python3-dbg, a JavaScript or Python script that imports or runs code by a computed name such as import(dir + file), execSync(cmd) or module["requ" + "ire"](path), a Python script that loads a module by name or runs source text even with a literal argument, such as importlib.import_module("other.gate") or exec("import x"), a Python import that Orgabot cannot read as a statement (an import inside a one-line block such as if ok: import x or try: import x is read and followed, and importing a.b also follows the package files a/__init__.py and a/b/__init__.py), an import of a local relative module that is not in the tree or of a workspace package by name, or a script in a language other than shell, JavaScript, TypeScript or Python), the job is keyed to the whole tree instead.
• The context. A plain commit, or a synthetic merge onto one exact base. A moved base is new work.
• The definition and policy. The job, every job it needs, and the target's settings.
• What actually runs. The registered project's setup and verification commands and its working directory. A job with no commands runs the project's verification, so adding a check there means earlier results no longer count. If that configuration cannot be read, the job always runs.
• The toolchain. The controller's runtime version and every pinned image, action and tool version. An image or action that is not pinned to a digest or commit (for example node:22 or actions/checkout@v4) makes the job always run.
• The platform. Operating system and CPU architecture, unless the job is declared portable.
• Trust. A fork pull request never shares results with a trusted push or pull request, so an untrusted pull request cannot plant a result a trusted run later stands on.
• Who ran it. The controller that signed the result.
• Non-secret configuration. The values of the environment variables a job lists in configEnv. A name that looks like a secret is refused, and the job always runs.
• External state. The vulnerability database version a security job checked against, or the resolved version of a mutable dependency set.

The key is signed into the result. A reused job carries a second signed statement naming the run it reused and the key they share, and the check that GitHub sees is computed from both. An edited run record cannot turn into a cached pass.

What is never reused

• Signing, notarization, publishing and deployment. Jobs with kind set to sign, notarize, publish or deploy always run. Reusing artifacts and reconciling side effects are separate contracts. A job with no kind whose id or commands look like one of these (for example npm publish or codesign) is treated the same way; set kind: "verify" if it only checks something.
• Always-run controls. A job with reuse: "never", or one that lists network in external, runs every time.
• Security audits without fresh advisory data. A job with kind: "security" is reused only when the advisory database version is known and was refreshed within the last 24 hours, and only while that version is unchanged. New advisory data means the audit runs again.
• Scheduled, manual and forced runs. They exist to run. They only reuse their own earlier attempt when you retry the same request, and they never wait on another run that is executing the same job at the same moment.
• Failures. A failed result is never reused as a pass. If the newest run with the same key failed, an older pass with that key is not reused either, so a flaky or newly failing test cannot hide behind it.
• Old results. A pass is reused for up to 7 days by default (maxEvidenceAgeHours), or 24 hours for a security job.

Affected jobs

By default a job's source input is the whole tree, so any change runs it. To run only the jobs a change affects, give each job the paths it depends on:

{
  "id": "ci",
  "sharedPaths": ["config/**"],
  "ignorePaths": ["docs/**"],
  "jobs": [
    { "id": "api", "required": true, "placement": "local", "commands": ["npm test -w api"], "paths": ["packages/api/**"] },
    { "id": "web", "required": true, "placement": "local", "commands": ["npm test -w web"], "paths": ["packages/web/**"], "needs": ["api"] },
    { "id": "macos", "required": true, "placement": "local", "kind": "platform", "commands": ["swift test"] },
    { "id": "audit", "required": true, "placement": "local", "kind": "security", "commands": ["npm audit"] }
  ]
}

The selection is conservative:

• A job also depends on the paths of every job it needs.
• Lockfiles, package manifests, toolchain pins, build configuration, .github/, .orgabot/, scripts/, bin/, ci/ and tools/ are inputs of every job, along with anything in sharedPaths. Changing one runs every job.
• A change to a file no job maps is an unknown change, and every path-scoped job runs.
• ignorePaths marks files whose change affects nothing, such as documentation. Platform and security jobs ignore both paths and ignorePaths and depend on the whole tree, so a change that looks cosmetic never skips them.

A job that is not affected is not skipped. It is satisfied only by a signed pass of the same job with the same key. If no such pass exists, it runs.

Identical jobs at the same time

When two runs reach the same job with the same key at the same time (a mission and the pull request check, for example), only one executes. The other waits for it and attaches its result, pass or fail, to its own run. If the run holding the job stops, the waiting run takes over and executes the job itself.

A portable job's result can stand in on another architecture. A platform job always runs on each architecture it is required on.

Forcing a run

• orgabot cicd verify <project-id> --force runs every job again for that commit.
• orgabot cicd run <owner/repo> <target> <sha> is always a fresh run.

Seeing why

orgabot cicd reuse <run-id> lists, for each job, whether its result was reused and why:

• For a reused result: the run it came from, that the reuse keys match, how old the result is, and the time saved.
• For a job that ran: what changed since the last result. For example, "base/merge context", "external state (advisory database, mutable dependencies)", "toolchain/image/action versions" or "platform/architecture", with the old and new values.
• For a job that can never be reused: the rule that applies.

orgabot cicd inspect <run-id> shows the same decision on each job line. Add --json to either for structured output.

Measured savings

orgabot cicd savings [<owner/repo>] totals reuse hits, coalesced jobs, jobs that ran, and the time saved. The time saved is the measured duration of each job whose result was reused. If that job did not record a duration, the reuse is counted but its saving is left out, never estimated. Each reused job is also recorded as a CI optimization signal (cached with saved_ms), and each run's summary carries the total.