Scoring methodology v1
This directory measures sscsb-control adoption — not general security. The rules below are versioned; every repo page names the version that scored it.
sscsb init before verifying —
which installs the very files many controls check for. So we snapshot the file
list first: evidence the scanner created never counts. And a
check that could not run is unverified — a third state, shown
hatched, outside every denominator. An unperformed check is never a verdict.The scan protocol
This directory measures sscsb-control adoption — how much of the supply-chain posture sscsb can bootstrap and verify a repository has actually committed to. It is not a general security audit, and in methodology v1 a repository using an equivalent tool sscsb doesn't model (Dependabot in place of Renovate, say) scores a gap for that control. Tool-equivalence mapping is a roadmap item for a future methodology version; every version bump is recorded here and displayed on each repo's page.
- Shallow-clone the repository's default branch. The target's code is never executed.
- Snapshot the committed file list (
git ls-files). - Run
sscsb init, thensscsb verify --format jsonandsscsb report --format json. - Reclassify: any control whose passing evidence was created by init scores gap, per the class rules below.
- Delete the clone. A maintainer reviews every record before it publishes.
Evidence classes
| Class | Controls | Rule |
|---|---|---|
| A — committed artifacts | secrets, gittuf, pr-template, sbom, vuln-scan, renovate, sigstore-signing, slsa-provenance, github-attestations, sbom-attestation, model-signing, provenance-verify, release-immutability, octo-sts, sast, codeql, fuzzing, wait-for-secrets, dependency-track, security-insights, best-practices-badge, osps-baseline | The control's evidence is files committed to the repository. If any registered artifact was created by the scanner's own `sscsb init` (absent from the pre-init file snapshot), the control scores gap — evidence the scanner installed seconds earlier is never the repository's evidence. A pre-existing artifact that fails sscsb's shape checks is a real fail. For tool-backed controls (secrets, sbom, vuln-scan, sast, provenance-verify) whose raw verdict only reflects scanner-machine tool availability, the committed artifacts decide instead. |
| A′ — static audits of committed workflows | actions-audit, harden-runner, workflow-audit-extended | actions-audit, workflow-audit-extended, and harden-runner parse every workflow file. With zero pre-existing workflows the verdict would be vacuous, so it scores unverified. Otherwise the raw verdict maps directly: sscsb's own installed templates pass its audit by construction, so init can only pollute toward pass — a fail always implicates the repository's own workflows. |
| B — live remote checks | branch-protection, scorecard | branch-protection and Scorecard query GitHub itself; init cannot influence them, so raw verdicts map directly. Scorecard's live alert feed requires permissions a cross-repo scan lacks — that half is recorded as unverified, never guessed. |
| C — local environment | commit-signing, agent-signing, signing-model, ai-trailers, ai-dep-gate, ai-receipts, package-trust, bumblebee, grype, socket-firewall, witness, sighthound, guac, openvex, oras | Commit signing, signing-model posture, AI trailers, package-trust hooks and similar controls describe the development machine, which a repository scan cannot observe. They always score unverified — an unperformed check is a third state, never a pass or fail. |
| M — meta / informational | secure-repo, compliance-map | compliance-map (about sscsb itself) and secure-repo (an external service pointer) are excluded from scoring entirely. |
Scope
A control is in scope when it is enabled by sscsb's defaults or by the
repository's own committed .sscsb/config.toml. Disabling a default-on
control scores a gap — the denominator cannot be shrunk. Enabling an optional
control puts it in scope against real evidence — never free points.
countable = pass + fail + gap
phase % = 100 · pass / countable
overall = Σ pass / Σ countable
coverage = Σ countable / |scope|Unverified and info are never in any denominator, and zero countable controls in a phase means "no evidence" — not 0%.
A+ = exactly 100% · A ≥ 90 · B ≥ 80 · C ≥ 70 · D ≥ 60 · F below. Coverage under 50% earns NA — insufficient evidence for any letter; under 75% the letter is provisional.
| Grade | Overall |
|---|---|
| A+ | exactly 100% |
| A | ≥ 90%, < 100% |
| B | ≥ 80%, < 90% |
| C | ≥ 70%, < 80% |
| D | ≥ 60%, < 70% |
| F | < 60% |
Evidence coverage below 50% yields NA — insufficient evidence for any letter. Coverage between 50% and 75% marks the letter provisional. This inherits sscsb's own doctrine: exit code 0 is not a clean bill of health, and an unperformed check is never converted into a verdict.
Changelog
- v1 — initial methodology: diff-based init reclassification, five evidence classes, academic grade scale with A+ reserved for exactly 100%.