AI Code Provenance & Agent Governance

The governance problem: code creation is changing faster than provenance controls

Traditional software provenance assumes that a human author submits a discrete code change into a version-control system, a reviewer looks at the diff, and a build system turns source into artifacts. NIST SP 800-218, the Secure Software Development Framework, makes this assumption explicit in its recommendations for maintaining source-code integrity, build reproducibility, and supplier artifact control. NIST SP 800-218, DOI 10.6028/NIST.SP.800-218, pub 2022-02-04 access 2026-04-23

Large-language-model coding assistants break several parts of that model. A single natural-language prompt can generate multiple functions across multiple files. The “author” is a compound entity: a model version, a prompt, a retrieval context, a set of enterprise policies, and a human accepting or rejecting the output. If the generated code is then modified through an agent loop, the human’s role may be reduced to approving a change set that no individual could fully review.

This is not hypothetical. Pearce et al. at IEEE S&P 2022 tested GitHub Copilot in 89 C/C++ scenarios and found that the model generated vulnerable code in roughly 40% of cases. Pearce DOI 10.1109/SP46214.2022.9833571, pub 2022, access 2026-04-23 That result is a peer-reviewed Tier-1 benchmark, but it is also one research group’s benchmark, and the authors themselves describe scenario limits. This note flags it [SINGLE-SOURCE] for any strong claim about Copilot’s real-world vulnerability rate: no fully independent peer-reviewed replication is cited here.

The governance consequence is that provenance is no longer just “where did this binary come from?” It must also answer:

• Which model or agent produced the code?
• What human approval occurred?
• What policy checks ran before merge?
• What generated artifact was built from which source tree?
• What third-party components are inside the resulting package?

1.1 Provenance formats that are actually deployed

Several independent projects attack parts of this problem.

SLSA — Supply-chain Levels for Software Artifacts. SLSA defines levels 0–3. At level 0 there are no guarantees; at level 1 the build process is documented; at level 2 the build is tamper-proof enough that provenance is generated automatically; at level 3 the build is hardened and provenance is non-forgeable by ordinary maintainers. SLSA v1.0, published 2023, access 2026-04-23 SLSA is not a government standard, but it is widely referenced in supply-chain security engineering. It is Tier-2 as an OSS specification, and its performance/security claims should be treated as self-reported engineering guidance rather than formal certification.

in-toto. The in-toto paper gives the academic underpinning for end-to-end supply-chain integrity. It uses “layout” files to require that each step in a pipeline — code review, test, build, package — be cryptographically signed by the party that performed it. Torres-Arias DOI 10.1145/3319535.3363212, pub 2019, access 2026-04-23 in-toto is part of the Cloud Native Computing Foundation and underlies several real packaging systems. It is a peer-reviewed Tier-1 source for the concept, but its deployment numbers are not established here.

Sigstore. Sigstore provides free keyless signing using short-lived keys bound to OpenID identities. Signatures are logged in the public Rekor transparency log, and Fulcio acts as a certificate authority for ephemeral code-signing certificates. Sigstore docs, access 2026-04-23 This is a technical architecture claim: Sigstore documentation describes it, but documentation is not independent performance or security validation. It should be read as Tier-2 self-reported material.

SBOM formats: SPDX and CycloneDX. SPDX was standardized as ISO/IEC 5962:2021, making it the highest-tier SBOM format because it is an international standard. ISO/IEC 5962:2021, pub 2021, access 2026-04-23 CycloneDX is another SBOM standard, maintained by OWASP, and is widely used in application-security tooling.

NIST’s AI-specific layer. NIST AI Risk Management Framework 1.0 adds the governance practices that traditional software supply-chain guidance does not address: accountability for automated systems, documentation of intended use, and monitoring for drift in deployed AI. NIST AI RMF 1.0, DOI 10.6028/NIST.AI.100-1, pub 2023-01-26, access 2026-04-23 A code agent is an AI system, so provenance records for its outputs sit inside the AI RMF’s documentation-and-accountability expectations.

Patent evidence and what it does not prove

The strongest available granted patent in the generation/provenance perimeter is US Patent 10,956,586 B2, assigned to Microsoft Technology Licensing LLC and granted 2021-03-30. US10956586B2, grant 2021-03-30, access 2026-04-23

The patent is a Tier-1 primary legal document, but it must be read narrowly:

• A patent is not proof of performance. It records claims that an applicant convinced an examiner were novel and non-obvious enough to issue. It does not demonstrate that the described system shipped, that it is commercially successful, or that its security properties hold.
• A patent is not proof of commercial viability. Microsoft’s patent claims do not show that provenance-enabled AI code generation has produced revenue or that customers adopted it.
• A patent from the same assignee that markets Copilot is not independent. If Microsoft patents and GitHub marketing are both in the same corporate lineage, then a patent plus company paper is not two independent sources for a load-bearing technical claim.

We did not find a granted patent in this retrieval window whose independent claims directly require “AI-generated code plus provenance metadata plus governance policy enforcement.” The intellectually honest positioning is that the patent landscape contains adjacent grants — natural-language code generation, supply-chain signing, provenance metadata — but the combined AI-code-provenance-agent-governance moat is not yet established by a single verified granted patent. Any stronger claim would require primary-source patent retrieval beyond this note.

Regulatory and policy drivers

3.1 U.S. federal procurement and SBOM

Executive Order 14028, issued 2021-05-12, ordered federal agencies to improve software supply-chain security. EO 14028, published 2021-05-12, access 2026-04-23

OMB M-22-18, dated 2022-09-14, instructs federal agencies to require software producers to provide SBOMs for software products purchased by the government. OMB M-22-18, published 2022-09-14, access 2026-04-23 CISA maintains SBOM guidance under that mandate. CISA SBOM guidance, access 2026-04-23

The operational consequence for AI code agents is direct: if a vendor sells software to the U.S. government and an AI agent contributed to that software, the provenance question is no longer a best practice; it becomes a procurement condition. But note the gap: M-22-18 requires SBOMs for software products, and the U.S. federal software-procurement rules still treat “AI-generated code” through the general SBOM language rather than a dedicated agent provenance requirement. That gap is a real governance risk, not merely a theoretical observation.

3.2 AI-specific regulation

The EU AI Act entered into force on 2024-08-01. Regulation (EU) 2024/1689, OJ, entry into force 2024-08-01, access 2026-04-23

For code-generation assistants, the EU AI Act generally imposes transparency and documentation obligations on deployers of AI systems, with heavier obligations for high-risk systems. A coding assistant is not automatically high-risk merely because it writes software, but if that software is incorporated into a high-risk system — medical device software, critical infrastructure, safety components — the code generation lifecycle becomes part of the regulated system’s technical documentation and risk management. This is an important governance hinge: the AI act regulates the AI system, while sectoral rules regulate the output artifact, and code provenance is the link between them.

3.3 NIST SSDF and AI RMF

NIST SP 800-218 Version 1.1 provides the Secure Software Development Framework. Its practices include establishing and maintaining supplier relationships, maintaining provenance for internal and third-party code, and verifying that released artifacts match their source. NIST SP 800-218, DOI 10.6028/NIST.SP.800-218, pub 2022-02-04, access 2026-04-23

NIST AI RMF 1.0 applies an AI governance lens to the same lifecycle. It does not replace SSDF; it adds requirements for documented intended use, validity and reliability, and monitoring. NIST AI RMF 1.0, DOI 10.6028/NIST.AI.100-1, access 2026-04-23

The actionable governance stack is therefore: SSDF for build integrity + SLSA/in-toto for attestation + SPDX/CycloneDX for SBOM + AI RMF for AI accountability + agency-specific procurement enforcement.

Marketing claims vs. filed disclosures

GitHub’s marketing and engineering research state that Copilot improves developer speed. A commonly cited GitHub research post claims a coding task completed 55% faster with Copilot than without. GitHub research blog, published 2022-09-07, access 2026-04-23

Microsoft’s SEC filings, by contrast, do not carry that 55% claim as an audited or committed financial metric. Microsoft’s 10-K risk factors discuss AI features in the context of legal, regulatory, and intellectual-property uncertainty, not as a guaranteed productivity outcome. Microsoft FY 10-K EDGAR search, access 2026-04-23

The discrepancy pattern is important:

• Marketing and engineering blogs make performance/security/ease-of-use claims.
• SEC filings use risk-factor language: AI adoption carries litigation, regulatory, and customer-contract risk.
• The security study cited above found Copilot-generated code was vulnerable around 40% of the time in a specific benchmark. Pearce DOI 10.1109/SP46214.2022.9833571

Thus, a “Copilot is 55% faster” claim and a “Copilot-generated code is 40% vulnerable in this study” claim are not directly comparable, but together they show that productivity marketing is not a substitute for provenance and security testing. Any investment or governance thesis that leans only on marketing materials is not relying on filing-grade evidence.

Worked quantitative calculation: false negative rate in provenance verification

This calculation illustrates how provenance verification reduces, but does not eliminate, unauthorized-code risk.

Formula:

Expected undetected malicious/unauthorized changes per week =

E_missed = N * p_fn

Where:

• N = AI-generated code changes entering a repository per week
• p_fn = probability that an unauthorized change is incorrectly accepted as verified by provenance tooling

Inputs and sensitivity:

Worked case:

`

E_missed = N * p_fn

= 1,000 * 0.01

= 10 unauthorized changes per week

`

Sensitivity:

• If p_fn = 0.001, E_missed = 1 per week.
• If N = 10,000, E_missed = 100 per week at p_fn = 0.01.
• If p_fn is actually 0.05 because provenance verification is not enforced on agent-mediated merge loops, E_missed = 50 per week at the original N = 1,000.

Interpretation. The SHA-256 collision probability is so small that accidental hash collisions are not a meaningful risk in provenance identifiers. The dominant risk is governance failure: a false-negative rate in human/policy enforcement, not cryptographic hash failure. Every p_fn above is labeled assumed because no standardized, regulator-published false-negative rate for provenance verification exists in this source set. Any stronger empirical p_fn would be needs primary source validation.

Unit economics / business model

Who monetizes this technology and how:

• AI coding-assistant subscriptions. GitHub Copilot Business is listed at $19 per user per month at the public pricing page. GitHub Copilot pricing, access 2026-04-23 This is a self-reported Tier-2 price, not an SEC-reported average selling price. Competitors follow the same per-seat subscription model.
• DevSecOps and supply-chain security platforms. Vendors embed provenance metadata, SBOM generation, vulnerability scanning, and policy enforcement into CI/CD and artifact repositories. Their public filings show software subscription revenue, but the specific provenance-tool margins are not separately disclosed. Needs primary source validation if a reader wants company-specific gross-margin figures.
• Open-source provenance infrastructure providers. Sigstore, in-toto, SLSA, SPDX, and CycloneDX components are available under open licenses, but monetization occurs through managed hosting, enterprise policy controls, compliance reporting, and support. Exact revenue splits are not public.
• Cloud platform bundling. Microsoft, Google, Amazon, Oracle, and GitLab bundle AI code generation into existing developer platforms, so code provenance governance is monetized indirectly through platform seat expansion and consumption, not as a standalone SKU.

Value capture is likely to remain with the platform that owns the developer workflow: the assistant generates code, the platform records provenance, the SBOM is attached at build time, and the policy gate blocks merge. The vendor that controls that merge gate captures the governance value.

Disconfirming Evidence

The thesis that AI-code provenance and agent governance are becoming a mandatory control would be falsified or materially weakened by:

• OMB M-22-18 and Executive Order 14028 being repealed or defunded, removing the procurement requirement for SBOMs.
• SLSA/Sigstore adoption stagnating and not becoming a default in mainstream CI/CD platforms.
• Independent replication of the Copilot security study finding near-zero vulnerability introduction, undermining the claim that AI-generated code needs special provenance controls. Pearce DOI 10.1109/SP46214.2022.9833571
• EU AI Act enforcement guidance exempting code-generation assistants from documentation requirements when used only as developer tools.
• Buyers accepting vendor self-attestation without independent verification, meaning provenance becomes a checkbox rather than an enforceable control.
• A dominant platform successfully bundling provenance into a proprietary vertical stack, so that open governance standards become irrelevant to market behavior.