An AI SBOM (Software Bill of Materials) is a machine-readable inventory of all components in an AI system: models, libraries, datasets, APIs, and dependencies. It extends the traditional software SBOM concept to include AI-specific elements such as model provenance (where the model came from, who trained it, on what data), training data lineage, and model version history. Think of it as the ingredients list for your AI system — except most organisations do not have one.

Why do AI systems need a bill of materials?

AI systems need a bill of materials because they are built from components with their own vulnerabilities, licences, and provenance chains. A typical enterprise AI system depends on foundation models, open-source libraries, third-party APIs, training datasets, and fine-tuning data — each a potential attack surface or compliance liability. Without an AI SBOM, you cannot identify which systems are affected when a vulnerability is discovered in a component, you cannot verify licence compliance for model weights, and you cannot trace a supply chain attack back to its entry point.

How does an AI SBOM differ from a traditional software SBOM?

A traditional software SBOM lists code libraries, packages, and their versions. An AI SBOM extends this with model-specific elements: model provenance (origin, author, training methodology), training data lineage (datasets used, consent basis, bias assessments), model version genealogy (base model to fine-tune chain), and inference dependencies (APIs, embedding models, retrieval indices). The AI SBOM must also track non-deterministic components — a code library produces the same output every time, but a model may not.

AI SBOM (Software Bill of Materials) — AI Control Index Glossary

AI SBOM

the ingredients list for your AI. most organisations don’t have one. imagine selling food with no label.

"asked what models we’re running in production. got four different answers from four different teams. one of them was ‘i think it’s GPT-something.’"

"a vulnerability was found in a library we use. took us three weeks to figure out which of our AI systems were affected because nobody tracks dependencies."

"our foundation model vendor changed their model version without telling us. our outputs changed. our tests broke. our SBOM didn’t exist to tell us why."

Canonical Definition

A machine-readable inventory of all components in an AI system: models, libraries, datasets, APIs, and dependencies. Extends the traditional SBOM concept to include model provenance and training data lineage. An AI SBOM answers the question no organisation wants to hear from a regulator: “What exactly is in your AI system, and where did each component come from?”

Why It Matters

The software supply chain crisis that produced Log4Shell, SolarWinds, and the xz utils backdoor is now arriving in AI. The attack surface is larger: AI systems depend not just on code libraries but on models, datasets, embedding indices, and inference APIs — each a potential vector for compromise, each with its own provenance chain that must be tracked.

The threat is not theoretical. Researchers have documented malicious models uploaded to HuggingFace containing hidden code that exfiltrates data when the model is loaded. AI coding assistants have been shown to hallucinate package names — packages that do not exist — and attackers have registered those exact names on PyPI and npm, loaded with malware. Developers who trusted the AI’s recommendation installed packages that were designed from inception to be attack vectors.

Without an AI SBOM, you cannot answer three questions that regulators, auditors, and incident responders will ask: (1) What components are in your AI system? (2) Where did each component come from? (3) When a vulnerability is discovered, which of your systems are affected? The US Executive Order 14110 on AI safety explicitly references SBOM requirements. The EU AI Act requires technical documentation that includes component inventories. The absence of an AI SBOM is not a documentation gap — it is a governance gap with operational, regulatory, and security consequences.

The Stress Test

A critical vulnerability is discovered in a popular machine learning library that your organisation uses. The CISO asks: which of our AI systems are affected? Your engineering teams start investigating. Team A uses the library directly. Team B uses it as a transitive dependency through another framework. Team C downloaded a model from HuggingFace six months ago that was built with the library, but the model itself does not list its build dependencies. Team D is unsure.

Four days pass. You still cannot produce a definitive list of affected systems. The vulnerability is being actively exploited in the wild. You cannot patch what you cannot find. An AI SBOM would have answered the CISO’s question in minutes. Instead, you are conducting a forensic investigation of your own infrastructure.

In the Wild

Supply Chain — Malicious Models on HuggingFace, 2024

Trojan Models in the Open-Source Registry

Security researchers at JFrog and others identified multiple malicious models uploaded to HuggingFace, the largest open-source model repository. These models contained hidden pickle-based payloads that executed arbitrary code when the model was loaded — not when it was run, but when it was loaded. The payloads included reverse shells, data exfiltration scripts, and credential harvesters.

The models were named to mimic popular legitimate models, exploiting the same typosquatting techniques used in traditional package manager attacks. Organisations that downloaded these models without verifying provenance — and without tracking what they downloaded in an SBOM — had no way to identify the compromise or determine which systems were affected.

The model registry is the new package manager. The same attacks that work on npm and PyPI work on HuggingFace. The difference is that most organisations have package SBOMs. Almost none have model SBOMs.

AI Engineering — Hallucinated Package Attacks, 2024–2025

When the AI Invents Dependencies and Attackers Build Them

Researchers documented a novel attack vector: AI coding assistants (including GitHub Copilot and ChatGPT) frequently hallucinate package names in code suggestions — recommending imports from packages that do not exist. Attackers monitored these hallucinated names and registered them as real packages on PyPI and npm, loaded with malware. The attack required no social engineering: developers trusted the AI’s suggestion, installed the package, and the malware executed.

A Vulcan Cyber study found that ChatGPT generated hallucinated package recommendations in approximately 35% of code generation queries that involved package imports. The attack is self-reinforcing: as more developers install the malicious package, it gains download counts and star ratings, making it appear more legitimate to the next developer.

The AI hallucinated a dependency. An attacker made it real. Your developer installed it. Your SBOM — if it existed — would have flagged a package with zero history and suspicious provenance. It did not exist.

Regulatory — US Executive Order 14110, 2023

The Government That Demanded the Ingredients List

US Executive Order 14110 on the Safe, Secure, and Trustworthy Development and Use of AI explicitly addressed AI supply chain transparency. The order directed NIST to develop standards for AI system documentation that include component inventories analogous to software bills of materials. It recognised that AI systems introduce supply chain risks beyond traditional software: model provenance, training data sourcing, and the cascading effects of foundation model vulnerabilities across thousands of downstream applications.

The practical implication: organisations selling AI systems to US government agencies will need to provide AI SBOM documentation. The requirement is moving from guidance to procurement mandate.

The US government is asking for the ingredients list. If you do not have one, you will not be able to sell to the largest single buyer of technology services in the world.

How to Govern It

If you cannot list every component in your AI system, you cannot secure it, audit it, or explain it.

Within the AI Control Index, AI SBOM governance spans multiple layers and shields:

Supply Chain (S3) — The primary control layer. Every AI system must have a machine-readable SBOM listing: models (with version, provenance, and licence), libraries and frameworks (with version and vulnerability status), datasets (with source, consent basis, and classification), APIs and inference endpoints (with provider and SLA), and all transitive dependencies.
Data (L6) — Data lineage feeds into the AI SBOM. Training datasets, evaluation datasets, and RAG knowledge bases are components that must be inventoried with the same rigour as code libraries.
AI Engineering (L5) — The model registry must integrate with the SBOM: every registered model links to its component inventory, and every model version update triggers an SBOM update. The SBOM must be maintained as a living document, not a one-time artifact.
GRC (S1) — Evidence Factory captures AI SBOMs as governance artifacts. When a vulnerability is discovered, the SBOM enables rapid impact assessment. When a regulator asks what is in the system, the SBOM is the answer.
Observability (S4) — Runtime dependency monitoring: detect when a vendor silently updates a model version, when an API endpoint changes behaviour, or when a transitive dependency introduces a new vulnerability.

When It’s Relevant

Every AI system in production or development. The AI SBOM is not a compliance-only artifact — it is an operational security requirement. You need it for vulnerability response, incident investigation, audit readiness, and procurement due diligence.

AI SBOM governance is critical when:

Your AI systems use open-source models downloaded from public registries
You depend on third-party model APIs where the vendor controls model versioning
AI coding assistants are used in your development workflow
You sell AI systems to regulated industries or government agencies
A supply chain vulnerability has been discovered and you need to assess impact within hours, not weeks

Related Terms

References

[1] OWASP Foundation (2025) OWASP Top 10 for Large Language Model Applications v2025.1. LLM05: Supply Chain Vulnerabilities. Available at: owasp.org/www-project-top-10-for-large-language-model-applications.
[2] The White House (2023) Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (EO 14110). Washington, D.C.
[3] JFrog Security Research (2024) ‘Malicious ML Models on Hugging Face: Pickle-Based Attacks in the Open-Source Model Registry’, JFrog Blog.
[4] Lanyado, B. (2024) ‘Can You Trust ChatGPT’s Package Recommendations?’, Vulcan Cyber Research.
[5] NTIA (2021) The Minimum Elements For a Software Bill of Materials (SBOM). National Telecommunications and Information Administration, U.S. Department of Commerce.
[6] European Parliament and Council (2024) Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (AI Act), Article 11: Technical documentation. Official Journal of the European Union.
[7] NIST (2024) Artificial Intelligence Risk Management Framework: Generative AI Profile (AI 600-1). National Institute of Standards and Technology. Supply chain risk management provisions.

AI Control Index v6.0 · Glossary · June 2026 · i-DEPOT 158508 (BOIP) · CC BY-NC-ND 4.0

By Jeroen Janssen, Apparens