The History of AI Governance

By Victor Hoang · Updated Oct 24, 2026

Until the late 2010s, there was little shared paperwork saying what a model was for. Nothing said what data it learned from, or who answered for it. Governance is what filled that gap, one document and one committee at a time.

AI governance is the set of rules, roles, checks and records an organization or a group of countries uses to keep AI systems pointed at the right purposes. It didn't start with a law. It started with a warning from the mathematician Norbert Wiener in 1960 and borrowed a habit of scientific self-policing from a 1975 biology conference. Published ethics principles exploded in 2016-2019. Risk frameworks, management standards and safety institutes hardened from 2023. The throughline: each layer appeared because the one before it was too vague to run a company on.

This article follows that build-up and says plainly which pieces are voluntary, which are standards, and which are law. For the field's wider story, see The History of AI. For the binding-law side, see The History of AI Regulation.

Timeline at a Glance

The Warning Years (1960-2015)

A mathematician and a biology meeting set the template

Governance of AI starts as an argument before there is any practice. In Science on 6 May 1960, Norbert Wiener published "Some Moral and Technical Consequences of Automation". His worry was a machine whose action is so fast and irrevocable that people can't step in before it finishes. In that case, he wrote, we'd better be sure the purpose in the machine is "the purpose which we really desire". That's a governance idea in one sentence: specify the goal carefully, because you may not get to correct it. Wiener's wider cybernetics work is part of the story in The History of AI Agents.

The working template came from another field. From February 24 to 27, 1975, scientists met at the Asilomar Conference Grounds in Pacific Grove, California, to decide how recombinant DNA research could continue safely. A committee including Paul Berg, Maxine Singer and David Baltimore organized it, and about ninety invitees from the United States and about sixty participants from twelve other countries attended, along with sixteen members of the press and four lawyers. On the last day they ended a voluntary moratorium and issued a provisional statement that research could go on if containment safeguards were in place. The NIH then created an advisory committee, and its guidelines followed in 1976. It wasn't about AI, but it showed a pattern the AI world would copy: researchers publish their own safeguards, then institutions formalize them.

The next milestones in this story arrive in 2016. What the early period supplied was a question (who checks the purpose?) and a procedure (open meetings, written statements), not instruments a company could adopt.

For teams today: the oldest governance question is still the best first one. Before you deploy a system, write down the purpose you actually want, and what you'll do if it acts faster than you can review.

The Principles Rush (2016-2019)

Everyone publishes a list of values

By 2016 powerful machine learning (see The History of Machine Learning) had moved out of labs. On September 28, 2016, Amazon, Facebook, Google, IBM and Microsoft announced a nonprofit partnership to develop best practices on ethics, fairness, transparency, privacy and reliability. Each held a seat on a 10-member board, and the group said it wouldn't lobby governments.

Researchers moved next. On August 11, 2017, the Future of Life Institute published the 23 Asilomar AI Principles, grouped under Research Issues, Ethics and Values, and Longer-term Issues. The name shares its venue with the 1975 meeting, but the content was new: a list of AI principles written in the research community.

Companies followed. On June 7, 2018, Sundar Pichai published Google's AI Principles: seven objectives, including avoiding unfair bias, building in safety testing and staying accountable to people, plus four kinds of application Google said it wouldn't pursue, among them weapons and surveillance that violates international norms. The company later updated the page, which now notes a revision on February 4, 2025. Governments and expert bodies joined in. On January 23, 2019, Singapore's IMDA announced a voluntary Model AI Governance Framework covering internal governance, the AI decision-making model, operations management and customer relationships. On April 8, 2019, the European Commission's High-Level Expert Group published its Ethics Guidelines for Trustworthy AI with seven requirements, from human agency and oversight to accountability. In May 2019 the OECD adopted five values-based principles: inclusive growth and well-being, human rights and democratic values, transparency and explainability, robustness and safety, and accountability. The OECD revised them in May 2024.

The rush had a visible stumble. Google announced an external advisory council in late March 2019, and dissolved it about a week later after a staff petition of more than 2,000 signatures over one member, saying the council "can't function as we wanted". The lesson wasn't that ethics boards are pointless. It was that membership, mandate and decision rights are governance design, not decoration.

This era solved the vocabulary problem: there was now a shared list of values. It created a new one: the gap between principle and practice. A principle like "transparency" doesn't tell an engineer what to write down on Tuesday.

For teams today: a values statement is a starting point and nothing more. Attach each principle to an owner, a check and a record, or it won't change a single decision.

The Paper Trail (2018-2022)

Documentation turns principles into artifacts

The answer to "what do we write down?" came from researchers. In a paper first submitted on March 23, 2018, Timnit Gebru and six co-authors proposed datasheets for datasets, modeled on component datasheets in electronics, describing a dataset's motivation, composition, collection process and recommended uses. On October 5, 2018, Margaret Mitchell and co-authors posted Model Cards for Model Reporting, presented at the FAT* conference in Atlanta in January 2019: short documents that report a model's performance across demographic and other groups and describe its intended use and evaluation. Both ideas are cheap and concrete, which is why they spread. For the models these documents later had to describe, see The History of Large Language Models.

Documentation also scaled up to the global level. In November 2021, 193 UNESCO member states adopted the Recommendation on the Ethics of Artificial Intelligence, which UNESCO describes as the first global standard on AI ethics.

Testing joined the paper trail next. In February 2022, Ethan Perez and colleagues proposed using one language model to generate test cases that expose harmful behavior in another, scored by a classifier. That's an early research version of red teaming for language models, in which a team tries to make a system fail before users do.

This era gave governance its working objects: the datasheet, the card, the test log. The new problem was consistency. Every company documented differently, and a buyer couldn't compare two systems.

For teams today: keep one page per AI system covering purpose, data, known limits and test results. If a vendor can't provide theirs, treat that as information.

The Standards Age (2022-2024)

Shared frameworks make governance auditable

Standard-setting bodies answered the consistency problem. On January 26, 2023, NIST released AI RMF 1.0, a voluntary framework organized around four functions: Govern, Map, Measure and Manage. NIST followed on July 26, 2024 with a Generative AI Profile, NIST-AI-600-1, to apply it to generative systems (see The History of Generative AI).

Then came the management-system standard. ISO/IEC 42001 was published on December 18, 2023. It specifies requirements for setting up, running and continually improving an artificial intelligence management system inside an organization of any size, which matters because it's something an auditor can check against. Industry also organized itself again: on July 26, 2023, Anthropic, Google, Microsoft and OpenAI launched the Frontier Model Forum to advance safety research, identify best practices and share knowledge with policymakers.

Governments were moving in parallel. The EU's AI Act, a binding law, entered into force on 1 August 2024, and its story belongs to The History of AI Regulation. The point for governance is that voluntary frameworks and standards became the practical way to prepare for it.

This era solved the comparability problem. Its new problem was scope: a management standard governs an organization, but what about the most capable general-purpose models, whose risks cross borders?

For teams today: pick a framework instead of inventing one. The NIST functions are a plain way to structure a risk register, and a standard like ISO/IEC 42001 gives you something to be audited against.

The Institute Era (2023-present)

Governments build technical bodies to test frontier models

The last layer is state-backed evaluation. On 1-2 November 2023, at Bletchley Park, 28 countries and the European Union signed the Bletchley Declaration, which says developers of the most powerful systems carry a particularly strong responsibility for safety, including testing and evaluation. The UK then described its Frontier AI Taskforce, announced in April 2023, as becoming the AI Safety Institute, a state-backed body to run evaluations, do safety research and exchange information. It was explicitly not a regulator.

The model spread. On February 8, 2024, the US Commerce Department announced the AI Safety Institute Consortium with more than 200 member organizations. At the Seoul summit on May 21-22, 2024, a statement of intent signed by ten countries and the EU committed national institutes to cooperate as a network. For where this meets the agent systems of the last few years, see The History of AI Agents, and for what comes next, The Future of AI History.

The history is still open. The live questions are the old ones from 1960: who sets the purpose, who can intervene, and who is accountable when they can't.

For teams today: you don't need an institute to borrow its method. Run your own evaluations on your own tasks before launch, and keep the results.

Key People

Key Facts

• Wiener's governance warning ran in Science on 6 May 1960.
• The Asilomar AI Principles list 23 principles.
• The EU expert group's guidelines set out seven requirements for trustworthy AI.
• UNESCO's 2021 Recommendation was adopted by 193 member states.
• NIST AI RMF 1.0 appeared on 26 January 2023, and ISO/IEC 42001 on 18 December 2023.
• The Bletchley Declaration was signed by 28 countries and the EU.

The Pattern Behind the History

Across five eras, AI governance grew by turning each vague layer into a more checkable one. We call it the Vague-to-Verifiable Pattern. It's our synthesis of the history above, not a research finding.

1. Warn. A thinker names the risk (Wiener, 1960).
2. Self-police. Practitioners write their own safeguards (Asilomar, 1975, then the 2017 principles).
3. Publish values. Companies and bodies post principles (2016-2019).
4. Document. Datasheets, model cards and test logs make claims inspectable (2018-2022).
5. Standardize and institutionalize. Frameworks, certifiable standards and institutes make checks repeatable (2023-present).

What It Teaches Teams Today

• Start with purpose. Write down what the system is for and who can stop it.
• Turn each value into a record. A principle without an owner and an artifact doesn't govern anything.
• Design the committee. Mandate, membership and decision rights decide whether a review board survives contact with a controversy.
• Borrow a framework. NIST's four functions and ISO/IEC 42001 save you from inventing a process.
• Test before launch. Red-teaming and evaluations are cheaper than recalls.

For the concept definitions behind these steps, see AI governance, the AI governance framework and AI governance for executives.

AI governance shows that people kept trying to answer a 1960 question with whatever tools each decade offered: essays, then meetings, then lists, then paperwork, then standards. When you review your own program, ask which layer you actually have. For related history, see the AI timeline and The History of Expert Systems.