From AI to SI: The Next Big Thing Is Intelligence

Superintelligence (SI) is a system that outperforms the best humans across essentially all cognitive work, and the practical gap between AI and SI is a verification-cost gap, not a raw-capability gap. A system is superintelligent for a given buyer only on tasks where its output beats the best available expert and checking that output costs far less than producing it. That makes "how smart is it?" the wrong question and "what would it cost me to verify this?" the useful one.

AI Overview

Superintelligence is a system that exceeds the best humans across essentially all cognitive work, a definition that traces to I.J. Good in 1965 and Nick Bostrom in 2014. Measured capability is uneven, so SI does not arrive at one threshold. It arrives task by task, ordered by how cheaply each task's output can be verified. The Verification Ceiling Model formalizes this with a break-even rule that compares the value of a correct output against verification cost and the expected cost of undetected error. Labs use the term as branding and fundraising language, so the decision-relevant test is local and economic rather than philosophical.

Key Facts

What Superintelligence Actually Means, and Who Defines It

Three definitions dominate, and they answer different questions.

I.J. Good, in "Speculations Concerning the First Ultraintelligent Machine" (1965), defined an ultraintelligent machine as one that can far surpass all the intellectual activities of any man however clever. Good's benchmark is the single cleverest individual, and his interest was the recursion: a machine that designs better machines.

Nick Bostrom's Superintelligence: Paths, Dangers, Strategies (Oxford University Press, 2014) defines it as an intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest. The benchmark shifts from the best person to humanity's general performance, and "virtually all" softens Good's "all."

Google DeepMind's "Levels of AGI" paper (Morris et al., 2023) drops the single threshold. It crosses performance (from emerging to superhuman) with generality (narrow versus general), producing a grid. Its top general level, Superhuman, is outperforming all humans, which the authors identify with artificial superintelligence.

These are not interchangeable. Good's bar is one person at the peak. Bostrom's bar is broad exceedance of human performance. DeepMind's bar depends on which cell of the grid is meant. A vendor who says "superintelligent" without naming one of these has not made a testable claim.

The word has also become branding. Safe Superintelligence Inc. launched in June 2024 with the stated aim of building safe superintelligence as its single product. Meta set out a "personal superintelligence" vision in July 2025 and groups its AI work under Meta Superintelligence Labs. Microsoft AI's Mustafa Suleyman described a "humanist superintelligence" approach in November 2025. Each uses the term differently, and none of the three publishes an operational test for it.

This article uses the terms consistently: narrow AI (strong within a bounded task), AGI (human-level breadth), and SI (beyond the best humans across essentially all cognitive work). Background on the parent field is in the artificial intelligence concept page, and on the current technology wave in the generative AI concept page.

Why the AGI-to-SI Ladder Is the Wrong Mental Model

The standard picture is a ladder: narrow AI, then AGI, then SI, each a threshold crossed once. A ladder implies a single number, a general intelligence, that rises over time. The evidence points to a different shape.

Measured capability is jagged. The same model family can beat expert practitioners on a graded exam and fail a simple multi-step chore that a new hire handles. A single rung cannot describe that profile, because the model sits on different rungs for different tasks at the same moment.

The METR study "Measuring AI Ability to Complete Long Tasks" (March 2025) shows what a better measure looks like. It does not score "IQ." It measures the length of task, in human-time, that a model completes at a stated reliability, and reports that this length has been doubling roughly every seven months over the period studied. The unit is task length at a given success rate. That is a per-task, per-reliability quantity, which is the shape a decision-maker needs.

Two consequences follow. First, "when is AGI?" has no single answer, because the answer depends on which tasks and what reliability. Second, the label is doing positioning work as well as descriptive work, an argument developed in the AGI claim as a business strategy. The compute side of the same story, where returns are moving from pre-training toward inference-time effort, is covered in Scaling Is Changing Shape.

If there is no single threshold, the useful question becomes which tasks have crossed their own threshold, and how a buyer would know.

The Verification Ceiling Model

The Verification Ceiling Model starts from a simple observation: delegating a task to a system only pays if the buyer can tell whether the output is right. Intelligence that cannot be checked is, economically, indistinguishable from confident noise.

Three variables.

• Generation cost: what it costs to produce the output, by machine or by human.
• Verification cost: what it costs to establish that the output is correct.
• Error cost: what an undetected wrong output costs once it is acted on.

The break-even rule.

Net value of delegating = (V × P) − C_v − ((1 − P) × (1 − d) × E)

Here V is the value of a correct output, P is the probability the output is correct, C_v is the cost of verifying it, d is the probability that verification catches a wrong output, and E is the cost of an undetected error. Delegation is worth it when this is positive and beats the alternative of doing the work directly.

The "superintelligent for you" test.

A system is superintelligent for you on a task when two conditions hold. Its output is better than your best available expert's, and C_v is a small fraction of what it would cost you to do the task yourself. The second condition is the ceiling. Without it, a better-than-expert answer is not usable, because acting on it means trusting it blind.

A worked break-even calculation (illustrative)

All numbers below are illustrative assumptions chosen to show the mechanism. They are not measurements of any real system.

Assume V = $1,000 for a correct output and E = $4,000 for an undetected error. Doing the task directly costs $600.

Table caption: all probabilities, costs and net values are illustrative assumptions, not data.

The arithmetic for two rows shows the mechanism. Code: 1,000 × 0.80 = 800; minus 50 verification = 750; minus (0.20 × 0.05 × 4,000 = 40) undetected error = +710. Strategy memo: 1,000 × 0.60 = 600; minus 400 verification = 200; minus (0.40 × 0.50 × 4,000 = 800) undetected error = −600.

The memo is not worse because the model is dumber. It fails because verification is expensive and only half-effective, so most wrong answers pass through. Raise P to 0.90 with everything else fixed and the memo row becomes 900 − 400 − (0.10 × 0.50 × 4,000 = 200) = +300. That is positive but still well below the code row, and it leaves verification at two-thirds of direct cost, which fails the fraction test. Capability gains help, but they cannot lift a task above its verification ceiling.

Why the order is what it is

The ordering falls out of one property: whether the world supplies a cheap, trusted check.

• Code with tests has an executable oracle. The test suite is the verifier, and it runs in seconds.
• Formal proofs have a proof checker. A machine-checked proof is correct or it is not.
• Drug-candidate screening has a cheap first filter (computational scoring) and an expensive final one (wet-lab confirmation), so verification cost is medium. In the table, medium means verification takes about half of the direct cost, and at these inputs that is enough to push the row negative.
• Strategy memos have no oracle. The check is expert judgment, which costs close to what producing the memo costs.
• Forecasts resolve only after the event, and a single probabilistic forecast cannot be judged right or wrong by one outcome. Verification is slow and statistical.

Superintelligence therefore arrives per domain, in order of verifiability, and the early pockets look like narrow dominance rather than general minds. For how to pair specific tools with jobs, see the verify-cost method by job. For the discipline of checking at scale, see Verification Is Not Optional.

Where Verification Is Already Cheap: Early SI Pockets

Some domains already show machine performance at the level of strong human specialists, and in some cases beyond it, with output that can be checked by something other than trust.

Formal mathematics. In July 2024 Google DeepMind reported that AlphaProof, which writes proofs in the Lean formal language, together with AlphaGeometry 2, solved four of six International Mathematical Olympiad problems at silver-medal standard. That is strong-specialist performance, not best-human performance. The relevant property is that Lean checks every step mechanically, so checking a finished proof is cheap regardless of how it was found. The verification cost is not literally zero, though: the problems were translated into the formal language by hand, so end-to-end checking still carried human effort.

Protein structure. The 2024 Nobel Prize in Chemistry recognized computational protein design and protein structure prediction, the latter through AlphaFold. Predictions can be compared with experimentally determined structures, which supplied a standing external check through community assessments.

Tested code. Coding agents operate against test suites, and their value tracks how complete those suites are. The economics are covered in When Code Writes Itself. A weak test suite lowers d in the model, which is why the same agent looks excellent in one repository and risky in another.

Drug-candidate screening. Computational ranking of molecules is cheap to produce and check against assay data, but the decisive confirmation is a wet-lab or clinical result that takes weeks to years. Machine output here is a prioritization aid whose value depends on how much of the expensive confirmation it lets a team skip.

What remains unverified outside these pockets is most of the economy. No public evidence shows a system that outperforms the best human experts on open-ended strategy, judgment under novel conditions, or long-horizon planning, with outcomes checked independently. Absence of evidence is not proof of absence, but a buyer cannot act on a claim that has no check attached.

What the Money Says: SI as a Financing Story

Read lab announcements as two kinds of statement. Some are observable: capital expenditure, hiring, funding rounds, announced compute commitments. Others are asserted: timelines, definitions, and claims that a threshold is near.

The observable layer is large and real. The asserted layer carries most of the narrative weight. OpenAI's February 2023 "Planning for AGI and Beyond" and its May 2023 "Governance of Superintelligence" set the template: a statement that systems could exceed expert skill in most domains within roughly a decade, paired with a governance proposal. Those texts attribute a possibility to the authors. They do not report a measurement.

The contrarian position here is that the AGI and SI labels function mainly as financing and positioning devices. A company that must justify very large compute commitments benefits from a goal with no upper bound, because it makes any spend look proportionate to the prize. A named lab such as Safe Superintelligence Inc. or Meta Superintelligence Labs also uses the word to recruit, which is a talent signal as much as a technical one. None of this implies the claims are false. It implies they carry information about incentives, and little about readiness.

Valuations built on that story are examined in AI Valuations Are Detaching, the spending pattern in Who Profits From the AI Buildout, and the pressure on model-level differentiation in the AI commoditization cliff. If models commoditize while verification stays scarce, the durable value shifts to whoever supplies the checks.

What Changes for Operators If Verification Is the Bottleneck

If the ceiling is verification, three practices follow.

1. Build evaluation before buying capability. A test harness, a labeled evaluation set and a regression suite raise d and lower C_v for every future model. They keep their value as models change. The production angle is covered in Offline Accuracy Is a Trap, and the discipline in the model evaluation concept page.
2. Price human review as the scarce input. If checking costs two-thirds of doing, headcount savings are illusory. Budget reviewer hours the way a factory budgets inspection.
3. Prefer tasks with machine-checkable outcomes. Where a task can be reshaped so that a test, a checker or a measured result exists, redesign it. The same underlying model becomes more valuable when its output is verifiable.

Three questions before accepting an "SI-level" vendor claim

1. Better than whom? Name the human baseline: the best expert, a median professional, or a student. Good, Bostrom and DeepMind each choose a different one.
2. Verified by whom? Look for an independent check, not a vendor-run benchmark. Benchmarks saturate and can leak into training data.
3. At what checking cost? Estimate C_v for your own workflow and compare it with the cost of doing the task yourself. If it exceeds a small fraction, the claim does not change your decision.

Why It Matters

Strategically, abundant capability moves the moat. If many models can generate a good answer, advantage accrues to whoever can verify, route and integrate answers reliably, a thesis developed in The Orchestration Layer. Buying decisions should follow the verification stack, not the headline label.

For labor, displacement follows verifiability. Tasks with cheap checks are exposed first, because the worker's remaining role is supervision and exception handling. Tasks whose quality can only be judged by experts keep their human core longer. The sequencing question is explored on the future of work concept page.

For governance, the term has already entered policy. In October 2025 the Future of Life Institute published a Statement on Superintelligence calling for a prohibition on developing it until there is broad scientific consensus that it can be done safely and strong public buy-in. The only binding regime currently in force that touches this area is the EU AI Act, Regulation (EU) 2024/1689, which regulates by risk category and sets obligations for general-purpose models with systemic risk. It does not define or regulate "superintelligence" as such. Its reach is discussed in the EU AI Act and the Brussels effect.

Limitations

The Verification Ceiling Model is a decision heuristic, not an empirical law. It has not been fitted to measured data.

• Ratings are editorial judgment. The cheap, medium and expensive labels reflect reasoning about each task class, not a survey.
• Numbers are illustrative. Every probability and dollar figure in the worked example is an assumption. Real P, C_v, d and E vary widely by organization.
• Benchmarks saturate and leak. Public tests lose discriminating power as models improve and can contaminate training data, which weakens any external evidence of superiority.
• Detectability is assumed. The model presumes errors are detectable in principle. For some strategy and forecasting work, a wrong output may never be identified as wrong, so d is unknowable.
• Baseline comparison is simplified. The rule compares net value against zero and a direct-cost fraction, not against every alternative workflow.
• No timeline is offered. The article does not estimate when, or whether, SI in the general sense will arrive.

Related Reading

• Agentic AI claims as a business strategy: why the AGI label works as positioning.
• Scaling Is Changing Shape: how the compute picture is shifting.
• Verification Is Not Optional: checking AI output at scale.
• The verify-cost method by job: tool choices by task.
• When Code Writes Itself: the economics of the cheapest-to-verify domain.
• Offline Accuracy Is a Trap: building evaluations that predict production.
• The artificial intelligence hub collects the full topic cluster.

References

• I.J. Good, "Speculations Concerning the First Ultraintelligent Machine," Advances in Computers, vol. 6 (1965).
• Nick Bostrom, Superintelligence: Paths, Dangers, Strategies, Oxford University Press (2014).
• Meredith Ringel Morris et al., Levels of AGI: Operationalizing Progress on the Path to AGI, Google DeepMind (November 2023).
• METR, Measuring AI Ability to Complete Long Tasks (March 2025).
• OpenAI, Planning for AGI and Beyond (February 2023).
• OpenAI, Governance of Superintelligence (May 2023).
• Safe Superintelligence Inc., launch announcement (June 2024).
• Meta, Personal Superintelligence (July 2025).
• Microsoft AI, Towards Humanist Superintelligence (November 2025).
• Future of Life Institute, Statement on Superintelligence (October 2025).
• Google DeepMind, AI solves IMO problems at silver-medal level (July 2024).
• The Nobel Committee, The Nobel Prize in Chemistry 2024, press release (October 2024).
• European Union, Regulation (EU) 2024/1689 (Artificial Intelligence Act) (2024).

The Bottom Line

Superintelligence will not announce itself with one event. It will appear first in the places where checking is cheap, in tested code, formal proofs and the first stage of screening, and it will lag wherever the only available check is another expert's time. Definitions differ, labels serve financing and recruiting, and capability is uneven, so none of those settles a buying, building or allocation decision. Verification cost does. The one question to carry away is: what would it cost me to verify this?