Application Layer vs Model Layer in AI

By Brian Tr · Updated Oct 5, 2026

Every AI startup sits somewhere on a stack. Some build chips and data centers. Some train the large models that everyone else rents. Some wire those models into tools, and some sell a finished product to a customer who has never heard of a token. Which layer you pick decides how much capital you need, how fast your costs fall, and who can copy you.

The phrase "application layer vs model layer" is shorthand for the most important choice on that map. The model layer is expensive, scale-driven, and crowded with giants. The application layer is cheaper to enter, closer to the customer, and exposed to a different risk: the layer below you can change what's possible overnight.

This article lays out the four layers, what it costs to compete in each, and how venture investors and researchers have argued that value spreads across them. If you're running a B2B company and deciding whether to build with AI, or evaluating a startup in the space, this is the map to keep in your head.

The generative AI stack in four layers

Most analyses of the stack use three layers: infrastructure, models, and applications. The firm a16z described exactly that split in January 2023: applications that bring models to users, the models themselves (offered as proprietary APIs or open-source checkpoints), and the infrastructure of cloud platforms and hardware makers that runs training and inference.

In practice a fourth layer has grown up in the middle. It's the tooling that connects raw models to real work: retrieval, orchestration, evaluation, guardrails, and the plumbing that turns a model call into a dependable workflow. Splitting it out makes the picture clearer, because a lot of startups live there.

Two things to notice. First, capital intensity falls as you move up. Second, so does your control over the inputs. An application company doesn't set the cost of the model it uses, and it can't stop the model from getting better at the very task it sells.

What each layer actually does

Compute and infrastructure. This layer provides the physical and cloud capacity that makes everything else possible. It's a scale game: a few companies with enormous balance sheets, long lead times for hardware, and demand that depends on the layers above. Startups rarely win here unless they bring a genuinely different approach to hardware or to serving models efficiently.

Foundation models. These are the large, general-purpose models trained on broad data. They're the closest thing the stack has to an engine. A lab trains one model and then sells access to it for thousands of use cases. Because the same model serves many buyers, the economics reward whoever has the most capital, compute, and talent, at least for the frontier.

Tooling and orchestration. This is the middleware. A good example of an archetype is a company that helps teams evaluate whether a model's answers are accurate, or one that manages the retrieval pipeline feeding a model with a customer's documents. It serves builders, not end users, so its fate is tied to how many builders there are.

The application layer. Here the product is a job done, not a model. A contract-review tool built on a third-party foundation model, a support assistant trained on a company's help center, a coding assistant inside an editor: each wraps model capability inside a workflow, a user interface, and a data connection that a business actually buys. If you're curious how the thinnest versions of this layer get criticized, we cover that in AI wrapper vs AI native.

Where startups compete and what it costs

The cost of entry differs by orders of magnitude. Epoch AI estimated in June 2024 that the amortized hardware and energy cost of the final training run for frontier models had grown at 2.4x per year since 2016, and warned that the largest training runs would cost more than a billion dollars by 2027. Epoch's breakdown for major models put hardware at 47 to 67 percent of development cost, staff at 29 to 49 percent, and energy at 2 to 6 percent. That's a projection from 2024, so treat it as a direction rather than a current price tag. But the direction explains why the foundation layer looks the way it does: a handful of well-funded labs, plus open-weight efforts backed by large companies.

For a typical founder with a team of 20 and a seed or Series A round, that math rules out the frontier model race. What's left is the application and tooling layers, and the cheaper edges of the model layer (small specialized models, fine-tunes of open weights). If you're weighing whether to adapt a model or steer it with prompts, fine-tuning vs prompting for agents walks through that decision.

Why does the application layer feel more reachable? Because the price of capability keeps falling underneath it. The Stanford AI Index reported in its 2025 edition that the inference cost for a system matching an early-2023 frontier-class model dropped over 280-fold between November 2022 and October 2024, and that hardware costs have declined 30 percent annually. A product that was uneconomic in 2022 can be profitable by the time you ship. The newest edition shows industry now produces over 90 percent of notable frontier models, which tells you who is spending to push the frontier, and why the rest of the market treats it as a given input.

Key Facts: Layers of the AI Stack

• a16z's early analysis found that infrastructure vendors were capturing the majority of dollars flowing through the stack, while app companies struggled with retention, differentiation, and gross margins (published January 2023).
• The same analysis reported app companies spending around 20 to 40 percent of revenue on inference and per-customer fine-tuning, with gross margins from as high as 90 percent in a few cases to more often 50 to 60 percent.
• Enterprise generative AI spend reached $37 billion in 2025, up from $11.5 billion in 2024, with $19 billion going to the application layer and $18 billion to the infrastructure layer (Menlo Ventures, December 2025).
• Foundation model APIs accounted for $12.5 billion of that 2025 enterprise spend, per the same Menlo Ventures report.
• Frontier training cost has grown about 2.4x per year since 2016 (Epoch AI, June 2024).
• Inference cost for early-2023 frontier-class performance fell over 280-fold from November 2022 to October 2024 (Stanford AI Index 2025).

How value has been argued to accrue

The honest answer is that the argument has moved, and it's still moving. Three snapshots show the shift.

Early 2023: infrastructure wins first. The a16z analysis cited above concluded that infrastructure vendors were likely the biggest winners so far. Applications showed strong top-line growth but struggled with retention, product differentiation, and gross margins, and model providers hadn't yet reached large commercial scale. At that point the money flowing through the stack stopped mostly at the bottom.

Late 2023: the app layer needs its own moat. Sequoia's Generative AI's Act Two (September 2023) said the authors had expected application and foundation model companies to separate cleanly, but that this hadn't happened yet, and that the most successful user-facing applications out of the gate had been vertically integrated. It argued that workflows and user networks seemed to create more durable advantages than data, since the next generation of foundation models could erase a data moat. It also flagged weak retention: the median generative AI app showed a 14 percent daily-to-monthly active user ratio, far below the best consumer apps.

Mid 2024: the infrastructure bill has to be paid for. Sequoia's AI's $600B Question (June 2024) argued that the capital being spent on GPUs far exceeded the revenue AI products were generating, and described a gap of roughly $500 billion a year. Its point on value was that falling compute prices are good for builders and long-term innovation, but bad for investors who bet on prices holding.

Late 2025: spending is flowing to apps. Menlo Ventures' 2025 enterprise report found that more than half of enterprise generative AI spend, $19 billion of $37 billion, went to the application layer. Within that, horizontal AI took $8.4 billion, departmental AI $7.3 billion, and vertical AI $3.5 billion. The report also found that AI buyers converted at 47 percent compared with 25 percent for traditional software.

Put those snapshots side by side and you get a useful, if provisional, pattern. Early on, dollars pooled where the scarce inputs were. As the inputs got cheaper and models got good enough, enterprise budgets moved toward products that finish a job. None of these sources is a prediction of the future, and each is one firm's view, often with a stake in the answer. But the direction is consistent enough to plan around.

The economics of each layer

Here's how the money tends to work at each level, based on the sources above and standard unit-economics logic.

For application companies, the most important number is gross margin. Software has historically been prized for margins in the 70s and 80s. AI products carry a variable cost on every use, because each request calls a model. That's why a16z's early finding of 20 to 40 percent of revenue going to inference and fine-tuning mattered: it meant the old software margin profile didn't automatically apply. Whether that's still the right range today depends on the product, the model mix, and how usage is priced, so check the current numbers for any company you're evaluating rather than assuming.

There's a second effect. Because the model layer is getting cheaper per unit of capability, an application's costs can fall without any work on your side. But so can your competitors' costs, and so can the price customers will accept. Falling input prices help every entrant at the application layer, which makes differentiation come from somewhere other than the model itself.

Application layer versus model layer: how to choose

Think of the decision as three questions.

Do you have a reason to own the model? If your edge is a capability a general model can't deliver, such as a narrow domain with proprietary data and a need for tight control over latency or privacy, a specialized or fine-tuned model may belong in your stack. If your edge is understanding a customer's workflow, renting the model is usually the better bet. The trade-offs are covered in buy vs build for SaaS AI features.

Where does your durable advantage sit? Sequoia's argument about workflows and user networks points to the application layer, where an advantage can come from distribution, integration with a customer's systems, and trust built over time. A model advantage can be matched by the next release. A workflow embedded in a team's daily routine is harder to dislodge. For the framework behind that thinking, see competitive positioning.

How exposed are you to the layer below? Every application depends on a model supplier, and every model builder depends on compute. Mapping your own product's value chain shows which suppliers you could swap and which you couldn't. Companies that can switch between models keep their bargaining power, and companies that can't are one pricing change away from a margin problem.

A useful test: if the model underneath your product improved tenfold next quarter, would your business get stronger or disappear? If it gets stronger because you own the workflow, the customer relationship, and the data, you're building a durable application. If it disappears because your whole product was one clever prompt, you're building something fragile. We look at that second case in AI wrapper vs AI native.

Layers versus focus: vertical and horizontal

The layer you choose is one axis. The other is breadth. A horizontal application serves a function across industries, and a vertical one serves a single industry deeply. Menlo's 2025 split shows both have real budgets, with horizontal at $8.4 billion and vertical at $3.5 billion. We compare the two in vertical AI vs horizontal AI, and the ways each layer earns money are in AI startup business models.

Common misreadings of the stack

"The model layer will take everything." It's tempting to think the labs will absorb every use case. The Sequoia analysis points the other way: the most successful applications were vertically integrated, which suggests that owning the workflow and the user matters as much as the model. But the risk is real for thin products, because a model getting better at a task can erase a product built on the gap.

"The application layer is just a wrapper." Some applications are. Many aren't. A product that embeds in a company's systems, accumulates its context, and learns its processes does a lot more than pass text to a model. The critique is accurate for the thinnest cases and unfair for the rest.

"Infrastructure always wins." The 2023 analysis found infrastructure capturing most dollars, and that was true for that moment. But the 2024 analysis argued that the revenue base hadn't yet caught up with the spending, and the 2025 data shows enterprise budgets tilting toward applications. Positions in the stack shift over time.

"Pick one layer and stay there." Plenty of companies straddle layers. A vertical application may fine-tune a small model, and a model company may ship its own application. The stack is a way to understand dependencies and economics, not a set of walls.

What this means for founders

Start with the customer's job, not the layer. Pick the layer that lets you solve that job with the least capital and the most durable edge, then design for the layers around you changing. Keep your model supplier replaceable, measure your gross margin per customer from day one, and build the parts of the product that a better model can't copy: your integrations, your workflow, your data position, and your relationships.

The stack is also a moving target. The price per unit of capability is dropping, the number of competent models is growing, and the enterprise budget is moving toward products that finish work. Revisit your layer choice at least once a year.

Related Reading

• AI wrapper vs AI native
• Vertical AI vs horizontal AI
• AI startup business models
• What is an AI-native startup
• Buy vs build for SaaS AI features
• Gross margin