AI Product Adoption Metrics

By Brian Tr · Updated Oct 7, 2026

Adoption of an AI product is the share of intended users who repeatedly complete a real task with it and keep doing so. It isn't signups, logins or seats sold. Those numbers are easy to collect and flatter almost every AI product, because people try AI tools out of curiosity and many stop after a week.

This page lays out a measurement system for founders: a framework to organize the metrics, the specific metrics that matter for AI, and the places where usage and value come apart. It's about how to measure, not how to improve. For the improvement side, see onboarding and time to value and AI product churn.

Why Signups and Logins Mislead

The problem is sharper for AI. A typical AI product has a novelty curve: heavy first-week use, then a drop once the buyer sees the limits. A buyer can also be paying for seats that nobody opens. And a user can use the product every day while the output goes into the bin. Each of these looks fine on a dashboard of logins.

Good measurement separates four questions: did people start, do they come back, do they use it deeply, and does it work? Those map closely onto a published framework, so use it instead of inventing one.

The HEART Framework

Rodden, Hutchinson and Fu of Google proposed HEART in Measuring the User Experience on a Large Scale (CHI 2010). It stands for Happiness, Engagement, Adoption, Retention and Task Success. The authors present it as a set of categories from which a team picks specific metrics, not as a fixed list of numbers.

The paper contrasts HEART with what it calls PULSE metrics: page views, uptime, latency, seven-day active users and earnings. PULSE metrics matter, the authors say, but they track business and technical health rather than user experience quality. HEART is meant to complement them.

The paper also supplies the working method: Goals, Signals, Metrics. First state what a product or feature should achieve for users, then name the signals that would show it happened, then build metrics from those signals.

Three details from the paper apply directly to AI products.

Not every category fits every product. The authors note that Engagement may not be meaningful in an enterprise setting where people are expected to use the product as part of their jobs. In that case a team might lean on Happiness or Task Success. Most B2B AI products sit in this situation, so don't force daily-use targets on a tool people open when the work arrives.

Report engagement per user, not in total. The paper recommends averages per user, because a rising total can come from more users rather than more usage. A founder who watches total prompts per month will see growth every month while the product stays shallow.

Define "adopted" as a completed task. The paper says that in some products visiting the site might count as using it, while in others a visitor counts as having adopted only after completing a key task. For an AI product, the second definition is almost always right.

Activation: Defining the First Real Use

Activation is the moment a new user gets a first result they'd have wanted to keep. It sits between signup and habit, and it's the metric most worth defining carefully, because everything downstream depends on it.

A usable activation definition has three parts:

1. A specific action, such as "ran the first workflow on real data" rather than "created an account."
2. A quality condition, such as "accepted or exported the output."
3. A time window, such as "within the first seven days."

Measure the share of new accounts that hit it, and the time it takes them to get there. If your activation definition can be met by clicking around with sample data, it's measuring curiosity. Tie it to the user's own work.

Depth and Frequency of Use

Once activated, the question becomes how much and how often. Three families of metric cover it.

Frequency. The HEART paper's Gmail example is a useful pattern. The Gmail team wanted a better engagement signal than seven-day active users, so it measured the percentage of active users who visited on five or more days in the last week. The authors report that this metric was strongly predictive of longer-term retention. The lesson for AI products: pick a frequency threshold that matches the job, and check whether it predicts who stays.

Stickiness ratios. DAU/MAU (daily active users divided by monthly active users) is the common shorthand. It suits products people use daily and misleads for products people use weekly or monthly by design. Choose the window that matches the natural cycle of the work, such as weekly active over monthly active.

Depth. Count what users do per session, not whether they showed up. Useful depth signals for AI products:

• Tasks completed per active user, per week
• Share of users running more than one type of task
• Share of work routed through the product versus done another way
• Use of features that need more setup, such as integrations or saved context

A product used for one narrow task is easy to replace. One used across four workflows has switching costs.

Acceptance and Task Success

This is where AI products need metrics that ordinary software doesn't. An AI output isn't a click that either happens or doesn't. The user can accept it, edit it or discard it, and each choice is a usage signal.

Acceptance rate is the clearest example. In GitHub's study Productivity Assessment of Neural Code Completion (Ziegler et al., MAPS 2022), the authors defined acceptance rate as the fraction of shown suggestions that were accepted for inclusion in the source file. They matched survey answers from 2,047 Copilot technical-preview users to usage data from the IDE. Acceptance rate was the usage measure that best predicted developers' perceived productivity, with a correlation of 0.24, better than measures of how long accepted code persisted unchanged. The authors are careful to say that considerable variance went unexplained and that acceptance rate suits coarse monitoring rather than fine-grained diagnosis.

Read that for what it is: a correlation with self-reported productivity in one product, among preview users. It supports acceptance rate as a first-line signal, not as proof of value. A user can accept a mediocre suggestion because editing it is more effort than rejecting it.

Build a small family of task-level metrics instead of one:

Pair these with a regular quality evaluation so a change in acceptance doesn't get misread. See AI trust and evaluation for how to build that.

Retention Cohorts

Retention is the metric that separates a product from a demo. The HEART paper defines it as how many of the users from a given period are still present in a later one, for example the percentage of seven-day active users in a given week who are still seven-day active three months later. It also notes the right interval varies, with week-to-week suited to some products and monthly or 90-day to others.

The standard way to read it is a cohort table. Group users by the week they started, then track what share of each group is still active in each later week. Rows are cohorts, columns are weeks since signup.

What to look for:

• A curve that flattens. If each cohort drops and then levels off at some percentage, you have a core of people for whom the product works. If the curve keeps falling toward zero, you don't have retention yet.
• Newer cohorts beating older ones. This is the cleanest evidence that product and onboarding changes are working. Blended retention hides it, because old users dilute the average.

Retention at the user level feeds the account-level numbers covered in net revenue retention. Adoption metrics are leading indicators for those lagging ones.

Seat-Level vs Account-Level Adoption

B2B AI products are sold to accounts and used by individuals, so you need both views.

Account-level adoption asks whether the customer is getting value as an organization: how many teams use it, whether it's embedded in a process, whether an executive sponsor can point to a result.

Seat-level adoption asks how many of the people with licenses actually use it, and how deeply. A common failure is an account that bought 200 seats after a successful pilot, where 30 people use it weekly. The renewal conversation starts with unused licenses, and the buyer's finance team will see them before you do.

Track the active share of paid seats, and the distribution of usage across them. An average hides the pattern. Often a handful of power users generate most of the usage and the rest barely log in.

Feed these numbers into customer health scoring so customer success sees low adoption before the renewal date does.

The Gap Between Usage and Value

High usage doesn't guarantee value, and low usage doesn't always mean failure. Keep these cases apart.

Usage without value. Users open the product because a manager told them to, or because the habit formed before the novelty wore off. Output goes unused. Signals: high retry rate, heavy editing, low task completion, and survey scores that lag behind activity.

Value without usage. The product handles a rare but costly task, so usage is low and the result is large.

Value that doesn't show up on the product dashboard. The benefit lands elsewhere, in cycle time, error rates, or a team's capacity. Brynjolfsson, Li and Raymond's NBER study Generative AI at Work measured an AI assistant among 5,179 customer support agents. It found productivity, measured as issues resolved per hour, rose 14 percent on average, and 34 percent for novice and low-skilled workers, with minimal impact on experienced and highly skilled workers. The point for founders isn't the headline number. It's that value was measured against a business outcome and split by user type, and that the gains were uneven. A flat adoption average would have hidden that.

This is why adoption metrics have to connect to an outcome the customer already measures. If you can't show the link from usage to a result, you can't defend the renewal. AI product ROI and the business case covers building that link, and AI pilots that convert to contracts shows how to agree the baseline before the pilot starts.

Building a Metrics Set That Won't Lie

You don't need thirty metrics. A practical set for an early AI product covers each HEART category once, and fits on one page.

Follow the Goals, Signals, Metrics order. Write the goal, such as "a support lead resolves tickets faster," name the signal, such as "tickets closed with an accepted AI draft," and only then define the metric.

A few habits keep the numbers honest:

• Exclude internal and test accounts before reading anything.
• Set definitions once and write them down. Moving the activation definition mid-quarter makes trends meaningless.
• Segment by customer type. Averages across very different customers describe none of them.
• Place metrics in your hierarchy. Adoption metrics are inputs to outcomes like retention and revenue. Keep clear which ones are drivers and which are results.

Key Facts

• HEART framework: Happiness, Engagement, Adoption, Retention and Task Success, introduced by Rodden, Hutchinson and Fu at CHI 2010 to complement PULSE metrics such as page views and seven-day active users.
• Method: the same paper pairs HEART with a Goals, Signals, Metrics process, so each metric traces back to a stated goal.
• Define adoption by a completed task, not a visit; the paper notes that some products should count a user as adopted only after a key task is completed.
• Acceptance rate: in GitHub's Copilot study, the share of shown suggestions accepted was the usage measure that best predicted perceived productivity (correlation 0.24, 2,047 matched survey responses), with considerable variance unexplained.
• Outcome-linked measurement: a 14 percent average productivity gain among 5,179 support agents hid a 34 percent gain for novice workers and minimal impact for experienced ones.
• Cohorts over averages: read retention by signup cohort, and look for a curve that flattens rather than one that keeps falling.
• Two views: track active share of paid seats as well as account-level embedding.