Reviews and Rating Systems in Marketplaces

By Brian Tr · Updated Oct 7, 2026

A stranger on one side of a marketplace is about to hand money to a stranger on the other. The buyer can't inspect the apartment, the freelancer's work habits, or the used camera's real condition. The seller can't easily tell whether the guest will treat the place well. Each side knows less than it needs to.

Reviews and ratings are the marketplace's answer. They turn past transactions into a public signal that future participants can read. When the system works, it's one of the cheapest forms of trust a platform can build. When it's badly designed, everyone ends up with a column of five-star ratings that tells them nothing.

This article covers why review systems exist, the main design choices, the known ways they go wrong, what regulators now require, and the metrics that show whether yours is healthy. It's a reference, so we'll stay with what the research and the rules actually say.

Why marketplaces need reputation systems

The core problem is old. The Royal Swedish Academy's 2001 prize announcement says George Akerlof demonstrated how a market where sellers have more information than buyers about product quality can contract into an adverse selection of low-quality products. That's the "market for lemons" idea: if buyers can't tell good from bad, they pay an average price, good sellers leave, and the pool gets worse.

Online marketplaces make the problem sharper. Buyers and sellers often have never met and may never meet again. A review system attacks the information gap directly, by letting past customers do what the buyer can't: inspect the thing and report back.

That gives reviews three jobs inside a marketplace:

• Inform buyers. Help them choose between sellers they can't evaluate themselves.
• Reward good sellers. Visible reputation lets reliable suppliers earn more, which keeps them on the platform.
• Feed the platform. Ratings become an input to ranking, quality enforcement, and removal decisions. We cover the quality side in marketplace supply quality.

There's a bootstrapping cost too. A new seller has no reviews, and buyers prefer sellers who have some. That's a version of the cold start problem, and it's why seller onboarding often includes ways to earn a first review quickly.

Do reviews actually move money?

The best-known evidence comes from Michael Luca's Harvard Business School study of Yelp. As summarized in Harvard Magazine, the study found that a one-star increase in a restaurant's Yelp rating led to a 5 to 9 percent increase in revenue. The same coverage reports that the effect applied to independent restaurants, while ratings did not affect restaurants with a chain affiliation.

The second finding is the useful one for operators. Reviews matter most where buyers have little other information. A chain has a brand that already tells you what to expect. An independent has only its reputation. For a marketplace full of small, unbranded sellers, the review system is doing the work a brand would do elsewhere.

One caution: this is a study of one platform, one city, and restaurants. It shows the mechanism is real. It doesn't give you a number to expect in your category.

Design choice one: one-sided or two-sided reviews

The first design decision is who gets reviewed.

One-sided systems let only buyers review sellers. Most product marketplaces work this way. It's simple, and buyers are rarely a risk to the seller in a way that needs a public record.

Two-sided (mutual) systems let both parties review each other. They fit marketplaces where the buyer is also a risk: a guest staying in someone's home, a client hiring a freelancer, a rider in a stranger's car. The host or driver wants to know who they're letting in, and a rating on the buyer side supplies that.

Two-sided systems have a built-in flaw, covered next.

Reciprocity, retaliation, and the double-blind fix

If I review you after you've already reviewed me, I can see what you wrote. A kind review invites a kind one back. A harsh one invites retaliation. People who anticipate that give softer reviews to avoid the risk, which compresses the scale toward the top.

Researchers tested a structural fix on Airbnb. In Reciprocity and Unveiling in Two-Sided Reputation Systems, published in Marketing Science in 2021, Andrey Fradkin, Elena Grewal, and David Holtz ran a large experiment on the timing of when feedback becomes visible. In the treatment group, feedback was hidden until both parties had submitted a review. In the control group, it was revealed immediately after submission.

As summarized on the paper's abstract page, the results were:

• The treatment stimulated more reviewing in total, which the authors attribute to users' curiosity about what their counterparty wrote and the desire to have their own feedback visible.
• It reduced retaliation and reciprocation in feedback, and ratings came out lower as a result.
• The changes in feedback did not translate into reduced adverse selection on the platform.

The last point deserves attention. The simultaneous reveal improved the behavior of the review system, but the paper's authors found no meaningful reduction in the adverse selection problem. A better-behaved review system isn't automatically a better marketplace outcome.

Airbnb's own help page, as fetched on 2026-10-07, describes the live version of this design. Its help article on how reviews work says both guests and hosts have 14 days after checkout to submit reviews, and that reviews are posted only after both parties have submitted or once the 14-day period has ended, whichever comes first. The same page says a listing's rating is shown in search results once it has at least three reviews.

That last detail is a design choice worth copying in spirit: don't display an average built on one or two reviews, because a single outlier dominates it.

Reputation inflation: when five stars stop meaning anything

Even without retaliation, ratings tend to drift upward. Apostolos Filippas, John Horton, and Joseph Golden studied this in Reputation Inflation (Marketing Science, 2022; also an NBER working paper). Their abstract says average buyer ratings of sellers have grown substantially more positive over time in five online marketplaces. The authors built a method to separate two explanations: marketplaces genuinely improving, versus raters giving higher ratings without being more satisfied. Using one marketplace with extensive transaction-level data, they found that much of the observed increase in ratings is attributable to reputation inflation.

Why would raters inflate? The NBER summary points to social pressure: raters feel pressure to leave above-average ratings, and they hesitate to give low scores because they don't want to harm sellers. But the potential to harm is exactly what makes ratings informative. A system where nobody uses the bottom of the scale has lost most of its signal.

The practical consequence is that a five-star average becomes the price of entry, and the useful information moves into the small differences between 4.9 and 4.7. If your platform shows a rating to one decimal place and 90 percent of sellers sit above 4.5, buyers can't separate the sellers, and your ranking system has little to work with.

Ways platforms respond include:

• Ask more specific questions (cleanliness, accuracy, communication) instead of one overall score, so a mediocre experience has somewhere to land.
• Keep detailed ratings private. Airbnb's help page says individual subcategory star ratings and notes to the host are available only to the host and Airbnb, which lets guests be candid without publicly sinking a host.
• Show rank or percentile, not only averages, so a 4.8 can be compared to the rest of the supply.
• Weight recent reviews more, so old inflation doesn't mask current quality.

None of these is proven to work everywhere. They're options to test.

Who leaves reviews: selection bias

Review samples are not random samples of customers. A 2009 Communications of the ACM article by Nan Hu, Jie Zhang, and Paul Pavlou, Overcoming the J-Shaped Distribution of Product Reviews, looked at Amazon data collected between February and July 2005. It found that 78, 73, and 72 percent of ratings for books, DVDs, and videos respectively were four stars or higher. A chart of the distribution had a J shape: mostly five-star ratings, some one-star ratings, and few in between.

The authors argue the average is a poor proxy for quality in that situation, since an average is statistically meaningful mainly when the distribution is unimodal or symmetric. They attribute the shape to self-selection: people who buy are already inclined to like the product (purchasing bias), and the people with moderate opinions are less likely to bother writing (under-reporting bias).

This is 2005 data from one retailer, so don't treat the percentages as current. The pattern, though, is a standing warning. The reviews you see come from the happiest and the angriest customers. The satisfied middle is quiet.

Some implications for operators:

1. Show the distribution, not just the mean. A 4.4 built from mostly fives and a few ones is different from a 4.4 built from steady fours.
2. Prompt every buyer. A review request sent to all completed transactions, not only the ones that went well, reduces the skew.
3. Measure the response rate (see metrics below). A rating from 3 percent of buyers is a weaker signal than one from 40 percent.

Fake reviews and the rules around them

Reviews carry money, so people fake them, which makes review fraud a core trust and safety problem. Two regulatory regimes are worth knowing, even for a platform that's neither in the US nor the EU, because they show where the line is drawn.

United States: the FTC rule

The FTC's final rule is codified at 16 CFR Part 465, "Trade Regulation Rule on the Use of Consumer Reviews and Testimonials." Per the Federal Register notice, the rule is effective October 21, 2024. It prohibits:

• selling or purchasing fake consumer reviews or testimonials
• buying positive or negative consumer reviews
• certain insiders writing reviews without clearly disclosing their relationship
• a company-controlled review website that falsely presents itself as independent
• certain review suppression practices
• selling or purchasing fake indicators of social media influence

The FTC's announcement adds that the rule gives the agency civil penalty authority against knowing violators. The Federal Register text notes that the maximum civil penalty was $51,744 per violation at the time, with courts weighing statutory factors and able to impose much lower amounts. That figure is adjusted over time, so check the current one before quoting it.

Two points matter for marketplace operators. First, the Federal Register preamble says the rule's fake-review provisions in section 465.2(b) and (c) do not apply to mere consumer review hosting, even if the business prompts review submissions or aggregates star ratings. A platform that simply displays third-party reviews is not automatically liable for them. Second, the same preamble says moderation that suppresses reviews based on their ratings or negative sentiment could violate the review suppression provision in section 465.7(b). Deleting your sellers' bad reviews isn't a neutral moderation policy.

European Union: Directive 2019/2161

Directive (EU) 2019/2161, the Omnibus Directive, amended the Unfair Commercial Practices Directive on reviews. The text adds to the blacklist of banned practices:

• submitting, or commissioning another person to submit, false consumer reviews or endorsements, or misrepresenting consumer reviews or social endorsements, in order to promote products
• stating that reviews are submitted by consumers who actually used or purchased the product without taking reasonable and proportionate steps to check that they originate from such consumers

It also provides that when a trader gives access to consumer reviews, information about whether and how the trader ensures the published reviews come from consumers who have actually used or purchased the product is treated as material.

The design takeaway is common to both regimes. Verified-purchase gating is the practical answer. Airbnb's review policy says reviews must be submitted by or on behalf of someone who participated in the reservation. Tying a review to a completed transaction costs little and removes a large class of fakes. If you can't verify, say what you do and don't check.

How ratings feed ranking

Ratings don't only inform buyers directly. They usually flow into search ranking as a quality signal, which we discuss in search and matching in marketplaces. That creates a loop with consequences:

• Rich get richer. A seller with early good reviews ranks higher, gets more orders, and collects more reviews. This can lock out newcomers.
• Inflation damages ranking. If most sellers cluster at 4.8 or higher, rating differences are noise and the ranker is effectively sorting on random variation.
• Gaming pays off. The more ranking depends on ratings, the more a seller gains from faking or pressuring for reviews, which is why network effects can cut against you when trust erodes.

Good practice is to use ratings as one signal among several, to apply a confidence adjustment for sellers with few reviews, and to give new sellers a path to visibility that doesn't depend on history. None of this has a single right formula, so test against your own transaction data.

Metrics for a review system

Review systems are easy to launch and hard to evaluate. These are common measures. None has a universal benchmark, so set targets from your own baseline.

The last row is the one that tells you the most. If ratings don't predict what buyers do next, they aren't doing their job. Place these metrics along the buyer's path, and note that a review system matters most in categories where managed marketplaces would otherwise vet each supplier by hand.

Where to start

A sensible order, based on reasoning from the sources above and not a rule from them:

1. Tie every review to a completed transaction. It's the cheapest defense against fakes. It also keeps the deal on the platform, where buyer guarantees and dispute handling apply, which gives users one more reason not to take the deal elsewhere.
2. Choose one-sided or two-sided deliberately, based on which party carries risk.
3. If two-sided, use a simultaneous reveal with a fixed review window, as Airbnb describes.
4. Ask specific questions and consider keeping some feedback private.
5. Hide averages until a minimum review count.
6. Show distributions, not just means.
7. Write a plain moderation policy that doesn't remove reviews for being negative.
8. Track review rate and rating spread monthly.

Key Facts: Reviews and Rating Systems in Marketplaces

• The Royal Swedish Academy's 2001 announcement says Akerlof showed how a market where sellers know more about quality than buyers can contract into an adverse selection of low-quality products.
• Luca's Yelp study, per Harvard Magazine, found a one-star rating increase raised restaurant revenue 5 to 9 percent, with no effect for chain restaurants.
• A large Airbnb experiment found that hiding feedback until both sides submit stimulated more reviewing and reduced retaliation and reciprocation, but did not reduce adverse selection.
• Reputation Inflation reports average ratings rising across five online marketplaces, and in the one studied in depth, much of the rise was inflation, not improved satisfaction.
• Hu, Zhang, and Pavlou's 2009 article found 72 to 78 percent of Amazon ratings in three categories were four stars or higher, a J-shaped distribution they link to purchasing and under-reporting bias.
• The FTC rule on reviews and testimonials (16 CFR Part 465) took effect October 21, 2024.
• EU Directive 2019/2161 bans false reviews and requires traders to disclose whether and how they check that reviews come from real users.

Related Reading

• Search and Matching in Marketplaces
• Marketplace Supply Quality
• Marketplace Seller Onboarding
• Managed Marketplace
• Trust and Safety in Marketplaces
• Guarantees and Buyer Protection in Marketplaces
• Disintermediation: Why Users Go Around the Platform
• Network Effects