All posts

The AI summary of your reviews may be wrong

Above your reviews there is now often a short paragraph written by AI, and plenty of customers read that instead. Two new studies show these summaries can raise complaints nobody made and favour fake praise. What a small business can do about it.

An older shopkeeper sits behind a glass counter in a small shop full of jewellery and souvenirs, while a customer in the foreground holds a phone.
Photo by Michael Krahn on Unsplash

Look up almost any business on your phone now, on a map, in a search or on a big shopping site, and before the reviews there is a short paragraph. “Customers praise the friendly staff and quick service. Some mention parking can be difficult.”

Nobody wrote it. AI read the reviews and wrote it, and for a lot of customers in a hurry, that paragraph is the review.

That would be fine if the paragraph were a fair account of what people said. Two pieces of research published this year suggest it often isn’t.

The complaint nobody made

The first is a paper published by one of the main academic conferences on language and AI. University researchers built 50,000 customer reviews across 50 kinds of business, where they knew exactly what every review said, and had 11 different AI models summarise them. Then they checked their findings against real reviews from four kinds of business.

The summaries didn’t simply shrink the reviews. They tilted them. In the researchers’ words, “the same topics are amplified across different models, regardless of their prominence in the actual reviews.” Some topics got raised even when no review mentioned them at all. And none of the fixes the researchers tried, from rewording the instructions to using bigger models, brought the problem down to an acceptable level.

Put that in a small business’s terms. Your forty reviews talk about your bread, your staff and your opening hours. The summary may lead with something else, something the AI tends to think matters, because that’s what these summaries tend to say about everyone.

The fake praise that floats to the top

The second is a working paper from researchers at two US business schools, using data from one of the world’s biggest online marketplaces.

Its finding is uncomfortable. Summaries pick out what keeps coming up, and fake reviews keep saying the same generic things: great quality, fast delivery, would buy again. Real customers write about their own experience, in their own words, and they’re all different. So the paper found the fakes were over-represented in the summaries. Products more exposed to fake reviews got more positive summaries, even compared with products that had the same average star rating. And after the summaries were introduced, those products climbed the sales rankings relative to the rest.

That’s a fair warning for honest businesses. If a competitor has padded its reviews with bland praise, a summary may make it look better than you, and you may not be able to see why.

Why small businesses feel it most

A big chain with ten thousand reviews has a summary that’s roughly right because there’s so much to average. A plumber with twenty-three reviews doesn’t. In a small pile, one long, detailed complaint about a mix-up two years ago carries a lot of weight, and the summary can still be repeating it long after you fixed the problem.

You can’t edit the summary. You can’t write it. On most sites you can’t even see how it was made. What you can control is what it’s made from, and what customers find when they look past it.

The fair question

devkoi is an AI that builds and changes your website when you ask in a chat. So it’s fair to ask whether it has a say over these summaries. It doesn’t.

devkoi can’t change what map apps, search results or shopping sites write about you. It can’t remove reviews, it doesn’t collect them and it doesn’t write them, and we’d tell you not to let anyone write them for you. Review rules are getting stricter, and this isn’t legal advice.

What devkoi can do is give your own website the specifics a summary leaves out, in your words and your customers’ real words, so anyone who taps through finds the full picture.

What to do this week

1. Read your summaries like a customer. Look yourself up on your phone in the two or three places customers find you. Copy each summary into a note, one claim per line.

2. Check every claim against the reviews. For each line, find the reviews that actually say it. If none do, mark it. If something is only in one old review, note the date. Now you know which claims are fair and which aren’t.

3. Fix what’s fair, and flag what’s false. If the summary repeats a real, recurring complaint, that’s useful. Fix the cause. If it states something that isn’t true, and the site offers a way to report it, point to the factual error rather than the tone. “Too negative” rarely works. “There is no review that mentions this” might.

4. Ask for real, specific reviews. A summary is built from what people write, so detail helps. Ask every happy customer, not just the ones you expect to be glowing, and ask what they came in for rather than asking them to be positive. Never offer anything in return for a good review, and never write one yourself.

5. Answer the summary on your own website. If summaries keep saying parking is hard, don’t argue. Tell people where to park.

“Add a ‘Getting here’ section to the Contact page: ‘Free parking is behind the building, on [street name]. It’s two minutes’ walk from the front door.’”

6. Show real customers’ words, with their permission. Pick three or four real reviews that say something specific, ask the writers if you can use them, and put them on your site with the month they were written. No invented quotes and no made-up names.

“Add a ‘What customers say’ section to the homepage with these three reviews, each with the first name and month: [paste the reviews].”

7. Preview it, then publish. Every change in devkoi comes with a preview link first, and every version of your site is kept. Read it on your phone the way a customer would, then publish.


There will be more AI summaries, not fewer. Most of the time they’ll be roughly fair, and sometimes they’ll get you wrong in a sentence you never get to see written. You can’t control that paragraph. What you can do is give customers plenty of real, specific detail about you, and make sure your own website, the page you write yourself, is easy to find past it.

Connect devkoi to your assistant

devkoi runs as an MCP server. Add it once and build your site from the assistant you already have open.

Server URL

https://api.devkoi.com/mcp

Any MCP client that can talk to a remote server over HTTP will take this. The cards below are the shortcuts for the common ones.

Claude Code

One command
claude mcp add --transport http devkoi https://api.devkoi.com/mcp

Gemini CLI

One command
gemini mcp add --transport http devkoi https://api.devkoi.com/mcp

VS Code

One command
code --add-mcp '{"name":"devkoi","type":"http","url":"https://api.devkoi.com/mcp"}'

Cursor

Config file

~/.cursor/mcp.json

{
  "mcpServers": {
    "devkoi": {
      "url": "https://api.devkoi.com/mcp"
    }
  }
}

Claude desktop and web

Settings
  1. Settings
  2. Connectors
  3. Add custom connector
  4. Paste the URL above

ChatGPT

Settings
  1. Settings
  2. Connectors
  3. Add custom connector
  4. Paste the URL above

Menus move. If the steps do not match what is in front of you, the server URL is the only part that matters — add it wherever that client keeps its MCP servers.