How to prioritise negative reviews by severity

Why the star rating is not enough

Sorting reviews by rating puts every one-star review in one pile. That pile mixes customers who were mildly let down and chose the harshest button with customers who lost money or were put at risk. It also misses three-star reviews that describe a real defect politely.

A rating is how the customer felt. For triage you need what happened to them — which is in the text, not the number.

The loudest complaint is not always the most common one, and neither is always the worst.

A five-level severity scale

A useful scale describes concrete situations, so two people reading the same review land on the same level. This is the one the Review Analyzer uses, word for word:

The instruction that goes with it matters as much as the levels: judge the problem itself, not how strongly it is worded. Capital letters are not severity. See review severity.

Step 1: pull out what is urgent

Anything at level 4 or above goes to the top, regardless of how many other reviews say the same thing. A single report of a safety issue or of a customer charged twice is worth acting on before it becomes a pattern.

In the Review Analyzer, severity comes back as a score between 1 and 5 for each review, and reviews at 4 or above are counted as urgent on the summary. The score can fall between levels; treat it as a ranking, not a precise measurement — 3.4 and 3.5 are the same review for practical purposes.

Step 2: rank topics by negative volume

Below the urgent line, frequency takes over. Group reviews by topic — shipping, sizing, quality, customer service — and count how many negative or mixed reviews each topic has. Not how many reviews mention it: a topic that turns up in two hundred happy reviews is a strength, not a problem.

This is the order the Review Analyzer’s summary uses: topics sorted by their negative-or-mixed count, with the mean severity of the topic as the tie-breaker. The result is a fix list that starts with the complaint the most unhappy customers share.

Step 3: read the worst few, not all of them

Counts tell you where to look. Quotes tell you what to fix. For each of the top topics, read the handful of most severe reviews in it — they describe the failure in the customers’ own words, and are usually enough to see whether it is one defect or several.

The summary shows three verbatim excerpts per topic, most severe first, taken from the negative and mixed reviews. They are real text from your file, not a generated paraphrase, so each one can be traced back to its review.

Step 4: a separate queue for replies

Some reviews need an answer to this customer, whatever their severity: a question, a missing refund, a broken item still in their hands. That is a different question from “how bad is it”, so treat it separately — a level-2 review asking how to return an item still needs a reply.

The analyzer answers it as its own yes/no judgment and marks a review as needing a response when the probability is 0.5 or more. The threshold leans towards catching every customer who needs a reply, at the cost of flagging a few who do not. See actionable review.

Doing it again next month

Triage works best as a routine. Export the new reviews, run them, and compare the topic order with last time: a topic moving up the list is the early warning. If your reviews carry dates, the analyzer also plots a monthly trend. Where to get the file for each store is in how to export product reviews.