Plot.plot({
...chartBase(topics.length * (narrow ? 40 : 24) + 60),
marginLeft: narrow ? 8 : 340,
marginRight: 36,
x: {label: "Candidate entries mapped to this topic", grid: true, domain: [0, 46]},
y: {domain: topics.map(d => d.topic), label: null, axis: narrow ? null : "left", paddingInner: narrow ? 0.6 : 0.2},
marks: [
Plot.ruleY(topics, {y: "topic", x1: 0, x2: "variants", stroke: C.neutral, strokeWidth: 1.5}),
Plot.dot(topics, {y: "topic", x: "variants", fill: C.accent, r: 4.5, tip: {channels: {Category: "category"}}}),
Plot.text(topics, {y: "topic", x: "variants", text: "variants", dx: 10, textAnchor: "start", fill: C.ink, fontWeight: 600}),
narrow ? labelAbove(topics, "topic", "topic", topics.length * 40 + 60, 0.6) : null
]
})From 113,518 Reviews to 23 Topics
How to discover useful topics in customer reviews and turn them into a consistent list for analysis.
Before you can count what customers talk about, you need a consistent list of topics. This case study shows how we built that list from the reviews themselves.
Why the topic list comes first
We started with 113,518 cleaned customer reviews of electricians, plumbers, roofers, and other local tradespeople in Lower Saxony, Germany. We wanted to understand what customers discussed across these businesses. Before we could count mentions of pricing, punctuality, or workmanship, we needed a consistent list of topics.
Choosing that list ourselves would be straightforward, but it could miss concerns we had not anticipated. Asking a language model to invent labels independently for every review creates a different problem: similar concerns receive different names. “Punctuality,” “Keeping appointments,” and “Arriving on time” might become three labels even when they describe the same customer experience. Counting them separately would fragment the result.
An analyst could instead define topics and write keyword rules for each one. That can work well for known questions, but discovering unexpected concerns still requires looking beyond the topics already on the list.
We therefore separated discovery from labeling. First, we used a sample of 30,000 reviews to identify candidate topics. Then we consolidated those candidates into 23 topics, organized into ten groups. The aim was a manageable list that an analyst could review before using it to label individual reviews across the corpus.
The workflow at a glance
A taxonomy is simply the agreed list of topics and how they are grouped. In this workflow, the topic names emerge from the review text; we do not supply a predefined list. We do give the model context about the businesses and a target range for the number of topics, so the result is guided by both the data and our instructions.
01 Prepare the reviews
113,518 cleaned reviews
sample 30,000 reviews
02 Discover candidate topics
414 candidate entries
30,000 reviews in 30 batches
gpt-5.4-nano
retain names and supporting phrases
03 Consolidate overlapping topics
23 topics
proposed in one global pass
deepseek-v4-pro
group the topics for analysis
04 Organize and review
10 groups
Check definitions and assignments before labeling
Discovery ran in 30 batches of 1,000 reviews. Each batch was processed independently, producing a total of 414 candidate topic entries. These were not 414 distinct customer concerns: many entries described similar things in different words. Consolidation compared those suggestions together and produced the 23-topic list.
The two stages used different models. A small model, gpt-5.4-nano, handled batch discovery; deepseek-v4-pro handled consolidation and grouping. The reasoning behind this division was to keep repeated discovery calls inexpensive while using a more capable model for decisions about which topics belong together. These were the models used in this run, not a benchmark of the best models for the task.
Scroll on to follow the reviews through the pipeline.
01
113,518 reviews
Cleaned reviews of local tradespeople. Discovery does not need all of them: a sample of 30,000 is enough to surface the recurring concerns.
One dot = 100 reviews. Blue: the discovery sample.
02
30 batches of 1,000
The sample is split into batches. Each batch is read independently by a small, inexpensive model (gpt-5.4-nano), which proposes topics with supporting phrases.
Each tile is one batch of 1,000 reviews.
03
0 candidate entries
Batches suggest topics on their own, so the same concern comes back many times in slightly different words. These are candidates, not 414 distinct concerns.
Each mark is one candidate topic entry from one batch.
04
Merge what means the same
A more capable model (deepseek-v4-pro) compares all candidates at once and folds overlapping ones together. Pricing alone absorbed 43 entries. Five entries could not be mapped and are flagged for review.
Each cluster is one final topic; the number is how many candidate entries it absorbed. 5 entries remain unmapped (outlined).
05
23 topics in 10 groups
The result is a list an analyst can read, question, and edit before any review is labeled. Only then does counting start.
Discover topics and keep the evidence
For each batch, the model proposes topics and includes short phrases from the reviews to support them. We retain both: the name describes the concern, while the phrases let us inspect what the model meant. A label such as “reliability” is easier to interpret when we can see whether its examples concern appointments, emergency help, or keeping promises more generally.
The excerpts below illustrate the kind of evidence carried through the workflow. The English translations and concern labels are included here to help explain them.
| Review excerpt | English meaning | Concern illustrated |
|---|---|---|
| “Termine wurden eingehalten” | “Appointments were kept” | Keeping agreed appointments |
| “pünktlich vor Ort” | “On site on time” | Arriving punctually |
| “Baustelle sauber verlassen” | “Left the worksite clean” | Cleanliness after the job |
| “Keine Antwort auf Rückrufbitten” | “No response to requests for a callback” | Communication and responsiveness |
These are short excerpts from the discovery artifacts, selected for publication without names, businesses, or places. The translations are explanatory; the original phrases remain the evidence.
Because batches are processed independently, the same concern can appear repeatedly under different names. One batch might suggest “Punctuality and meeting deadlines,” another “Punctual appointments and schedule adherence.” At this stage, we keep those overlapping suggestions so that consolidation can compare their meanings and supporting phrases.
This repetition gives recurring concerns several opportunities to surface. It does not guarantee that we find every important issue, but it gives us more evidence to inspect than a single topic name would provide.
Several candidate entries can feed one final topic. Mapped candidate entries per topic, not the number of reviews mentioning it. Hover a dot to see its group.
Source: topics_consolidated.csv. 409 of 414 candidate entries are mapped; five need review.
For example, the saved consolidation groups 43 candidate entries under “Pricing transparency and value for money.” That number describes repeated topic discovery. It does not mean that only 43 reviews mentioned price, or that pricing was the most common concern in the corpus.
Turn overlapping topics into a consistent vocabulary
Consolidation turns the batch outputs into a list that can be used consistently. To see what this involves, consider an illustrative grouping of suggestions about scheduling:
| Candidate suggestions | Proposed final topic | Reason for grouping |
|---|---|---|
| Punctuality and meeting deadlines; punctual appointments and schedule adherence; reliable scheduling | Appointment scheduling, punctuality, and timely completion | They concern arranging appointments, keeping agreed times, and completing work on schedule. |
| Fast emergency help; rapid response to a breakdown | Emergency and rapid response | They concern availability and speed when something urgent goes wrong. |
Both topics concern time, but they answer different questions. “Did the electrician arrive at the agreed time?” is different from “Could someone come immediately when the heating failed?” Keeping that distinction makes the result more useful than a broad label such as “speed.” The table illustrates the decision; the full saved names appear in the taxonomy below.
In our run, the consolidation model received all 414 candidate entries and their supporting phrases in one call. This lets it compare possible groupings across the whole set. With a long chain of pairwise merges, an early grouping can become progressively broader as later topics are added. Considering the set together avoids that particular sequence of decisions, although it still leaves room for mistakes.
We use three rules to guide the grouping:
- Compare meanings and evidence together. Similar names are a starting point, but the review phrases help establish whether the suggestions concern the same experience.
- Give each candidate entry one explicit assignment. This makes omissions and duplicate assignments checkable. The final definitions should also explain where neighboring topics differ; an instruction to avoid overlap is not enough on its own.
- Keep the subject separate from the opinion. “Fair pricing” and “overpriced” express different opinions about the same subject. Both belong under pricing; their sentiment can be recorded separately during labeling.
The one-assignment rule applies to the candidate entries being consolidated. It does not mean that an individual review can discuss only one topic.
Review the taxonomy before using it
The saved output contains the following 23 topics in ten groups. The group names provide a broader view for reporting, while the individual topics are the labels used for more detailed analysis.
The complete topic list: 23 topics in ten groups Each card lists one group. Bars count mapped candidate entries, not review frequencies.
Source: topic_hierarchy.csv and topics_consolidated.csv.
The definitions matter as much as the names. In this output, professional consultation and advice covers expert recommendations and planning, while communication and responsiveness covers reaching the business, receiving replies, and staying informed. After-sales service and follow-up focuses on support after the work, including complaints, warranty questions, and rework.
Similarly, workmanship quality and precision concerns how well a job was executed, while workmanship cleanliness and site tidiness concerns mess, protection, and cleanup. A customer can praise one and criticize the other. These distinctions should be written into the labeling instructions so that the same wording is treated consistently across reviews.
Before adopting the list, an analyst should check whether the supporting phrases fit each definition, whether neighboring topics are distinguishable, and whether a broad topic needs to be split. Trying the definitions on a small set of reviews is a useful way to expose ambiguity before labeling the full corpus.
For example, the invented review “The electrician arrived on time but left the room dirty” would receive both a punctuality label and a cleanliness label. The first mention is positive and the second negative. The review can therefore contribute to two topic counts, with a different sentiment attached to each.
This is also the point to check the saved assignments for missing candidate entries, agree any changes, and save a version of the topic list. Keeping that version fixed during a labeling run makes the resulting counts easier to interpret and compare.
What this approach can and cannot tell you
The workflow produces a topic list supported by review examples. It does not establish that the list is exhaustive, that every grouping is correct, or how frequently each concern occurs. Three boundaries matter when using the result:
- The aim is to miss as few relevant concerns as possible. We processed a sample of 30,000 reviews in independent batches and retained supporting phrases so the proposed topics could be checked. This approach aims to capture a broad range of concerns while keeping discovery manageable. Uncommon concerns may still need targeted sampling, and consolidation needs review to ensure they remain visible.
- The saved output still needs an assignment check. This run contains 409 mapped candidate references out of 414. Five entries remain unaccounted for in the mapping and need review before it can be treated as complete. That does not establish that five distinct concerns are missing, but it shows why model instructions need to be followed by validation.
- Discovery counts are not review frequencies. A topic suggested repeatedly across batches is not automatically the most prevalent topic. Measuring prevalence requires applying the agreed list to individual reviews and defining how those labels will be counted.
The same sequence can be adapted to restaurant reviews, support tickets, or survey responses, with domain context and sampling suited to each source. In each case, the useful deliverable is a topic list that people can inspect, refine, and then apply consistently.
Once that review is complete, the next step is to label individual reviews and measure which topics appear. The companion article, “Reading 200,000 Reviews with LLMs: Without Overpaying”, examines how to carry out that labeling step economically.