Plot.plot({
...chartBase(narrow ? 200 : 160),
marginLeft: narrow ? 8 : 170,
marginRight: 110,
x: {label: "Cost per 1M reviews (USD)", domain: [0, 130], grid: true},
y: {domain: costSplit.map(d => d.part), label: null, axis: narrow ? null : "left", paddingInner: narrow ? 0.55 : 0.3},
marks: [
Plot.barX(costSplitShares, {y: "part", x: "cost", fill: d => d.part.startsWith("Output") ? C.accent : C.neutral, tip: {format: {x: usd}}}),
Plot.text(costSplitShares, {y: "part", x: "cost", text: d => usd(d.cost) + " · " + d.share + "%", dx: 6, textAnchor: "start", fill: C.ink, fontWeight: 600}),
narrow ? labelAbove(costSplitShares, "part", "part", 200, 0.55) : null
]
})Reading 200,000 Reviews with LLMs: Without Overpaying
How output format, reasoning settings, caching, and batch size changed the cost of labeling customer reviews.
Small changes to a review-labeling workflow can make a large difference to its API cost. The useful test is what those changes save, and what they change in the labels.
From customer reviews to usable labels
We wanted to turn roughly 200,000 German reviews of local tradespeople into a dataset we could analyze. For each relevant passage, a language model assigned a topic, recorded its sentiment, and copied the phrase that supported the label.
At that scale, keeping the API bill low was part of the task. For a fixed model and token prices, cost grows with the input and output tokens used: every request sends instructions and reviews, and every extracted label adds output. We therefore asked which parts of that usage we could reduce: the input tokens per review, the output tokens per label, and the repeated instructions across requests. We also tested caching to make repeated input cheaper. The aim was to lower the cost while retaining labels useful for the analysis.
We then compared different configurations on the same 1,000-review sample. Shorter outputs, disabled reasoning, prompt caching, and larger batches brought the estimated API cost down to about $15 per million reviews. This article explains which changes made the difference and what we checked before deciding whether the cheaper configuration was useful.
The model receives instructions, a fixed list of 23 topics, and a batch of reviews. It returns a set of labels that can be stored and counted. The topic list comes from a separate discovery step, described in “From 113,518 Reviews to 23 Topics”.
01 Select and batch reviews
Choose the records needed for the analysis and split them into batches.
reviews to label
02 Build the prompt
Keep instructions and the 23-topic list first; put the review batch last.
send to the model
03 Extract labels
Return each review ID, copied phrase, topic code, and sentiment.
structured output
04 Store the results
Look up full topic names and keep the labels linked to their source reviews.
The easiest way to see what labeling produces is to follow a single review through it. Scroll through the steps below.
Start with one review. This one is written for illustration and shown in English; the reviews in the study were German. In a few sentences it covers timing, staff, advice, tidiness, and price.
Find what is said, not the overall mood. The model looks for passages that make a specific point and copies each one exactly as written.
Match each passage to a topic. Every passage gets one of the 23 topics from the fixed list. A single review can receive several labels, here six.
Add a sentiment to each label. Four labels are positive, one is neutral, and one is negative. The four-star rating alone would hide the complaint about the invoice.
Store the labels as data. Each row keeps the review ID, the copied phrase, the topic, and the sentiment. Because the phrase is copied word for word, we can check it against the review automatically. That check is the exact-phrase match rate used later in this article.
Only then count. Each label adds one mark to its topic. Across roughly 200,000 reviews these marks become the topic counts, and every count can still be traced back to the phrases behind it.
Keeping the original phrase with each label makes it possible to inspect what the model extracted, rather than relying on a topic count alone.
Establish a baseline on the same workload
The main experiment used deepseek-v4-flash, the fixed taxonomy, and the same 1,000-review slice. We varied the output format, reasoning setting, and batch size, recording token usage and checking the returned labels. Holding the review sample constant made the configurations easier to compare.
Costs below are estimates per million reviews, calculated from those small runs using the token prices applied to the experiment on May 30, 2026: $0.0028 per million cached input tokens, $0.14 per million uncached input tokens, and $0.28 per million output tokens. They describe provider API charges, not the cost of data collection, engineering, or human review.
The verbose batch-10 run cost approximately $104 per million reviews with its recorded cache hits. Pricing exactly the same token usage as if every input token missed the cache gives a $129 reference baseline. This second number is a calculation, not a separate observed run.
We tracked two kinds of result:
| Measure | What it tells us |
|---|---|
| Estimated API cost | How recorded input, output, and cache usage translate into a bill. |
| Exact-phrase match rate | The share of returned label phrases that appear verbatim in their source review. |
| Labeling rate and label count | How many reviews receive any label, and how many labels the model emits. |
| Distinct topics emitted | How many of the 23 allowed topics appear at least once in the output. |
The output checks describe copying fidelity and labeling behavior. They do not, by themselves, establish whether the topics or sentiments are correct. We return to that distinction when comparing the baseline with the optimized result.
Other exploratory runs used OpenAI models on different review slices. Those comparisons need matched inputs before they can support a model ranking; the savings described here come from changing the configuration of the same DeepSeek model.
Reduce the cost of the model’s answer
Language models charge for both the text they receive and the tokens they generate. In the uncached reference calculation, output accounted for about 80% of the bill. That made the answer an important place to look for savings.
Output accounted for about 80% of the uncached reference cost. Cost per 1M reviews for the verbose, batch-10 run priced without any cache hits: output is about 80% of the total.
Source: compact-JSON × batch-size experiment, 2026-05-30, deepseek-v4-flash, same 1,000-review slice (comparison_summary.csv). Input re-priced at the cache-miss price.
Test whether reasoning helps this task
In the verbose batch-10 run, about 69% of the output tokens were recorded as reasoning tokens. Those tokens contributed to the bill without appearing in the final labels. We therefore tested whether disabling reasoning would reduce cost while retaining useful output.
Disabling reasoning reduced output tokens in both prompt formats. Output tokens per review. With thinking on, 69% of the verbose prompt’s output was reasoning. Verbose pair at batch size 10, compact pair at 50.
Source: compact-JSON × batch-size experiment, 2026-05-30, deepseek-v4-flash, same 1,000-review slice (comparison_summary.csv).
The reasoning-disabled configurations used roughly 70% fewer output tokens in these pairs and had higher exact-phrase match rates. That supports testing this setting for a similar extraction task. It does not tell us that reasoning is unnecessary for every labeling problem, or that the cheaper output has identical topic and sentiment accuracy.
Return codes instead of repeating names
The other source of avoidable output was information already available in the topic list. A verbose label repeated the full topic name and its broader category for every extracted phrase.
The following illustrative outputs represent the same label. In the compact version, the application can recover the topic and category names by looking up T003.
Verbose output
{
"item_id": "I001",
"extracted_phrase": "freundliche Mitarbeiter",
"topic_id": "T003",
"high_level_category": "Customer Interaction & Professionalism",
"canonical_topic": "Staff friendliness and courtesy",
"sentiment": "positive"
}→
Compact output
{
"i": "I001",
"p": "freundliche Mitarbeiter",
"t": "T003",
"s": "positive"
}The key legend is defined once in the instructions: i is the review ID, p the copied phrase, t the topic code, and s the sentiment. The phrase remains unchanged; only the representation becomes smaller.
With reasoning enabled and ten reviews per request, switching from verbose labels to compact codes reduced estimated cost from $104.19 to $97.64 per million reviews. Both configurations emitted 22 of the 23 topics. The saving was smaller than the reduction from disabling reasoning, but it removed repeated text that the application could supply through a lookup.
Reuse the prompt and batch reviews efficiently
Every request includes instructions and the topic list, even though only the reviews change. Caching can reduce the price of that repeated input; batching spreads the repeated material across more reviews.
Keep the reusable prompt first
A provider-side prompt cache can reuse a previously processed prefix. To make that possible, keep the instructions and topic list stable and put changing content after them:
| Reusable beginning | Changing content at the end |
|---|---|
| Instructions → topic list | Batch metadata → reviews |
Use the same topic order and serialization on each call. A changing identifier or timestamp placed before the topic list can prevent reuse of that part of the prompt. Stable layout supports caching, but cache hits are best-effort behavior and should be checked in the provider’s returned usage fields.
Recorded cache hits lowered the reference run’s estimated cost. The same token usage priced with observed cache hits ($104.19) and with all input billed as a miss ($129.11), per million reviews.
Source: comparison_summary.csv, deepseek-v4-flash, 2026-05-30. The no-cache estimate is a price calculation, not a separate run.
At the prices used here, cached input tokens were 50 times cheaper than uncached input tokens. The whole job did not become 50 times cheaper: output still had to be generated, and the changing review text still incurred input cost. For this run, recorded cache hits reduced the estimated bill from the $129.11 no-cache reference to $104.19, about 19%.
The separate article “Reducing LLM Costs with Prompt Caching” tests prompt layout directly. Here, the chart compares two prices for the same recorded token usage.
Choose batch size by the tradeoff
We tested 10, 25, 50, and 100 reviews per request. Larger batches needed fewer calls, but the relevant question was whether the remaining saving justified changes in the output. These results use compact codes with reasoning disabled:
| Reviews per request | Cost per million reviews | Exact-phrase match rate |
|---|---|---|
| 10 | $17.80 | 98.93% |
| 25 | $15.81 | 99.08% |
| 50 | $14.57 | 99.19% |
| 100 | $13.94 | 98.89% |
Increasing the batch size from 50 to 100 saved $0.63 per million reviews, while exact-phrase matching fell from 99.19% to 98.89%. That made 25–50 a reasonable range for this workload. Longer reviews or different instructions could change the tradeoff.
Fewer requests do not necessarily mean a faster job. These figures compare cost and phrase matching; they do not establish a latency improvement.
Reduce the workload without changing the question
The previous changes lowered cost per review. A separate decision is how many reviews the analysis needs and which model calls can be reused.
Date filtering defines the population being analyzed. Of 197,827 eligible reviews, 100,611 were dated from 2023 through 2026. Restricting the analysis to that period removed about 49% of the workload. That is useful for a question about recent customer experience, but it would exclude information needed for a longer historical comparison.
Duplicate handling can avoid repeated model calls. When identical text is labeled under the same instructions and context, the result can be reused. Attach those labels back to every original record so that review counts and business-level analysis retain their meaning. Repeated phrases such as “Sehr gut” (“Very good”) still represent separate customer reviews, even if they do not need separate model calls.
Restricting the analysis to recent reviews roughly halved the workload. 197,827 eligible reviews, 100,611 from 2023–2026, and an estimated 95,640 after normalized-text deduplication. The final count is not a measured intersection.
Source: corpus counts in the labeling project notes (QUALITY_AND_COST_NOTES.md); not a recorded run.
The corpus notes report 4.58% exact-text duplicates, or 4.94% after lowercase and whitespace normalization. Applying the latter rate to the recent-review subset gives an estimated 95,640 texts to label. That combined figure was not measured on the filtered subset and should be recomputed before budgeting a production run. If normalization is used to find duplicates, any returned phrase must also be checked against each original review.
These workload reductions are separate from the per-review savings. Filter only when the filter matches the analytical question, and reuse labels where doing so preserves the original records and their counts.
What the savings mean for quality
The chart below brings the configuration changes together. Starting with the no-cache reference, it shows the recorded cache benefit, compact output, disabled reasoning, and a batch size of 50.
One route through the tested configurations: about $129 to $15 per million reviews. The first bar is a no-cache price estimate. The other four use recorded usage, including cache hits. All underlying runs in this sequence emitted 22 of the 23 allowed topics.
Source: compact-JSON × batch-size experiment, 2026-05-30, deepseek-v4-flash, same 1,000-review slice (comparison_summary.csv).
The optimized estimate of $14.57 per million reviews is about 89% below the $129.11 no-cache reference, or about 86% below the $104.19 observed baseline. The reduction is substantial, but it needs to be read alongside changes in the labels:
| Output check, on the same 1,000 reviews | Verbose baseline, batch 10 | Compact, reasoning off, batch 50 |
|---|---|---|
| Reviews receiving a label | 658 (65.8%) | 601 (60.1%) |
| Labels emitted | 1,532 | 1,234 |
| Distinct topics emitted | 22 of 23 | 22 of 23 |
| Exact-phrase match rate | 98.37% | 99.19% |
The optimized configuration copied phrases more consistently, but emitted fewer labels and labeled fewer reviews. Without a manually evaluated reference set, we cannot tell how much of that difference is useful restraint and how much is missed information. A high phrase-match rate also does not establish that a phrase received the correct topic or sentiment.
Before choosing a configuration for production, inspect reviews where the outputs disagree, especially those labeled by the baseline but left unlabeled by the cheaper setup. Evaluate topic assignment and sentiment separately, and check whether the differences change the conclusions you intend to draw.
The practical sequence is to choose a representative sample, record token usage, compare configurations, and inspect what the cheaper setup stops labeling. This experiment shows how much the API cost can move within one model. The final choice depends on whether the resulting labels answer the analytical question well enough.