🔍 Read the full analysis: My News Pipeline’s AI Results: The Best Configuration Isn’t In Use on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thorsten Meyer says a test of 71 AI configurations for news writing found GPT-6 Astra had the highest scores, while GPT-6 Luna offered lower-cost production performance. The test also found two fallback models produced fabricated quotes or unusable output; both were removed from the pipeline on October 7, 2026.
Thorsten Meyer says GPT-6 Astra earned the highest marks in his test of 71 AI configurations for producing news articles, but his pipeline uses the much cheaper GPT-6 Luna. In results published October 13, Meyer also reported that two backup models generated fabricated quotations or failed to produce usable articles, prompting their removal from the news route on October 7.
Meyer tested configurations between September 8 and October 13, 2026, using his production prompt and real news sources to create English- and German-language articles. The evaluation covered API and subscription models, free-tier services, and open-weight models run on his own hardware. Two AI judges assessed source fidelity, fabrication, clarity about uncertainty, news value, structure and polish, with German language quality also scored.
GPT-6 Astra scored 4.00 out of 5 in both languages on the initial test and received a 5.00 source-fidelity score there. In a further 12-article robustness run, it averaged 3.93, with a reported 95% bootstrap interval of 3.83 to 4.02. Meyer put its cost at about $0.12 per article and its production time at roughly 50 seconds.
GPT-6 Luna scored 3.67 in English and 3.68 in German in the initial results. In the robustness run it averaged 3.67, with source fidelity of 4.71 and valid output for every article. Meyer reported a cost of about $0.0012 to $0.0013 per article and a typical production time of 17 seconds. He said Astra’s stronger score did not justify its roughly 100-times-higher cost for his news workload.
Infographic 01 · Who writes the best news article?
The best writer is not the one in production
71 model configurations, one real job: turn a news source into an article for an English and a German edition. GPT-6 Astra writes the best articles. GPT-6 Luna writes them for about a hundredth of the price.
Robustness run: 12 real news sources, English edition
Dot = mean score (1–5), bar = 95% bootstrap interval. German edition mean below each score. Overlapping bars are not a real difference.
Two judges, two caveats: GPT-6.1 Sol and Claude Opus 5.5 grade every article and were also tested as writers, so they partly grade themselves. Claude models ran with effort medium unless noted.
Source: Thorsten Meyer AI model-scout benchmark, 8 Sep – 13 Oct 2026. Task: write a news article from a real source with the production prompt, English and German editions. Judges: GPT-6.1 Sol (reasoning high) and Claude Opus 5.5, both with the source in context; score 1–5, mean of both judges. Robustness run: 2 test articles plus 10 more real sources, mostly short like production; mean with 95% bootstrap interval.
Powered by Thorsten Meyer AI
Why Luna Stays in Production
The results illustrate why a model’s general reputation or benchmark rank may not predict how well it performs a specific publishing task. For a high-volume news pipeline, the choice involves quality, cost, speed and reliability, not just the highest score. Meyer’s test suggests Luna meets his working quality bar at a fraction of Astra’s reported price, while Astra remains the strongest writer in the measured sample.
The findings also point to a risk beyond the primary model: failover systems can introduce serious errors if backup models are not tested under the same conditions as production. Meyer said one fallback invented quotations and another repeatedly returned unusable output. In news publishing, such failures can undermine trust even if the main model performs well.
As an affiliate, we earn on qualifying purchases.
How the Pipeline Was Tested
The evaluation used two source texts: a 6,340-character product filing and a much shorter, 619-character political news item. Meyer said short sources better reflect much of his actual workload, so the test included them as a particular challenge. Each source was turned into English and German articles using the same prompt as his production pipeline.
The judges were GPT-6.1 Sol and Claude Opus 5.5, and both could consult the original source. Meyer also ran a robustness round in which leading configurations produced 10 additional articles, mostly from short sources. He cautioned that the initial comparison had only two test articles and treated differences below about 0.3 points as possible noise; he considered the larger robustness run a more reliable ranking. Results therefore describe his test setup and task, not every kind of writing or every production environment.
“For a high-volume task, the right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”
— Thorsten Meyer, writing on ThorstenMeyerAI.com
AI content generation tools for news
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Limits of the Model Comparison
The published results come from Meyer’s own pipeline, prompt, two source texts in the standard test and AI-based judging. The additional robustness run broadened the sample for leading configurations, but it does not establish how the models would perform across other publishers, topics, prompts or languages. The judges’ scores are evaluations, not independent verification of every article’s factual accuracy.
Meyer described the cost figures as approximate and specific to his setup. The report does not provide enough detail to generalize those prices to other users or account for changes in provider pricing. It also remains unclear whether the retired fallbacks will be replaced or whether the pipeline will continue with Luna alone if its primary service is unavailable.
As an affiliate, we earn on qualifying purchases.
What Meyer’s Pipeline Uses Now
Meyer said GPT-6 Luna remains the production model and that the two weak fallbacks were removed on October 7. The source describes the route after those removals as using Luna, but does not clearly identify a separate replacement backup. No further deployment date or additional model change was announced in the report.
The next meaningful test would be continued performance on real production articles, particularly whether Luna maintains its reported source fidelity and valid-output rate over a larger sample. Meyer’s results make a case for task-specific checks, but the published comparison does not settle which model is best for other newsrooms.
As an affiliate, we earn on qualifying purchases.
Key Questions
Which model scored highest in Meyer’s news-writing test?
GPT-6 Astra scored 4.00 out of 5 in both English and German in the initial test. It averaged 3.93 across 12 articles in the robustness run.
Why does the pipeline use GPT-6 Luna instead?
Meyer reported that Luna averaged 3.67 in the robustness run and cost about $0.0012 to $0.0013 per article, compared with Astra at about $0.12. He judged Luna’s lower cost and consistent output a better fit for his high-volume workflow.
What went wrong with the fallback models?
Meyer said GPT-4.1 nano produced fabricated quotations and received a 1.00 source-fidelity score. He reported that local gpt-oss-120b produced no valid articles in four attempts and generated invented named quotes. Both were removed from the news route on October 7.
Did more reasoning effort improve the articles?
Not consistently in Meyer’s test. He said score differences across six GPT-6 Luna effort settings were within the noise, while the maximum setting cost about 12 times as much and took about 12 times as long as the no-effort setting.
Do these results show which model is best for every newsroom?
No. The test used Meyer’s own prompt, sources, languages and judging process. Its findings offer a comparison for that workflow, but performance may differ with other tasks, editorial standards or production systems.
Source: ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
