Keep a business directory current with a local-model fact-checking pipeline

The owner of USA Arms runs agents on three local GPU boxes to verify facts, resolve business identities and extract hours for tens of thousands of listings: about 4,477 validated jobs an hour at 98/100.

Done with

Custom agent

ReportedOct 15, 2026

What they did

u/NickReese runs USA Arms, a directory of tens of thousands of firearms-related businesses such as dealers and ranges, whose details change constantly. A custom agent harness keeps the site current: models check source facts, work out which listings are the same business, and extract services and opening hours. Every answer is verified against what the agent actually saw, and the evidence is stored so bigger models can audit the work later. When hosted models began refusing ordinary research for the business, the poster moved the work to three local machines, each with four Radeon AI PRO R9700 cards, running Qwen 3.8 27B in FP8 on vLLM. The same boxes also run the company's CI builds.

How it went

On the poster's graded extraction benchmark one box does about 4,477 validated jobs an hour at 98/100 correct in FP8, against 95.3/100 and 4,819 an hour with a smaller MXFP4 quantisation. Six Mac Studios had scored 99/100 but managed only 723 jobs an hour combined; matching throughput would have meant about 41 more Macs, roughly $280k, against about $10.7k per GPU box. Running builds alongside inference slowed builds 3 to 8% without hurting throughput.

Worth knowing

The poster chose FP8 because a wrong fact that gets into the directory spreads and is costly to remove, so accuracy beat speed. Hosted models refused tasks such as identifying a licensed dealer or reading an ammunition seller's site, which broke the harness at random. GLM gave the best results in their tests; Qwen gave the best cost to performance.

Try it yourself withTry it withCustom agent

5 parts to fill in

You maintain listings for [my directory] at [data location]. For each listing in [the queue], visit its sources, confirm the business identity, and extract [services, hours, contact details]. Record only facts you can quote from a page you actually opened, with the URL and the quoted text saved as evidence for later audit. If sources disagree or a fact cannot be confirmed, mark it unverified rather than guessing. Process [number] listings, then stop and show me a sample of changes with their evidence before writing any of them to the live site.

Square brackets are the parts to fill in. The prompt is ours; the story is theirs.