Write and train a tiny language model that runs on DOS from a floppy

A Redditor had Claude write a 367K-parameter transformer and generate its training data; the model and program fit in about 435 KB and run under DOS on a Pentium.

Done withClaude· Nov 15, 2026

What they did

A Redditor on r/ClaudeAI wanted to see whether a working language model could fit the limits of old PC hardware. They let Fable 5.2 write the transformer code itself, and used Opus 5.5 to produce the training set: roughly 3,500 facts plus small talk and simple maths, each phrased as questions and answers in several different ways. The resulting model has 367K parameters. Its weights are stored as 8-bit integers with one scale value per row, so the program and the weights together come to about 435 KB, small enough for DOS memory and a floppy disk. It generates text one character at a time on an Intel Pentium, shown in a video in the post.

How it went

The poster says it answers well on words and facts it saw in training, but anything complex or long turns into nonsense. The demo was enough to draw close to 1,000 upvotes. The post does not say how long training took or what it cost.

Worth knowing

One commenter called it fake, arguing a language model cannot fit on a floppy; others replied that a narrow model trained on a small fixed set of facts can, and pointed to similar demos on a Commodore 64. The poster names only the models, not the tool Claude ran in, and does not say who ran the training job.

Try it

With Claude· 6 parts to fill in

Help me build a tiny character-level transformer that runs on [target hardware, e.g. a Pentium under DOS] with a total size under [size limit, e.g. 1.2 MB]. First write a training set of about [number] facts on [topics], each phrased as questions and answers in several ways. Then write the model and training code, keep parameters under [parameter budget], quantise weights to 8-bit integers with one scale per row, and write an inference program in [language and compiler for the target] that reads and prints one character at a time. Stop after training and show me the file sizes and ten sample answers before you port anything to the target machine.

Square brackets are the parts to fill in. The prompt is ours; the story is theirs.

Where they said it

Related