Fix a text-to-speech model by editing its weights directly

The poster, u/N1kuson, is building Silencio-2, a finetune of Kyutai's Pocket TTS with inline tags for emotions and sounds. Lacking tagged data, most tags were unreliable and the whisper tag worked in about one take in five. Following a published method for weight-space editing, they had Claude on Opus 5.5 inspect the model's weights, write attention probes, run causal experiments and build an edit, keeping a change only if it improved a held-out set of unseen phrases, seeds and voices. The poster steered and listened to the audio.

Claude found the model read the happy tag 100 to 3,000 times more than average text but the others only 1 to 10 times. Boosting attention to the whisper tag at inference gave a whisper in 62% of takes. The final fix adds one rank-1 term per layer to the key weights, with about 5,000 free parameters and no gradient steps. Whisper improved, but some takes glitch mid-sentence, some whispers slur, and surprised went up in pitch. A commenter said sad, angry, fear and surprised sounded largely unchanged.

The harness was not named; the poster only says Claude did the probing and experiments. The metrics are proxies (Whisper WER, voiced-frame share, a nearest-centroid classifier), so judge by ear; the poster published before and after samples for every test take. Several ideas, like moving tag embeddings, did nothing.