We renamed a class, and the hero forgot how to shoot

While building the Ukrainian cells for a language experiment, we translated a class name: «следопыт» became «слідопит». Nothing crashed. The DM kept running the session, the prose read fine. The hero just stopped getting his weapon-proficiency bonus.

There was no error. There was also no test that went red.

The name was the key

The save held the class’s Russian name, not a code for it. The rules that decide which weapons a class is trained with took that name and looked it up in their table. «следопыт» was in there. «слідопит» was not. Nothing matched, the table returned nothing, and nothing meant “no restrictions” — so the check quietly stopped applying.

There were ten places like that. Item lookup by name. The heavy-armour condition the game applies itself and forbids the DM from setting. The “circumstances” reason on a roll. The word list the game uses to notice that the DM is describing a group of people rather than one person. All of it resolved through Russian strings.

They have one thing in common: when they break, they don’t complain. The game doesn’t crash or raise anything — it goes on narrating beautifully while quietly ceasing to obey its own rules. Armour stops counting. A class stops being stopped from wearing what should stop it. A crowd reads as one person. That’s the worst kind of failure, because from the outside it looks like success.

Which inverted the order of the work: separate the mechanics from the language first, translate the text second. Do it the other way round and you get a game that speaks convincing English and silently doesn’t work.

The separation: every item got a Latin identifier, every class and condition a Latin key. The name became what it should always have been — the way you show the thing to a person. The DM still reads and writes words, but everything resolves through keys, which is why a translated catalogue keeps obeying its own rules.

We also re-checked an old decision

Five days earlier we had decided that the descriptions in the schema the DM answers through had to be in Russian: with an English schema, calques crept into the Russian prose. The implication was that every new language means another copy of the schema and the system prompt, forever.

Before paying that a third time, we measured it. Twelve live runs, two languages, two models, varying proportions of English in the instructions — rated blind, first by a native speaker and separately by a second model.

The old decision didn’t hold up. An English schema with a short per-language block produces perfectly good Russian and Ukrainian. What helped back then was, most likely, not the translated descriptions but two other edits that shipped in the same change.

So there is one instruction tree now, in English, plus roughly 1,800 characters written per language — instead of a translated prompt of 11,885 characters and 13,554 characters of descriptions. The saving is not in money: the English schema is longer in characters and cheaper in tokens (36,015 against 35,277 characters, but 15,393 against 15,699 tokens). You cannot reason about this from text length. You have to measure.

Then we opened three languages, and the tests didn’t notice

Now the honest part. We turned English and Ukrainian on, 1,249 tests were green, and the first day turned up this.

A class’s starting kit arrived on the character-creation screen in Russian while the shop beside it was in the chosen language. The screen matches one against the other by name, so it never matched once: 17 lines out of 17 in an English game. The kit looked unequipped and the money looked unspent.

A helmet the DM invented went into the body slot in an English game. Helmets were identified from a list of Russian words which had become trilingual on the server long ago and had stayed Russian in the browser.

Losing a fight earned the hero a scar — and in an English game, a Russian one. Scars cannot be removed, so it stays in the save for good and comes back to the DM later as fact.

Then there is the check that should have caught all of this. It listed the fields it looked at: title, description, class name, catalogue. The field the bug was in wasn’t on the list. The check was green, standing right next to the defect.

What we changed about how we work

Stop checking against a list. That same check now walks the whole response and objects to any Russian text outside a few deliberately technical fields; a new field falls under it automatically, without anyone remembering. And aim at the shape, not the alphabet: a check for Cyrillic would have found a Russian word list, but not an English one — so the test now forbids the browser from looking at an item’s name at all.

And the dullest one: log in and look. Of eleven defects, not one was found by a test run. Three came out of live sessions, two from looking at a screen that happened to show three campaigns in three languages side by side, and the rest from walking the code with a parser instead of grep.

We don’t write the Ukrainian as native speakers, and no check will catch that: surzhyk is assembled from letters both alphabets share. So a person reviews the Ukrainian here, and always will. If you spot an unhappy word, tell us — that’s exactly the case where a message from a player is worth more than another test.