A keyboard that cannot reach the network cannot download a language pack. So every language FrankenKeys knows has to be in the package on install day, and the package has to stay small enough that people install it. This is what is in there, where it came from, and what the trade cost.
The thirty
English, Afrikaans, Basque, Catalan, Croatian, Czech, Danish, Dutch, Estonian, Filipino, Finnish, French, Galician, German, Hungarian, Icelandic, Indonesian, Italian, Latvian, Lithuanian, Malay, Norwegian, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Turkish.
Every one has prediction, autocorrect and swipe. Every one has its own accent long-presses merged into the layout when it is your primary or your "also type in" language, so Czech gets ř and ů on the keys where they belong and Catalan gets the middle dot for l·l. The language key cycles through whichever of the thirty you have chosen, and the speech recogniser is told which one you are dictating in.
Missing, and I will not pretend otherwise: Arabic, Vietnamese, Hindi, Chinese, Japanese, Korean, Russian, Greek, Hebrew, Thai. The current thirty are the Latin-script languages for which I had a clean frequency corpus and a layout I could test. Non-Latin scripts need input methods, not just dictionaries, and I would rather ship none than a bad one. If your language is on that second list, this keyboard is not yet for you.
What a dictionary is
Two plain-text files per language. A word list: one lowercase word per line, ordered by corpus frequency, so the line number is the rank. A bigram list: pairs of words that follow each other, also frequency-ordered, capped at 30,000. No binary format, no database. Twenty-seven languages carry exactly 25,000 words and 30,000 pairs. English carries 25,008 words and 27,434 pairs. Tagalog has 9,877 and 5,012, Afrikaans 17,576 and 15,494, because their source lists are shorter and I did not pad them.
Prediction works from these plus what you type. The engine opens only the one or two languages you have selected, so the cost of the other twenty-eight is download size, not memory or startup time.
Where the words came from
English is packed from Peter Norvig's web-corpus n-gram counts, which derive from the Google Web Trillion Word Corpus and are freely usable. About 29 English contractions the corpus tokeniser drops are boosted back in by hand.
The other twenty-nine come from the hermitdave/FrequencyWords repository on GitHub, MIT licensed, which derives word frequencies from the OpenSubtitles 2018 corpus via the OPUS project. Subtitle corpora skew conversational, which for a phone keyboard is the right skew. The attribution is in the app's notices file and in the repository documentation, and both sources are public, so the dictionaries can be regenerated by anyone who wants to check them.
What it weighs
All thirty dictionaries together are about 15 megabytes of text, about 6.5 megabytes once compressed inside the APK. Every install carries all of it, including a person who will only ever type English. That is the price of "no download": paid once, by everyone, on the day they install.
A cloud dictionary would be smaller on install and would let me add languages without a release. It would also mean every language choice is a request to a server that now knows what language you speak, and a keyboard that holds a network permission to make the request. I chose the 6.5 megabytes.
Bilingual typing
Since 3.13.0 the engine ranks candidates across two active languages at once, so a Spanish speaker texting in English does not have to switch to type "mañana". The two dictionaries are both open, the ranker merges them, and the personal dictionary is kept per language so a learned word is learned in the language it was typed in.
What is learned, and where it goes
Words you type that the dictionary does not know can be learned into a personal list, per language, stored in app-private storage on the phone. They are never uploaded, because nothing is. They are not learned in password fields, in fields an app marks as no-learning, or in incognito modes. "Forget learned words" on the Privacy page deletes them, and the privacy inventory lists every other store beside them.
The numbers, and where to check them
- 30 languages:
LangCycle.CHOICESin the shipped build, and the 60 files underassets/dict/. - 25,000 words and 30,000 bigrams in twenty-seven of the thirty, with the English, Tagalog and Afrikaans exceptions above: count the lines.
- Sources: norvig.com/ngrams and github.com/hermitdave/FrequencyWords.
- The format rules are asserted by a checker script before every release: UTF-8, LF only, lowercase, NFC, no duplicates, no blank lines.
The thirtieth language cost the same as the second: two text files and a layout. The thirty-first will cost a script.