Autocorrect and the language models behind it are subtly shaping how we use language. This article explains the two key components driving modern autocorrect systems—the spatial model and the language model—and examines their limitations as well as a state-of-the-art solution: federated learning combined with on-device personalization. The article then moves beyond autocorrect to discuss and raise public awareness of the homogenizing effects of language models.
Introduction
Everyone has an autocorrect embarrassment story to tell. You try to type something perfectly normal, only to have your phone warp it into something ridiculous, and worse—you already hit “send”. Or, if any iOS user is reading this, you might still recall the “duck” ages when you, usually a very civil person, for whatever valid reason, finally indulged in using the f-word for once, but your Apple keyboard had to make you feel bad about it, and decided to correct it to “duck.” Thankfully, Apple fixed this issue upon the release of iOS 17 in 2023 [1].
Everyone has an autocorrect embarrassment story to tell. You try to type something perfectly normal, only to have your phone warp it into something ridiculous, and worse—you already hit “send”. Or, if any iOS user is reading this, you might still recall the “duck” ages when you, usually a very civil person, for whatever valid reason, finally indulged in using the f-word for once, but your Apple keyboard had to make you feel bad about it, and decided to correct it to “duck.” Thankfully, Apple fixed this issue upon the release of iOS 17 in 2023 [1].
Beyond the cringe-worthy but nonetheless arbitrary mistakes, why does autocorrect consistently fail certain demographics, and what could be the consequences of similar culturally or ethnically charged autocorrect failures? We will have to unearth the hidden mechanisms behind our simple-looking keyboards to truly understand.
How Does Autocorrect Work?
In the 1990s, Dean Hachamovitch led the development of the AutoCorrect feature in Microsoft Word [3]. Around the same time, Nokia adopted the T9 predictive-text system on its numeric keypads, which would later turn my mother’s antique handset into a childhood treasure of mine. You’d press keys, and the phone would guess the word through a dictionary lookup, and on my mother’s phone, it worked in both English and Chinese. It was a real wonder. However, today’s smartphones with our QWERTY keyboards and telepathic text suggestions make my childhood fascination seem unimpressive. What made autocorrect evolve from that rigid, rule-based system to what it is today—almost a weather forecaster for words?
It turns out that there are two big “brains” in every modern autocorrect system that provide a two-part solution to the problem:
1. The Spatial Model: This part of the system handles what you actually typed, which is harder than it sounds. Ever notice how you can be a bit sloppy on a touchscreen, even swiping from key to key, yet still get the right word? That’s thanks to the spatial model. It knows when you hit a standalone “O” but tapped a bit to the left, you probably meant “I” instead. Engineers call this the “fat finger” problem, where your finger covers a large area and might hit unintended keys [4]. For instance, if you tap somewhere in between the “T” and “Y” on screen, the model assigns a higher probability to “T” if your tap is slightly closer to “T”, but it doesn’t rule out “Y” entirely. For each position you tap, the system returns not a definitive letter but a list of possible letters ranked by likelihood.
2. The Language Model: This second brain has no idea where your fingers landed, but it knows a lot about human language. A language model is essentially a vast statistical map of how words tend to connect in the language you’re typing in. It can predict, for example, that “good job” is a more likely phrase than “good hob”, although both “job” and “hob” are real English words. In practice, the language model moves beyond the single-tap/single-character level dealt with by the first brain. It assigns a probability to any given sequence of words or letters based on how frequent they are overall and how well they fit the context of your sentence [4]. Thus, a good language model will replace “their” with “there” in “see you their”, but not “latte” with “later” in “see you later”, because “see you latte” is not a statistically likely usage.
Once these two powerful brains are in place, the real magic happens when they work together through what computer scientists call a “noisy channel” model [5]. The idea comes from information theory: think of your intended text as a signal that gets distorted, or “noisy”, by the time you hit the keys, like how your voice gets fuzzy on a wireless call when the connection is bad. The autocorrect system has to act as a “decoder” and reconstruct the original message. It does so by using information provided by both brains: 1) from the spatial model, it predicts what you possibly meant to type given where you tapped, and 2) from the language model, it predicts which of those possibilities forms a grammatically and pragmatically acceptable sentence in the context of the surrounding words.
In mathematical terms, the decoder is searching for the word that maximizes the P (intended word) x P (taps | intended word) [5]. In plain English, it asks: “Which word, if the user wanted to type it, would most likely produce the taps I saw and also fit into the sentence?” The decoder then assigns a score to various candidate words based on the formula above and picks the candidate with the highest score as the autocorrection.
When Autocorrect Fails Them: Names, Dialects, Stranded Identities
Now, we have all the tools in our toolbox to answer why autocorrect repeatedly fails certain demographics. Autocorrect operates based on a simple principle: pick the word with the highest language prior (how common it is) times the touch likelihood (how likely your taps are meant to produce the pressed key), which is great math for regular typos—but terrible for rare words. When your word isn’t well represented in the keyboard’s dictionary or training data—because it’s a less-frequent name, a dialect form, or a slang term specific to a community—it ends up with a small prior, and a higher-frequency candidate wins. While our models do not wake up seeking to ruin somebody’s day, the math, coupled with current datasets, does indeed tilt away from minority groups. Over time, those tiny tilts can accumulate into the feeling of being forced to assimilate into the mainstream.
This is exactly what the “I am not a typo” campaign in the UK was pushing back against; people whose names were constantly “fixed” by their phones demanded more inclusive autocorrect systems [2]. But names are only the most visible example. The same problem shows up whenever we try to type the way we speak. Another common autocorrect frustration is when you’re trying to use slang or a casual expression, and your phone keeps fighting you. This can happen even if you mostly use “standard” English, but it gets infinitely more frustrating for speakers of dialects like African American Vernacular English (AAVE) and frequent code-switchers from diverse cultural backgrounds.
In a study published in May 2025, surveyed AAVE speakers reported that typing the way they speak is an uphill battle with AI-supported writing tools—that “wasn’t built for us” [6]. Native American writer CMarrie Fuhrman also noted a cruel historical recurrence: for generations, indigenous children were forced to abandon their native languages and names in boarding schools, and now, a century later, our digital tools are still subtly enforcing a similar cultural erasure by refusing to accept non-standard forms [7].
…And How to Make it Right
One solution already in use is personalized training. This means that the keyboard doesn’t apply a one-size-fits-all model and instead lets a small part of the model adapt to you, so that the prior for your names, dialects, and slang rises from “rare” to “expected.” This approach has been shown to improve the discovery of out-of-vocabulary words [8] and could potentially benefit millions of users [9]. As shown in Figure 1, two complementary layers make this possible: on-device adaptation and federated learning.
1. On-device adaptation to raise your priors locally. Modern keyboards already keep a short-term cache of words you’ve recently typed and a long-term user dictionary for words you explicitly accepted; both can be used to reorient the math toward your idiolect. Cache-based language models treat recent words as temporarily
more probable, so if you’ve typed a rare name three times this week, the decoder will stop correcting it the fourth time. Similarly, when you tag a slang word as “correct”, the system will learn that this is a deliberate choice and update your personal dictionary. Controlled studies of text entry show that on-device personalization measurably improves typing efficiency compared to static language models [9], [10].
2. Federated learning to lift the tide for everyone, without collecting raw text. The promise of federated learning (FL) is that your private words shouldn’t have to leave your phone to help the global model become more inclusive. Phones can train locally on their own data and send encrypted, aggregated updates to the server, which then averages these into the next release of the global model [11]. Google’s Gboard demonstrated that FL can train next-word prediction and autocorrection at consumer scale while offering local differential privacy (LDP) guarantees so that no single user’s data can be extracted [12]. The combination of local training, secure aggregation, and LDP allows the keyboard’s global priors for underrepresented words to rise over time without compromising user privacy.
Of course, personalized training doesn’t erase bias in a snap. Your first uses may still get “corrected,” devices differ in computational power, and non-identically distributed data (i.e., each device holds a unique slice of language instead of the same pattern as others) can be tricky to train on. But we do have a working and improving solution that addresses—rather than silences—the problem.
Beyond Autocorrect: Are Language Models Paving the Way to Homogeneity?
Although predictive text does not use the spatial model, it uses the same underlying language model as autocorrect—just with a different mission. Autocorrect looks backward, with its decoder fed by current keypresses, whereas predictive texting looks forward, with its decoder fed by completed words to output the most likely next word or phrase. Just like how autocorrect can harbor subtle (or not so subtle) biases, predictive text may have an insidious effect of homogenizing how we use our language.
In a 2020 Harvard study titled “Predictive Text Encourages Predictive Writing,” 109 participants were asked to write image captions with varying features of predictive suggestions enabled. In the interface shown in Figure 2, before they typed a single word, the app suggested labels such as “woman,” “man,” or “person” along the top of the keyboard. The results were striking—when predictive suggestions are enabled, although participants could catch more typos, their writing becomes more succinct and predictable as well. Faced with the baseball image in Figure 2, for instance, writers were more likely to work from the given categories and type something generic like “a baseball player with the number eight jersey has just hit the ball with the bat,” instead of a more specific line such as “a man in a black shirt with the number eight and grey pants swinging a baseball bat with many players in the background” [13]. In other words, the individual flavor in people’s word choices and phrasing has been predicted away.
In addition, language and creative homogenization are not the work of predictive text alone. Evidence suggests that large language models (LLMs) are homogenizing creative ideas at scale as well. Researchers Doshi and Hauser conducted an online experiment to gauge the impact of ChatGPT-4 assisted ideation on short-story writing and found that writers allowed access to ChatGPT produced stories that were rated as more creative (by both self-evaluation and peer review), but those stories were more similar to one another, suggesting that LLMs can lead to “an increase in individual creativity at the risk of losing collective novelty” [14].
A more recent study in 2024 reinforced these findings. Participants who used ChatGPT to brainstorm answers to open-ended prompts—such as, “Suppose that a great fog has fallen over the earth and all we can see of people is their feet. What would happen?”—produced “less semantically distinct ideas” and exhibited a weaker sense of ownership to their work [15]. Indeed, if ideas are not earned but simply put into your head by ChatGPT, how responsible can you feel for them? AI writing tools powered by language models take much of the dirty work off your shoulders, but they also diminish the “youness” of the final product. Taken together, these studies echo sci-fi writer Ted Chiang’s statement, “[Generative AI] reduces the amount of intention in the world” [16]. As a species, we seem to have ceded part of our volition to AI and are filling the world with fluent text no one truly meant.
Conclusion
Autocorrect and other language-model-based technologies are now an integral part of our lives. Illumin’s editors would be overwhelmed with typos if not for modern autocorrect systems, and what’s not to love about predictive text for perfunctory emails? Yet convenience comes at a price. We must learn to master the tools rather than let the tools master us. That means pushing for solutions that adapt the tools to the diversity of our identities and languages, instead of allowing that diversity to be flattened to fit the tools’ probabilistic math. Scholarly attention to the marginalizing and homogenizing effects of language models is on the rise, and we, as users, have a stake in staying informed and vigilant.
References
[1] R. Treisman, “‘What the duck’ no more: Apple will stop autocorrecting your favorite swear word,” NPR, June 07, 2023. Accessed: Sept. 16, 2025. [Online]. Available: https://www.npr.org/2023/06/07/1180791069/apple-autocorrect-ducking
[2] R. Booth and R. B. S. affairs correspondent, “People with commonly autocorrected names call for tech firms to fix problem,” The Guardian, May 22, 2024. Accessed: Sept. 16, 2025. [Online]. Available: https://www.theguardian.com/technology/article/2024/may/22/people-with-commonly-autocorrected-names-call-for-tech-firms-to-fix-problem
[3] G. Lewis-Kraus, “The Fasinatng… Fascinating History of Autocorrect,” Wired. Accessed: Sept. 25, 2025. [Online]. Available: https://www.wired.com/2014/07/history-of-autocorrect/
[4] D. Weir, H. Pohl, S. Rogers, K. Vertanen, and P. O. Kristensson, “Uncertain text entry on mobile devices,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, Ontario, Canada: ACM, Apr. 2014, pp. 2307–2316. doi: 10.1145/2556288.2557412.
[5] M. D. Kernighan, K. W. Church, and W. A. Gale, “A Spelling Correction Program Based on a Noisy Channel Model,” in COLING 1990 Volume 2: Papers presented to the 13th International Conference on Computational Linguistics, 1990. Accessed: Sept. 25, 2025. [Online]. Available: https://aclanthology.org/C90-2036/
[6] J. Basoah et al., “Should AI Mimic People? Understanding AI-Supported Writing Technology Among Black Users,” Aug. 09, 2025. doi: 10.1145/3757423.
[7] Cm. Fuhrman, “Even the computer codes that autocorrect our words enforce a subtle racism,” Inlander. Accessed: Sept. 26, 2025. [Online]. Available: https://www.inlander.com/comment/even-the-computer-codes-that-autocorrect-our-words-enforce-a-subtle-racism-24933708
[8] Z. Sun, P. Kairouz, H. Sun, A. Gascon, and A. T. Suresh, “Private federated discovery of out-of-vocabulary words for Gboard,” Apr. 18, 2024, arXiv: arXiv:2404.11607. doi: 10.48550/arXiv.2404.11607.
[9] K. Wang, R. Mathews, C. Kiddon, H. Eichner, F. Beaufays, and D. Ramage, “Federated Evaluation of On-device Personalization,” Oct. 22, 2019, arXiv: arXiv:1910.10252. doi: 10.48550/arXiv.1910.10252.
[10] P. Sakkos, D. Kotsakos, I. Katakis, and D. Gunopulos, “Anima: Adaptive Personalized Software Keyboard,” Aug. 29, 2015, arXiv: arXiv:1501.05696. doi: 10.48550/arXiv.1501.05696.
[11] A. Hard et al., “Federated Learning for Mobile Keyboard Prediction,” Feb. 28, 2019, arXiv: arXiv:1811.03604. doi: 10.48550/arXiv.1811.03604.
[12] Z. Xu et al., “Federated Learning of Gboard Language Models with Differential Privacy,” July 17, 2023, arXiv: arXiv:2305.18465. doi: 10.48550/arXiv.2305.18465.
[13] K. C. Arnold, K. Chauncey, and K. Z. Gajos, “Predictive text encourages predictable writing,” in Proceedings of the 25th International Conference on Intelligent User Interfaces, Cagliari Italy: ACM, Mar. 2020, pp. 128–138. doi: 10.1145/3377325.3377523.
[14] A. R. Doshi and O. P. Hauser, “Generative AI enhances individual creativity but reduces the collective diversity of novel content,” Sci. Adv., vol. 10, no. 28, p. eadn5290, July 2024, doi: 10.1126/sciadv.adn5290.
[15] B. R. Anderson, J. H. Shah, and M. Kreminski, “Homogenization Effects of Large Language Models on Human Creative Ideation,” in Creativity and Cognition, Chicago IL USA: ACM, June 2024, pp. 413–425. doi: 10.1145/3635636.3656204.
[16] T. Chiang, “Why A.I. Isn’t Going to Make Art,” The New Yorker, Aug. 31, 2024. Accessed: Sept. 26, 2025. [Online]. Available: https://www.newyorker.com/culture/the-weekend-essay/why-ai-isnt-going-to-make-art
