From words to numbers: how AI builds meaning

For centuries, language was considered a border between the human and the artificial…

For centuries, language was considered a border between the human and the artificial. Speaking, understanding, interpreting: all of it seemed to require consciousness and experience. Yet today we talk with machines that translate, write and answer with a fluency we once attributed exclusively to human intelligence.

How can an AI "understand" language without really understanding it? How does it turn words, so loaded with history and ambiguity, into mathematical structures that let it anticipate the meaning of a sentence?

From symbol to geometry

In its early days, computational linguistics treated language as a system of rules. Each word had a predefined meaning, and the machine only had to apply it. But human language does not work that way: it is ambiguous, changeable and deeply contextual. The conceptual turn came with a simple but revolutionary idea from the British linguist John R. Firth:

"You shall know a word by the company it keeps."

Formulated in 1957, that insight¹ gave rise to a whole new way of thinking about language: if meaning depends on context, it can be inferred statistically. Instead of trying to "explain" what a word means, we can observe which other words it appears with and under what conditions.

From that idea came the distributed representation of meaning: each word is turned into a series of numbers, a vector in a space of many dimensions. Instead of a dictionary of definitions, the machine builds a mathematical map where meaning is expressed as distance.

Language turned into space

In the early 2010s, that principle took shape in models such as Word2Vec² and GloVe³. Both showed that machines could learn meaning simply by observing how words combine with one another.

Each word becomes a point in space. "King" and "queen" are close, as are "man" and "woman." What is more, the relationships hold: if we subtract man from woman and add king, the result points toward queen. That is the magic of embeddings: meaning is not defined, it emerges from the geometry. Language becomes topography. And in that topography, the AI does not "know" what each word means, but it understands how they relate to each other.

As the linguist Alessandro Lenci explains, distributional models "do not seek to capture pre-existing meanings but to reproduce the patterns of use that shape them"⁴. Meaning, in other words, is inferred from use.

From words to context

Word embeddings were the first step. A deeper question soon arose: could an AI understand that the word bank does not mean the same thing in "park bench" as in "central bank"? Models like ELMo⁵ and later BERT⁶ answered yes. They introduced so-called contextual embeddings, where the meaning of a word depends on the words around it. There is no longer a single representation per word, but many, adjusted to each context.

That is how deep language understanding was born: a neural network that no longer processes sentences in a line but as networks of simultaneous relationships. That innovation, the attention mechanism, was presented in Google's paper "Attention Is All You Need"⁷ and changed the history of AI for good. From that model came today's generative systems, capable of holding conversations, translating with nuance and even writing in a style of their own.

Understanding without understanding

When an AI answers, it does not look anything up in a database. It predicts, from its map of relationships, which word makes the most sense in context. It does not interpret, but it simulates comprehension so effectively that the line between understanding and predicting blurs.

It is not thought. It is geometry with semantic consequences. And perhaps that is where its fascination lies: not in what it understands, but in how it forces us to redefine what it means to understand.

References

1. Firth, J. R. (1957). A synopsis of linguistic theory 1930-1955. Studies in Linguistic Analysis.
2. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.
3. Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global vectors for word representation. Proceedings of EMNLP.
4. Lenci, A. (2018). Distributional models of word meaning. Annual Review of Linguistics, 4(1), 151-171.
5. Peters, M. E., Neumann, M., Iyyer, M., et al. (2018). Deep contextualized word representations. Proceedings of NAACL-HLT.
6. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT.
7. Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS).