An interactive journey from words to meaning — inside the transformer.
Over the past few years, large language models have taken a gigantic leap forward in our decades-long quest to build intelligent machines. But how do they actually understand text?
This article explains the core mechanism — the transformer — through interactive visualizations. As you scroll, the graphics update to show exactly what happens inside the model when it reads a sentence like "Excellence Consulting by Mashup helps regulated companies adopt AI safely."
Tip: Scroll slowly. Each section triggers a new visualization.
To understand text, an AI must first translate words into a language it understands: numbers.
Take the sentence: "Excellence Consulting by Mashup helps regulated companies adopt AI safely."
The model does not "read" this the way you do. It needs to convert every word into a mathematical representation.
First, the sentence is broken into tokens — basic units the model can process.
Some words become single tokens. Others get split into pieces. Punctuation becomes its own token.
The model works with these tokens, not with the original words.
Each token gets converted into a vector — a long list of numbers.
These numbers are not random. They encode meaning. Words that appear in similar contexts get similar vectors.
For example, "consulting" and "advisory" would have vectors pointing in similar directions.
We can visualize these vectors in 2D space (in reality they have hundreds of dimensions).
Words with similar meanings cluster together. "Excellence", "quality", and "standard" form one cluster. "AI", "model", and "algorithm" form another.
This is the foundation of how the model "understands" language.
But words do not have fixed meanings. "Bank" in a financial document means something different from "bank" on a river.
Early models used the same vector for every occurrence of a word. Modern transformers create contextualized embeddings — vectors that change depending on surrounding words.
This is where self-attention comes in.
Self-attention is the breakthrough that makes transformers powerful. It allows every word in a sentence to "look at" every other word.
When the model processes "regulated", it needs to know: regulated what? The attention mechanism draws a connection to "companies".
Every word gets to decide which other words are most relevant to its meaning.
For each word, the model creates three vectors: a Query (what am I looking for?), a Key (what do I contain?), and a Value (what information do I hold?).
The Query of one word is matched against the Keys of all other words. A high match means strong attention.
This is how "helps" knows to connect to "companies".
The model computes a score for every pair of words. These scores determine how much information flows between words.
In our sentence, "Mashup" pays strong attention to "Excellence Consulting" — it knows those words define what Mashup is.
The scores are normalized so they sum to 1, creating a probability distribution of attention.
Here is the full attention map for our sentence. Thicker lines mean stronger attention.
Notice how "adopt" connects strongly to "AI" — the model understands what is being adopted. And "safely" connects back to "adopt" — the model grasps that safety modifies the adoption process.
This web of connections is built for every layer of the model.
The model does not just build one attention map. It builds many — in parallel.
Each "head" learns a different type of relationship. One head might track grammatical subject-verb agreement. Another might track semantic similarity. Another might track regulatory terminology.
This is why transformers can capture such rich linguistic structure.
Understanding text is only half the story. The model can also generate new text, one token at a time.
Given "Excellence Consulting by Mashup helps", what comes next? The model computes a probability for every word in its vocabulary.
"companies" might have a 35% probability. "organizations" 18%. "teams" 12%.
The model ranks every possible next word by probability. Only a small number are serious candidates.
The visualization shows the top candidates and their probabilities. The model does not "know" the right answer — it simply estimates what is most likely based on everything it has seen during training.
Instead of greedily picking the highest-probability word each time, advanced models use beam search.
They keep track of multiple candidate sequences simultaneously. A word that looks good immediately might lead to a dead end. A slightly less likely word might open up a much better path.
This is how the model produces coherent, flowing paragraphs.
The model has a "creativity" dial called temperature.
At low temperature, the model always picks the safest, most probable word. The output is predictable and factual — good for regulatory summaries.
At high temperature, the model takes more risks. The output becomes more diverse and surprising — useful for brainstorming.
The transformer combines all these mechanisms: tokenization, embeddings, multi-head self-attention, and probabilistic generation.
This architecture powers the AI systems that are reshaping industries — from drug discovery to regulatory compliance to organizational design.
Understanding how these models work is the first step toward using them responsibly in regulated environments.
Large language models are not magic. They are mathematical machines that learn patterns from vast amounts of text. The transformer architecture — with its elegant attention mechanism — has made it possible to build models that capture nuanced meaning and generate human-like text.
For organizations in regulated industries, understanding these fundamentals is critical. Whether you are implementing AI for clinical trials, regulatory submissions, or quality management, knowing how the model processes language helps you ask better questions and manage risk more effectively.
We help regulated companies build AI governance, compliance frameworks, and safe deployment strategies.
Get in TouchAbout the author: Milos is a consultant at Excellence Consulting by Mashup, specializing in AI governance, regulatory compliance, and organizational transformation for regulated industries.