Kako AI razume besedilo

From Words to Numbers

To understand text, an AI must first translate words into a language it understands: numbers.

Take the sentence: "Excellence Consulting by Mashup helps regulated companies adopt AI safely."

The model does not "read" this the way you do. It needs to convert every word into a mathematical representation.

Tokenizacija

First, the sentence is broken into tokens — basic units the model can process.

Some words become single tokens. Others get split into pieces. Punctuation becomes its own token.

The model works with these tokens, not with the original words.

Word Vectors

Each token gets converted into a vector — a long list of numbers.

These numbers are not random. They encode meaning. Words that appear in similar contexts get similar vectors.

For example, "consulting" and "advisory" would have vectors pointing in similar directions.

Visualizing Meaning

We can visualize these vectors in 2D space (in reality they have hundreds of dimensions).

Words with similar meanings cluster together. "Excellence", "quality", and "standard" form one cluster. "AI", "model", and "algorithm" form another.

This is the foundation of how the model "understands" language.

Context Changes Meaning

But words do not have fixed meanings. "Bank" in a financial document means something different from "bank" on a river.

Early models used the same vector for every occurrence of a word. Modern transformers create contextualized embeddings — vectors that change depending on surrounding words.

This is where self-attention comes in.

The Attention Mechanism

Self-attention is the breakthrough that makes transformers powerful. It allows every word in a sentence to "look at" every other word.

When the model processes "regulated", it needs to know: regulated what? The attention mechanism draws a connection to "companies".

Every word gets to decide which other words are most relevant to its meaning.

Query, Key, Value

For each word, the model creates three vectors: a Query (what am I looking for?), a Key (what do I contain?), and a Value (what information do I hold?).

The Query of one word is matched against the Keys of all other words. A high match means strong attention.

This is how "helps" knows to connect to "companies".

Attention Scores

The model computes a score for every pair of words. These scores determine how much information flows between words.

In our sentence, "Mashup" pays strong attention to "Excellence Consulting" — it knows those words define what Mashup is.

The scores are normalized so they sum to 1, creating a probability distribution of attention.

Visualizing Attention

Here is the full attention map for our sentence. Thicker lines mean stronger attention.

Notice how "adopt" connects strongly to "AI" — the model understands what is being adopted. And "safely" connects back to "adopt" — the model grasps that safety modifies the adoption process.

This web of connections is built for every layer of the model.

Večglava pozornost

The model does not just build one attention map. It builds many — in parallel.

Each "head" learns a different type of relationship. One head might track grammatical subject-verb agreement. Another might track semantic similarity. Another might track regulatory terminology.

This is why transformers can capture such rich linguistic structure.

Generating Text

Understanding text is only half the story. The model can also generate new text, one token at a time.

Given "Excellence Consulting by Mashup helps", what comes next? The model computes a probability for every word in its vocabulary.

"companies" might have a 35% probability. "organizations" 18%. "teams" 12%.

Probability Distribution

The model ranks every possible next word by probability. Only a small number are serious candidates.

The visualization shows the top candidates and their probabilities. The model does not "know" the right answer — it simply estimates what is most likely based on everything it has seen during training.

Iskanje s snopi

Instead of greedily picking the highest-probability word each time, advanced models use beam search.

They keep track of multiple candidate sequences simultaneously. A word that looks good immediately might lead to a dead end. A slightly less likely word might open up a much better path.

This is how the model produces coherent, flowing paragraphs.

Temperature and Creativity

The model has a "creativity" dial called temperature.

At low temperature, the model always picks the safest, most probable word. The output is predictable and factual — good for regulatory summaries.

At high temperature, the model takes more risks. The output becomes more diverse and surprising — useful for brainstorming.

Združevanje vsega skupaj

The transformer combines all these mechanisms: tokenization, embeddings, multi-head self-attention, and probabilistic generation.

This architecture powers the AI systems that are reshaping industries — from drug discovery to regulatory compliance to organizational design.

Understanding how these models work is the first step toward using them responsibly in regulated environments.