What Generative AI Actually Is

Most explanations of generative AI are either marketing or mathematics. This chapter aims at the useful middle: a model of what is happening that is simple enough to hold in your head and accurate enough to predict behaviour.

Here is the core idea. A generative model has been trained on an enormous amount of text, and from that it learned the statistical shape of language — which words tend to follow which, in which contexts, at every scale from a phrase to an argument. When you give it a prompt, it repeatedly answers one question: given everything so far, what comes next?

Why 'it predicts the next word' is not dismissive

People sometimes say this to belittle the technology — it is 'just' autocomplete. That undersells what it takes to predict well. To reliably continue a paragraph about Nepali tax law, or a half-written function, or a customer complaint, the system has to have absorbed a great deal of structure about how those things work.

But the framing is genuinely useful, because it explains the failures. The system is optimizing for a plausible continuation. Usually the most plausible continuation is also the true one — which is why it is right so often. When those two come apart, it produces the plausible one. Confidently.

Tokens, not words

The model does not read words. It reads tokens — chunks of text that may be a whole word, part of a word, or a single character. In English, a token averages about three-quarters of a word. In Nepali and other Devanagari scripts, the same sentence takes far more tokens, because the training data had less of it to learn efficient chunks from.

This has two practical effects. First, cost and limits are measured in tokens, so writing in Nepali costs more than the same content in English. Second, the model's grasp is weaker where tokens are inefficient — which is part of why Nepali output is worse than English output.

The context window

Everything the model can 'see' at once — your prompt, the documents you attached, the conversation so far, and its own reply — must fit inside a fixed budget called the context window. Nothing outside that window exists for it.

This explains behaviour that otherwise seems strange. A long conversation starts forgetting earlier details, because they have fallen out of the window. A model that summarized your document perfectly cannot answer a question about a section you did not include. It is not being careless; that text is simply not there.

What training does and does not give it

Training gives the model general capability: language, reasoning patterns, common knowledge, coding conventions. It does not give it your business, your data, today's news, or anything that happened after training ended.

Everything specific has to arrive in the prompt. This is the single biggest lever you have over output quality, and it is the subject of most of this course: the model supplies capability, you supply context.

Why the same prompt gives different answers

At each step the model has a distribution over possible next tokens, and it samples from it rather than always taking the single most likely one. That randomness is deliberate — always taking the top choice produces flat, repetitive text.

For you this means two things. Run the same prompt twice and you get two different answers, both valid. And if an answer is nearly right, regenerating is often faster than arguing with it.