How AI Models Work (Basic Understanding)

A 5-minute mental model of an LLM. Skipping the math: an LLM is a giant function that, given some text, predicts the next most-likely token. Repeat that prediction 200 times and you've got a paragraph. That's it. Everything else (chain-of-thought, RAG, agents) is clever ways of steering that one core operation.

LLMs don't read characters or words — they read tokens, which are sub-word chunks. Roughly, 1 token ≈ 0.75 English words ≈ 4 characters.

'Hello world' →   ['Hello', ' world']                       (2 tokens)
'codekilla'  →    ['cod', 'ek', 'illa']                     (3 tokens)
'😀'          →    ['<emoji bytes>']                        (1-3 tokens)

This is why your model has a token limit, not a word limit. GPT-4o = 128 K tokens. Claude Sonnet 4.5 = 200 K. Gemini 3 Pro = 2 M.

1. You send prompt → tokens.
2. The model computes P(next_token | tokens_so_far) — a probability over its vocabulary (~100 K-200 K tokens).
3. It samples one token (controlled by temperature and top-p).
4. Append the new token to the input. Repeat until <stop> or max_tokens.

That's the entire algorithm. The magic is in step 2 — the model has trillions of parameters that learned, from the entire public internet, how to weight every possible next token.

Step 2 above is conditional on everything you put in the prompt. Add a role and the probability mass shifts toward expert-sounding tokens. Add a JSON example and the model will produce JSON. Add 'Let's think step by step' and it dedicates compute to reasoning tokens before answering. Your prompt literally re-shapes the next-token probability distribution.

• An LLM is a next-token predictor — repeated 100s of times to make a paragraph.
• Tokens ≠ words — 1 token ≈ 0.75 word for English. Other languages are much pricier.
• Token limit is the real constraint — counts both your prompt AND the model's reply.
• Your prompt re-shapes the probability distribution over the next token — that's why it has so much power.

1. Estimate the token count of: 'The quick brown fox jumps over the lazy dog.' Then check at https://platform.openai.com/tokenizer.
2. Run the same prompt at temperature 0.0 and 1.0. Compare 5 outputs each. Which is more consistent? Which is more creative?
3. Compare token counts of 'I love programming' in English vs. its Hindi translation 'मुझे प्रोग्रामिंग पसंद है'. Why is Hindi 3-4× more expensive?

• Always log prompt_tokens + completion_tokens in production — that's your bill.
• Non-Latin scripts cost 2-5× more tokens. Translate prompts to English internally if possible.
• Use max_tokens as a safety net — protects against runaway generations.