Part 1: What Does Generative AI Learn as the "Answer"? — Self-Supervised Learning and LLMs

What Is the Thing We Call “AI,” and How Is It Built?

People often say “AI is amazing these days.” But what that “AI” actually refers to is surprisingly vague. The thing that writes text like ChatGPT, the thing that draws images, the thing that reads text aloud — day to day we lump them all together as “AI,” but technically these are called generative AI, and at their center is a mechanism called the large language model (LLM). Machine learning, deep learning, LLMs, generative AI… there are many words, yet the boundaries between them are rarely noticed. For most people, it’s all just one bundle called “AI.”

What this series takes on is one layer deeper. Beyond “AI is amazing” lies the question: how are LLMs and generative AI actually built? In particular, we focus on what the model is made to learn — how its “study material” is created.

It may come as a surprise, but LLMs like GPT, image generators like Stable Diffusion, speech synthesis, video generation, and robot world models look completely different, yet the skeleton of how they learn is remarkably shared. In one line, it’s this:

Don't feed the raw data in as-is.
Mechanically construct "input" and "answer" pairs from it.

In this series, we follow this “how the study material (training data) is made” model by model. Part 1 starts with the idea that runs through all of them, and the most fundamental case: the LLM.

An LLM Is a “Text-to-Text” Model

Before we get into how it’s made, let’s pin down what our protagonist, the LLM, actually is. A large language model (LLM), put roughly, is a model that takes in text and returns text that fits as its continuation. Since both input and output are text, it’s sometimes described by its initials as “TXT2TXT (text-to-text).”

Type “What is a B-tree” and it continues with “It’s a tree structure that speeds up search…”. The reason it can answer questions, translate, and write code like ChatGPT all comes down to the same single behavior: generating a plausible continuation.

So how does the model acquire that “plausible continuation”? That’s what the rest of this article is about.

What “Learning” and “Training Data” Even Are

Before the main topic, let’s check the words one at a time. First, what is “learning”? Just as a person memorizes multiplication tables through repetition, a model’s learning is the work of seeing a large number of examples and capturing the patterns hidden in them as numbers (parameters). The collection of examples to learn from is the learning data.

Machine learning has several styles for how you supply that learning data, but a representative one is supervised learning. Here, each example comes with a set: “this is the input, this is the correct answer.” That “input-and-answer pair” is the training data. Think of a school worksheet with problems (inputs) and model answers (correct answers). The model gradually adjusts its parameters so that the gap (error) between its own answer and the correct answer shrinks.

Turned around, this means that to train a model, you must have both an “input” and a “correct answer” lined up. How you obtain that correct answer is the very first hurdle in building generative AI.

Raw Data and Training Data Are Not the Same

Here’s one easily confused point. The raw data gathered from the world and the training data built from it are not the same.

• Raw data: the raw material gathered from the world — web pages, books, papers, code, images, audio, video.
• Training data: the input/answer pairs constructed from raw data to show the model “this is the input, this is the answer.”

Organized by model, it looks like this:

Notice that for most models, humans do not attach the “answer” one item at a time. The answer is extracted automatically by exploiting the structure of the raw data. This is called self-supervised learning.

An LLM’s Raw Data — First, Collect Text and Clean It

For LLM pretraining, you collect an enormous amount of text:

• Web pages, books, papers, Wikipedia
• GitHub code, Q&A, manuals, news
• Conversation logs, formulas, tables, structured data like JSON

But you never use it as-is. First you run preprocessing to strip out noise.

import re

def clean_text(text: str) -> str:
    text = re.sub(r"\s+", " ", text)       # collapse runs of whitespace
    text = re.sub(r"<[^>]+>", "", text)    # drop leftover HTML tags
    return text.strip()

In reality the processing is far heavier — deduplication, language detection, quality scoring, harmful-content removal — but the goal is simple: turn it into clean text usable for training.

Tokenization — Turning Text into a Sequence of Integers

Clean text still can’t enter the model directly. You tokenize it into a sequence of integers.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")

text = "B-tree indexes are widely used in databases."
ids = tokenizer.encode(text)

print(ids)                    # an integer sequence like [33, 12, ...]
print(tokenizer.decode(ids))  # can be restored to the original text

In practice you don’t use plain split() but subword splitting like BPE or SentencePiece. This reduces “unknown words” while keeping the vocabulary at a realistic size.

This Is How the Training Data Is Made — Just Shift by One Token

Here is the heart of the LLM. Once you have the integer sequence, you make the input shifted by one token the answer.

import torch

ids = torch.tensor(ids)

x = ids[:-1]   # input: the sequence minus the last token
y = ids[1:]    # answer: the sequence minus the first token (x shifted one ahead)

Written out as text, it looks like this:

input:  B-tree indexes are widely used in
answer:        indexes are widely used in databases

You’ve simply turned it into the problem of guessing, at each position, “the token that comes next.” Humans are not attaching “the answer to this sentence is this.” Just by shifting existing text, you get a near-infinite amount of training data automatically. This is one reason LLMs can learn at web scale.

The training loop, conceptually, is very plain:

import torch.nn.functional as F

def train_step(model, input_ids, optimizer):
    x = input_ids[:, :-1]
    y = input_ids[:, 1:]

    logits = model(x).logits

    loss = F.cross_entropy(
        logits.reshape(-1, logits.size(-1)),
        y.reshape(-1),
    )

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    return loss.item()

Just keep solving the “guess the next token” problem, at massive scale. Grammar, knowledge, style, and even simple reasoning come along as byproducts.

From “A Machine That Writes Continuations” to “A Partner That Follows Instructions”

That said, a model built only from next-token prediction is nothing more than a machine that writes the continuation of text. Ask it a question and it won’t necessarily answer dutifully — it may just write a plausible continuation.

To give it the behavior of “following instructions” like ChatGPT, you add instruction data. The format is pairs of instruction and ideal answer.

{
  "instruction": "Explain a B-tree.",
  "output": "A B-tree is a self-balancing tree structure that keeps search, insertion, and deletion fast."
}

You reshape this into conversation form and train on next-token prediction over the answer portion.

User:
Explain a B-tree.

Assistant:
A B-tree is a tree structure that speeds up search.

The mechanism itself is the same “next-token prediction” as pretraining, but by aligning the data’s content to “instruction → desirable answer,” the character of the model’s responses changes. Pretraining handles language and knowledge; instruction tuning handles obedience — the roles are split.

AI Has Begun Making the Training Data

One more major recent shift. Instruction data used to be written by hand; now it’s common to have another AI make it.

For example, in a technique called “Self-Instruct,” you give an existing LLM some seed prompts and have it generate large volumes of derived questions and answers.

Explain a B-tree.
   ↓ auto-generated by the LLM
Explain a B+ tree.
Compare a B-tree and an AVL tree.
Explain insertion into a B-tree.

The training data multiplies itself this way. Being both “an AI that trains models” and “an AI that makes training data” — this dual role is an important theme that recurs across the images, audio, video, and world models we cover in later parts.

Summary

Part 1 boils down to three points:

• The essence of generative-AI learning is not “feed the raw data in as-is” but “mechanically construct input/answer pairs from the raw data.”
• An LLM can auto-generate “next-token prediction” training data just by tokenizing text and shifting by one — which is why it can learn at web scale.
• Pretraining gives it language and knowledge; instruction tuning gives it instruction-following. And that instruction data itself is increasingly made by AI.

Next time, we’ll see how this idea of “constructing input and answer” takes on a completely different shape in image generation (TXT2IMG) — as the problem of guessing the noise.