Textual Inversion

ELI5 (Explain Like I'm 5)

• The Big Idea: The model already knows how to draw almost anything — it just doesn't have a word for your specific subject yet. Instead of touching any of its drawing skill, Textual Inversion invents one brand-new "word" and teaches the model only what that one word should point to. Every dial and every existing skill stays frozen; you're only writing one new dictionary entry.
• Analogy: Imagine an incredibly versatile illustrator who can draw literally anything you describe. You don't send them back to art school to learn your bag — you just teach them one private nickname for it, say "Bloop." Say "draw a Bloop" and out comes your bag, rendered using every drawing skill the illustrator already had. You taught them a label, not a new skill, so a Bloop can only ever look as good as their existing style allows.
• Example: We freeze a digit-drawing model completely and train a single 128-number "word" for a bag using 20 photos. The whole result fits in about the size of a text message (512 bytes) — and every digit the model could already draw still looks exactly the same, because we never touched a single one of its original settings.

Key Insight

Textual Inversion takes the opposite tack from DreamBooth: it changes nothing in the diffusion model itself and instead learns a single new word embedding — a fresh row added to the text encoder's embedding matrix — that points at your subject. Because only that one vector is trained, the result is a few kilobytes, the smallest personalization artifact there is. The catch is capacity: a single vector can capture a recognizable "vibe" but cannot match the fidelity of LoRA or DreamBooth, because the frozen weights can only render what the model already knows how to draw.

What's in this directory

File	Role
textual_inversion.py	InvertedToken — frozen base + one trainable token vector spliced in wherever the label equals V — and the training loop

Our base is the phase-5 class-conditional DDPM, whose class-embedding table plays the role of the text encoder's vocabulary. A "token" here is one row of that table.

# reuse the shared conditional base (also used by projects 51/55/56):
python ../51-dreambooth/train_cond_base.py --out checkpoints/cond_base.pt
python textual_inversion.py     # ~2 min

How one vector is trained

The base has an embedding row per digit; a normal prompt looks a row up and adds it to the time embedding. Our new token has no row. InvertedToken keeps the entire model frozen and holds a single learnable vector; at every forward pass it splices that vector in wherever the label equals V. Back-propagation reaches only that vector — 128 numbers — and nothing else can move. Because the LR touches one tiny tensor we can push it high (5e-2).

Results

Top row — the real subject bags. Bottom row — samples for the learned token V. The token captures the gist of a bag — a rounded body, a handle-ish top — but the strokes are soft and the fidelity is well below the LoRA result on the same 20 images. That is the ceiling of a single vector: it can only steer the frozen model toward shapes it already knows how to render.

The model is untouched — digits 0-9 still render perfectly, because training changed exactly zero of its weights:

Artifact size (outputs/params.txt):

Textual Inversion trains ONE vector: 128 parameters (512 bytes as float32).
Compare: LoRA ~ tens of thousands (project 50);
DreamBooth ~ the entire 367,905-param model (project 51).

The trilogy

Same subject (20 bag images), three methods, three points on the size/fidelity curve:

Method	Trains	Artifact	Fidelity
Textual Inversion (this)	one token vector	~512 bytes	vibe only
LoRA	low-rank residuals	tens of KB	good
DreamBooth	every weight	full model	best, but heaviest

Things to try

• Raise N_SUBJECT or STEPS — a single vector saturates quickly; more data barely helps, which is the whole capacity lesson.
• Learn two tokens instead of one (a tiny table of new rows) and see fidelity climb toward LoRA — the bridge between this method and richer adapters.