Long-Prompt Handling

Key Insight

A text-to-video model is only as good as the text encoder that reads your prompt, and the popular CLIP text encoder was trained on short captions and silently cuts off anything past 77 tokens — so a detailed, paragraph-long prompt loses most of its words before the model ever sees them. This project fine-tunes with T5-XXL prompts (up to 256 tokens), a language transformer built to read long, detailed sentences, and compares its prompt adherence against CLIP-L conditioning. The payoff is concrete: with the longer encoder the model can honor clauses like "a red car behind a blue truck at sunset" that a 77-token encoder would clip or scramble. Many frontier models hedge by feeding both encoders into cross-attention — CLIP for a quick gist of style, T5 for the fine-grained wording.