Skip to main content

Glossary

Terms from all guides in this repository, sorted alphabetically. Each guide's own concepts are included here; see individual guides for deeper context.

(2+1)D​

A way to build a video network cheaply by factorizing a full 3D convolution (which mixes space and time at once) into two smaller steps: first a 2D spatial layer that processes each frame on its own, then a separate 1D temporal layer that mixes information across time at each pixel location. The name reads "2 plus 1": two spatial dimensions handled together, plus one time dimension handled separately. Splitting them this way costs far less compute than a true 3D layer and — crucially — lets you initialize the spatial half from a pretrained image model and add the temporal half fresh (see temporal inflation). The trade-off is that space and time never interact within a single layer, which can miss fast, complex motion that a full spatiotemporal layer would catch.

3D VAE​

A VAE (variational autoencoder — a network that squeezes data into a small code and reconstructs it) built for video, so it compresses along time as well as the two spatial dimensions. A plain image VAE shrinks each frame's height and width; a 3D VAE also merges groups of nearby frames, exploiting the fact that consecutive frames barely differ. Typical ratios are about 4× in time and 8× in each spatial direction, cutting a clip's data by roughly 100× overall (the spatial compression applies to both height and width, so it shrinks 4 × 8 × 8 = 256 times in size, but the number of channels usually grows, e.g., from 3 RGB channels to 8 latent channels, so the final footprint is about 100× smaller). This is what makes modern video diffusion affordable: the diffusion model runs on the small compressed latent grid instead of raw pixels, so it sees maybe 30 latent "frames" where the original clip had 120. Because the same compressor is reused for every clip, the heavy work of learning to reconstruct video is paid once while the VAE is trained, not on every generation.

A100​

A high-performance data center graphics processing unit (GPU) designed by NVIDIA (built on the Ampere architecture) specifically for artificial intelligence training, inference, and scientific computing.

  • Why it matters: Prior to the A100 (released in 2020), training deep learning models was severely bottlenecked by how fast data could be loaded into the processor. The A100 introduced massive High-Bandwidth Memory (HBM2) and specialized third-generation Tensor Cores, establishing it as the global workhorse for training large language models (LLMs).
  • How it works: The A100 features thousands of processing cores working in parallel, but its main speedups come from its Tensor Cores. These units are hardware-hardwired to execute matrix multiplications (the primary math in neural networks) in a single clock cycle. It also supports TF32 (TensorFloat-32), which allows the chip to run standard FP32 calculations at 16-bit speeds without requiring code changes.
  • Analogy: Imagine trying to move a massive pile of bricks. A standard CPU is like a high-speed sports car: it can carry a few bricks incredibly fast, but must make millions of trips. A consumer GPU is like a fleet of delivery vans: they can carry many bricks in parallel. The NVIDIA A100 is like a heavy-duty freight train: it has a massive cargo hold (huge HBM memory bandwidth) and tracks (Tensor Cores) built to move tons of bricks all at once with maximum efficiency.
  • Example: Large scale training of GPT-3 or Llama models is typically distributed across clusters of thousands of A100 GPUs, connected via high-speed InfiniBand networks to act as a single virtual supercomputer.

A2C​

Advantage Actor-Critic — a policy-gradient algorithm that pairs two networks: an actor that picks actions (the policy) and a critic that learns a value function V(s). The critic's estimate serves as the baseline, so each action is weighted by its advantage — how much better it did than the critic expected — which sharply lowers the variance of plain REINFORCE. A2C is the synchronous version of A3C (Asynchronous Advantage Actor-Critic): A3C lets many worker copies of the environment run independently and push updates whenever they finish, while A2C steps all the workers in lockstep and averages their data into one batch before each update — simpler, and it uses the GPU more efficiently. Think of the actor as a student making moves and the critic as a coach who, instead of waiting for the final score, says "that was better than I expected" or "worse" after each move. It is an actor-critic method and a direct ancestor of PPO.

ABA​

Articulated-Body Algorithm (ABA) is an efficient method used in robotics and physics engines to calculate forward dynamics—determining how a robot with connected joints (like a robotic arm or humanoid) will accelerate in response to given joint forces and torques.

  • Why it matters: Standard methods for calculating motion from forces require solving complex matrix equations that scale poorly, taking O(n³) operations for n joints. ABA solves this in O(n) time, meaning its calculation cost scales linearly (a robot with 20 joints takes only twice as long to simulate as one with 10 joints, instead of eight times as long).
  • How it works: It does this by pretending each segment (link) of the robot is temporarily isolated, calculating its individual response to forces, and then passing those forces recursively along the chain from the base to the tip and back.
  • Analogy: A group of people holding hands in a line. If you push the first person, that force travels down the line, causing each person to pull or push their neighbor. To predict how everyone will sway, you could write a massive system of equations for the whole group (slow). Instead, you can calculate the motion person-by-person: first figure out how much each person resists being moved based on their weight, pass this "felt resistance" down the line to the end, and then sweep back from the end to compute exactly how fast each person will actually accelerate.
  • Example: In a physics simulator like MuJoCo, when you apply motor torque to a virtual robot dog's joints, the simulator uses ABA to instantly calculate how its legs will swing and bend in response, allowing the simulation to run in real-time or even faster.

An extension of Dijkstra's algorithm that finds the shortest path in a graph much faster by using a heuristic function h(n). At each step, it chooses the node that minimizes f(n) = g(n) + h(n), where g(n) is the actual cost from the start to the current node, and h(n) is an estimate of the remaining cost to the goal (such as straight-line distance). To guarantee finding the shortest path, the heuristic must be admissible (it never overestimates the real cost). Analogy: If you are navigating a maze, Dijkstra's algorithm is like exploring equally in all directions (north, south, east, west) from the start. A* search is like prioritizing paths that lead toward the general direction of the exit (e.g., if you know the exit is to the east, you explore east first). Example: Grid-based path planning for a mobile robot. In a grid of cells, A* uses the Manhattan distance (grid-aligned steps) or Euclidean distance (straight line) to the goal as a heuristic, allowing it to bypass obstacles without searching in directions that lead away from the goal.

Ablation​

A controlled experiment that changes exactly one factor (a data step, a layer, a hyperparameter) while holding everything else fixed, to measure that factor's true effect.

Acceptance rate​

In speculative decoding, the share of the draft model's guessed tokens that the big target model agrees with and keeps — accepted ÷ proposed. Like a junior writer drafting sentences that the editor either approves or crosses out: the higher the approval rate, the less the editor has to redo and the faster the work goes. Higher acceptance means bigger speedups.

Ackermann steering​

A geometric arrangement of linkages in the steering of a car or other wheeled vehicle designed to solve the problem of wheels on the inside and outside of a turn needing to trace circles of different radii.

  • Why it matters: In car-like robots, when turning, the inner wheels must turn at a sharper angle than the outer wheels because they travel along a tighter curve. Without this linkage, the tires would scrub and slide sideways against the ground, causing high mechanical wear, poor energy efficiency, and inaccurate dead reckoning.
  • How it works: It uses a four-bar linkage (steering geometry) where the steering pivots are angled so that when you turn the steering wheel, the inner wheel turns slightly more than the outer wheel.
  • Analogy: Imagine drawing concentric circles with a compass. The inner circle is smaller and has a tighter bend than the outer circle. The two wheels of an axle are like two pens on the compass: to trace their paths smoothly without dragging, the inner pen must steer at a steeper angle relative to the frame than the outer pen.
  • Example: Delivery robots designed like miniature cars use Ackermann steering to navigate sidewalks smoothly without damaging their tires or slipping on grass.

Action conditioning​

Telling a generative video model not just what scene to show but what just happened to it — you feed in an extra input encoding an action (a button press, a steering angle, a chosen game move) alongside the current frames, and the model predicts the frames that should follow that action. Turning the generic "predict the next frames" task into "predict the next frames given this action" is exactly what turns an ordinary video generator into a world model: the action becomes the knob a user (or a policy) turns to steer what happens next. Like the difference between a film that simply plays and a video game that reacts to the controller in your hands.

Action saturation​

When a continuous-control policy stops producing nuanced actions and instead slams every one of them against the edge of the legal range — full throttle or full reverse, nothing in between. It is the usual end state of a policy whose pre-squash outputs have grown large, because tanh squashing maps any big number to nearly ±1. Saturation is bad for three compounding reasons: the gradient of tanh is almost zero out there, so the policy can barely be corrected (vanishing gradients); the policy has effectively stopped exploring, since every state gets the same extreme action; and the log-probability correction that SAC needs becomes numerically unstable exactly there, which is how a saturated actor turns into a NaN loss.

  • Analogy: A steering wheel turned all the way to its stop. Pull harder and the car does not turn any further — and you have lost the ability to make small corrections, which is what driving mostly consists of.
  • Example: A robot arm policy that has saturated outputs maximum torque on every joint at every timestep. It looks decisive and is useless; the arm flails to its limits instead of reaching.

Activation checkpointing​

A memory-saving trick that throws away the intermediate activations from the forward pass and recomputes them during the backward pass — trading a little extra compute for a lot less memory. Also called gradient checkpointing.

  • Analogy: Imagine solving a long, 10-step math problem where you need to show your work (intermediate steps) to check it later (the backward pass). You could write down all 10 steps on a large sheet of paper, which takes up a lot of space (memory). Instead, to save paper, you only write down step 1, step 4, and step 7 (the "checkpoints"). When you need step 5 to check your work, you start at step 4 and recalculate steps 5 and 6 on a small scratchpad, then throw them away when you're done. You spend a little more time re-doing the math, but you use much less paper!
  • Example: When training a deep model like Llama, storing the activations for all 80 transformer layers would exceed GPU memory (VRAM). By checkpointing the outputs of every 8th layer and discarding the rest, PyTorch only has to store 10 activation layers. During the backward pass, it dynamically re-runs the forward pass for 8 layers at a time using the saved checkpoints. This reduces the activation memory footprint by up to 80% while only adding about 33% extra compute time.

Activations​

The intermediate outputs that flow between the layers of a network — the numbers each layer hands to the next during the forward pass. If weights are the fixed recipe a model learned, activations are the half-finished dish moving down the kitchen line, changing with every new input. Unlike weights, they are not saved after training; they are recomputed fresh each time the model runs on a new input.

Activation range​

The interval between the minimum and maximum values that a layer's activations take on during a forward pass. Knowing this range is critical for quantization because it determines how to map those floating-point values to a smaller range of integers (like 8-bit integers) without clipping large values or losing precision on small ones.

  • Analogy: Imagine you are packing clothes for a trip and need to choose a suitcase size. The activation range is the difference between your smallest item (a pair of socks) and your largest item (a thick winter coat). If you choose a suitcase based on average size, the coat won't fit (clipping/overflow). If you choose a suitcase that is too large, your small socks will get lost in the corners (loss of precision/underflow). You need to measure the exact range of your clothes to pack efficiently.
  • Example: In int8 quantization, if a layer's activations range from −2.0 to 5.0, the activation range is [−2.0, 5.0]. We compute a scaling factor to map this range to the 256 available integer values in int8 ([−128, 127]). In LLMs, different tokens or channels can have wildly different activation ranges (such as outlier channels with very large values), which makes tracking activation ranges crucial.

Actor-critic​

A family of RL algorithms that learns two things at once: a policy (the actor, which chooses actions) and a value function (the critic, which judges how good a state or action is). The actor improves itself using policy gradients; the critic supplies a low-variance baseline so the actor's updates are guided by the advantage — "was this action better than average from here?" — rather than the noisy raw return. It sits between the two pure approaches: value-based methods like Q-learning learn only a critic and act greedily, while pure policy-gradient methods like REINFORCE learn only an actor; actor-critic keeps both and lets them teach each other. Analogy: a chess student (actor) who plays moves and a coach (critic) who, after each move, says whether it was better or worse than expected — the student adjusts from that running feedback instead of waiting for the game to end. A2C, PPO, and SAC are all actor-critic methods.

AdaGN (adaptive group normalization)​

The trick diffusion models use to push a condition — like a class label or the current denoising step — into the network through its normalization layers. First group normalization wipes a group of activations clean to mean 0 and variance 1, erasing their current style; then a tiny layer — a single small linear layer, just one little matrix of weights rather than a deep stack of layers — reads the condition and predicts two numbers, a scale and a shift (scale-and-shift), that re-stretch and re-center those activations. The layer can stay this small because its only job is to translate the condition into those two knobs — not to do any heavy image work — so a lightweight layer is plenty, and being tiny means it costs almost nothing even when one is dropped in at every normalization layer. Picture resetting a photo to neutral brightness and contrast, then letting the label "cat" turn those two knobs to a setting the model learned for cats. It is the group-norm cousin of AdaIN and AdaLN, and it is how a single model can be steered to generate one chosen class on demand. The adaptive part is exactly this: those two knobs are not frozen constants but are re-predicted for whatever condition you hand in, so the layer re-tunes itself for "cat" versus "dog" instead of behaving the same way every time.

AdaLN​

Adaptive layer normalization; the conditioning mechanism in DiT. It is the layer-normalization cousin of AdaGN and AdaIN: a normalization layer first wipes a block's activations to a neutral mean-0, variance-1 state, then a tiny layer reads the condition (usually the current denoising step plus a class label) and predicts a fresh scale-and-shift. It is called adaptive because those scale and shift numbers are not fixed once and reused — they are re-computed for each condition, so the layer adapts itself to whatever you are asking it to generate. See also AdaLN-Zero.

AdaLN-Zero​

The conditioning trick that powers DiT. Start with plain AdaLN (adaptive layer normalization): a normalization layer first wipes a block's activations to a neutral mean-0, variance-1 state, then a tiny layer reads the condition — usually the current denoising step plus a class label — and predicts a fresh scale-and-shift to re-stretch and re-center them, the layer-normalization cousin of AdaGN and AdaIN. The "-Zero" part makes two changes. First, the tiny layer predicts a third number — a gate — that multiplies the whole block's output before it is added back onto the residual stream (a gated path). Second, it initializes the layer that produces scale, shift, and gate so they all start at zero. With the gate at zero, every transformer block contributes nothing at the very start of training — the input slides straight through untouched, exactly like the identity shortcut of a residual connection. As training proceeds the gate gradually lifts off zero, so each block learns to add its effect gently instead of jolting a fragile, freshly-initialized network. Picture adding a new musician to a band but starting them on mute: you slowly turn up their volume from zero as they learn the song, rather than letting them blast over everyone on the first beat. That gentle start is a big part of why deep DiTs train stably.

Adam​

Adaptive Moment Estimation (Adam) is a popular optimization algorithm used to train neural networks by dynamically adjusting the learning rate for each individual parameter (weight) based on how its gradients behave over time.

  • Why it matters: In deep learning, finding the optimal set of weights is like searching for the lowest point in a massive, rugged mountain range (the loss landscape). Standard optimization methods like gradient descent use a single learning rate for all weights, which can cause the training process to get stuck in flat regions or bounce back and forth uncontrollably in steep valleys. Adam solves this by tailoring the learning rate to each individual weight.
  • How it works: Adam maintains two running averages for each parameter at each step t:
    1. First Moment (m_t) — Momentum: A running average of past gradients. This helps the optimizer remember its general direction, accelerating training in consistent directions and rolling past local flat spots (similar to how a heavy ball gains momentum rolling downhill).
    2. Second Moment (v_t) — Variance: A running average of the squared gradients. This tracks how wildly the gradient for a weight is changing. By dividing the update by the square root of v_t, Adam scales down updates for weights with large, erratic gradients (acting as a speed limit to prevent unstable jumps) and scales up updates for weights with small, consistent gradients (speeding up slow-moving parameters). It also applies a bias correction to m_t and v_t to adjust for the fact that these averages start at zero.
  • Analogy: Imagine hiking down a foggy mountain range with a group of friends, where each coordinate (latitude, longitude, altitude) is controlled independently.
    • Standard gradient descent is like blindly taking steps of the same maximum size in whatever direction feels steepest right now.
    • Adding momentum is like sliding on a sled: if you have been sliding east, your velocity carries you east even if the slope momentarily flattens or turns slightly.
    • Adam is like a smart guidance system that monitors how bumpy the terrain is in each direction. If the path east-west has been incredibly rocky with sudden cliffs and drops (high variance), it dynamically reduces your maximum step size in that direction to keep you safe. If the path north-south has been smooth, flat, and consistent, it increases your step size in that direction to get you through the boring stretch faster.
  • Comparison with AdamW: While Adam is powerful, when it is combined with traditional L2 regularization (which penalizes large weights by adding a penalty to the loss function), the dynamic adjustment of the learning rate accidentally distorts the regularization effect. This led to the creation of AdamW, which separates the regularization step from the gradient updates.

AdamW​

An extension of the Adam optimizer that implements decoupled weight decay, ensuring that weight decay regularization behaves correctly and effectively.

  • Why it matters: To prevent overfitting (where a model memorizes training data rather than learning general patterns), we use weight decay to gently shrink the model's weights toward zero during training. In standard stochastic gradient descent (SGD), adding this penalty to the loss function (known as L2 regularization) is mathematically identical to directly decaying the weights by a small percentage at each step. However, in Adam, the optimizer divides the gradient updates by the running variance of the gradients (v_t). If weight decay is added directly to the gradients (as it was in original Adam), the decay term also gets divided by v_t. This causes weights with small or infrequent gradients to be decayed more than weights with large, frequent gradients, which is the opposite of what is intended and ruins the regularization.
  • How it works: AdamW fixes this by decoupling the weight decay update from the gradient-based update. It first computes the standard Adam step based only on the loss gradients, and then subtracts a small percentage of the current weight value directly: w_new = w − (learning_rate · weight_decay · w) − Adam_step This ensures that every weight is decayed proportionally to its size, completely independent of how active its gradient has been.
  • Analogy: Imagine running a bakery and trying to keep your recipes simple (regularization) by taxing complex, exotic ingredients so you don't over-complicate your menu.
    • Under classic Adam, the kitchen manager (the optimizer) dynamically scales ingredient adjustments based on how frequently they are used. When you try to tax ingredients inside the mixing bowl, the manager accidentally scales the tax along with the usage. If you use a tiny pinch of saffron very rarely, the manager scales up the saffron gradient step, which accidentally scales up the saffron tax, causing you to completely eliminate it. Meanwhile, common ingredients like flour don't get taxed enough.
    • Under AdamW, you decouple the tax. The manager handles the mixing bowl adjustments dynamically, but a separate accountant walks in at the end of the day and taxes every ingredient directly based on how much of it is currently in the recipe, regardless of how often it's used. This keeps all ingredients balanced and the recipes simple.
  • Comparison & Usage: AdamW is the industry-standard choice for training modern large language models (LLMs) and transformers. By properly regularizing the weights, AdamW achieves much better generalization performance (accuracy on unseen test data) and more stable training curves compared to the original Adam optimizer.

Adapter​

A small trainable layer slipped into an otherwise frozen pretrained network so it can pick up a new skill — or accept a brand-new input modality — without the cost of retraining the whole thing. The usual design is a bottleneck: squeeze the incoming features down to a tiny dimension, push them through a nonlinearity, expand them back to the original size, and add the result onto the untouched main path, so at the start the adapter outputs almost nothing and only gradually learns its small correction. Like a travel plug adapter that lets your existing appliance work in a foreign socket — the appliance (the big pretrained model) is left exactly as it is, and the cheap little adapter does all the converting. Example: bolt an adapter onto a frozen CLIP image encoder so it can suddenly read depth maps, training only the adapter's handful of weights while millions of frozen ones stay put. LoRA is one popular variety of adapter.

Adaptive​

"Adaptive" means a layer does not use one frozen setting for every input — it adjusts its setting on the fly based on what you ask for. In a plain normalization layer the scale-and-shift knobs are learned once during training and then locked: every single input gets the exact same two numbers, like a radio permanently soldered to one station. The word adaptive flips that. Instead of fixed knobs, a tiny layer reads a condition you hand in — a style code, a class label like "cat," or the current denoising step — and predicts fresh knobs for that specific input, so the same layer re-tunes itself every time. Picture the difference between an old thermostat bolted to 70°F no matter who walks in, and a smart thermostat that reads the room — who is home, the time of day, the weather — and picks a new target temperature on its own. Same machine, but its behavior adapts to the situation. That is exactly what the "Ada" stands for in AdaGN, AdaIN, and AdaLN: the scale and shift are predicted from the condition instead of staying frozen, so the network adapts to whatever you are generating right now.

Accelerometer​

The part of an IMU that measures linear acceleration — how fast its own velocity is changing — along three axes. It does not report speed or position directly; you would have to sum (integrate) its readings over time to get those, which is what makes dead reckoning drift. A quirk worth knowing: a still accelerometer sitting on a desk reads roughly 9.8 m/s² upward, not zero, because it cannot tell the constant pull of gravity apart from an equivalent upward acceleration — so you must subtract gravity before integrating. Analogy: it is like the lurch you feel pressing you back into a car seat when it speeds up; the accelerometer feels that same push and reports its strength.

Adaptive instance normalization (AdaIN)​

A way to push a "style" into a network's features: first normalize a feature map so it has mean 0 and variance 1 (wiping out its current style), then rescale and shift it using two numbers — a scale and a bias — predicted from a style code. Like erasing a drawing down to a plain pencil outline and then re-coloring it from a palette you hand in. StyleGAN applies AdaIN at every layer so a single style code can steer image features at every scale. The adaptive in the name means the scale and bias are not baked-in constants: they change with each style code you provide, so the very same layer re-paints features differently for every style instead of applying one fixed look.

Adaptive Monte Carlo Localization (AMCL)​

A probabilistic localization system for a mobile robot moving in a 2D environment, which implements a particle filter to track the robot's pose against a known map.

  • Why it matters: As a robot moves, small errors in its wheel encoders (odometry) accumulate, making it lose track of its exact location. AMCL corrects this drift by comparing sensor readings (like lidar scans) to a pre-existing map of the environment.
  • How it works: It represents the robot's possible positions with a cloud of "particles" (hypothetical poses). As the robot moves, particles are updated based on odometry. When the robot receives new sensor data, it calculates the likelihood of getting that data from each particle's pose. Particles that match the data well are cloned (resampled), while those that don't are discarded. The "Adaptive" part means it dynamically increases the number of particles when the robot is lost (to search more possibilities) and decreases them when the robot is confident (to save compute).
  • Analogy: Imagine you are dropped in a maze with a flashlight and a map, but you don't know where you started. You guess a hundred different spots you might be in. As you walk, you move all your guesses with you. If a guess says you should be standing next to a wall, but your flashlight shows open space, you throw that guess away. Eventually, all your remaining guesses converge on your actual location.

ADD (Adversarial Diffusion Distillation)​

The recipe behind few-step models like SDXL Turbo: distill a slow multi-step diffusion model into a 1–4-step student, but add a GAN-style discriminator that judges whether the student's quick output looks real. Plain distillation alone makes few-step images blurry, because regressing toward an average washes out detail; the discriminator punishes that blur and forces crisp results. Like training a sprinter to copy a marathoner's route in a fraction of the strides while a sharp-eyed judge rejects any shortcut that looks sloppy — so speed rises without the output going soft. Compared with an LCM (pure consistency distillation), ADD trades a fiddlier training setup for sharper few-step samples.

ADD-S​

See Average Distance of Model Points for Symmetric Objects.

Admission control​

Refusing requests early when capacity is saturated, to protect SLOs for accepted requests

Advantage​

A(s, a) = Q(s, a) − V(s) — how much better than average taking action a in state s is, measured against the baseline value V(s) of simply being in that state. A positive advantage means the action beat expectations and should be made more likely; a negative one means it underperformed. Using the advantage rather than the raw return to weight policy updates is what keeps policy-gradient methods from drowning in variance.

Advantage normalization​

Rescaling a batch of advantage estimates to have mean 0 and standard deviation 1 before they are used in a policy-gradient update. It is detail #7 of the 37 PPO implementation details and one of the few that is load-bearing rather than merely helpful. The justification: an advantage carries information in its sign (was this action better or worse than expected?) and in its size relative to the other actions in the batch — but its absolute scale is an accident of how the environment happens to denominate reward. A game that pays in points scores advantages in the hundreds; the same game rescaled to pay in fractions scores them in thousandths, and with a fixed learning rate the first would take enormous steps and the second none at all. Standardizing the batch removes that accident, which is why one set of hyperparameters can carry PPO across environments whose rewards differ by orders of magnitude. In PPO it is applied per minibatch rather than per rollout — a detail that is usually described as arbitrary, and mostly is.

Advantage-weighted regression​

A way to turn a fixed dataset into an improved policy without ever doing risky value maximization: it is plain behavior cloning (copy the dataset's actions) but each example is weighted by how good its action was, measured by the advantage A(s, a) = Q(s, a) − V(s). Actions that beat the state's average value get a large weight (typically exp(β·A), an exponential that rewards big advantages steeply), mediocre ones get a small weight, so the policy drifts toward the better-than-average actions already in the data while never inventing new ones. This keeps it safe in offline RL, where querying unseen actions is dangerous, and it is the actor update used by IQL and AWAC. Analogy: learning to cook from the recipes you already have, but paying far more attention to the ones that previously turned out delicious.

Aesthetic score​

A single number predicting how visually pleasing a human would find an image — used to judge generators when realism metrics like FID miss the question of "is it beautiful?" You produce it with a small predictor — usually a tiny linear head bolted on top of frozen CLIP image embeddings — that has been fit to a dataset of images people rated on a 1–10 scale (the LAION-Aesthetics predictor is the best-known example). To score a new image you embed it with CLIP and pass that embedding through the trained head; the output approximates the average rating a person would give. Think of a film critic who has watched thousands of movies alongside their audience scores and can now glance at a new one and guess its rating. Because it is a learned proxy for taste, it inherits the biases of whoever did the original rating.

Adversarial suffix​

A short string of tokens — often gibberish to a human — appended to a prompt so that an aligned model produces output it would otherwise refuse. It is the payload a GCG jailbreak searches for: rather than rewording the request, the attacker optimizes a fixed tail of tokens that reliably pushes the model past its safety training. Like a magic password whispered after an ordinary question that makes the guard open the gate. A suffix found against one open model often transfers to models it was never optimized on, which is what makes the attack class dangerous.

Agent​

An LLM placed in a loop so it can plan, choose a tool, act, observe the result, and repeat until a task is finished — turning a one-shot answerer into something that carries out multi-step work, like a worker who keeps taking the next action until the whole job is done.

AI (arithmetic intensity)​

Arithmetic intensity (also called operational intensity) is the ratio of mathematical operations (FLOPs) performed to the number of bytes of data read from or written to main memory (RAM or device memory) during an algorithm's execution:

arithmetic intensity = FLOPs / bytes

It measures how much computational work is done on each unit of data once it has been fetched from memory into the processor.

  • If an algorithm has low arithmetic intensity, it reads a large amount of data but performs very few calculations on it (such as adding two massive lists of numbers). The processor spends most of its time sitting idle, waiting for data to arrive from memory, which makes the operation memory-bound.
  • If an algorithm has high arithmetic intensity, it reads a small amount of data but performs many calculations on it (such as multiplying two large matrices). The processor is kept busy doing math, making the operation compute-bound (or compute-limited).

Analogy: Imagine reading a book.

  • Low arithmetic intensity: You turn the page, read a single word, and then immediately turn the page again. You spend almost all your time physically turning pages (reading bytes) rather than reading and thinking (doing math).
  • High arithmetic intensity: You turn the page and then spend 30 minutes reading and studying the dense text on that single page. You spend almost all your time thinking and analyzing (doing math) compared to the brief moment you spent turning the page (reading bytes).

Alignment (multimodal)​

Making embeddings from different modalities comparable in a shared space

Alignment stack​

The layered sequence of post-training steps that turns a raw base model into a helpful, safe assistant — typically SFT, then a reward model, then RLHF (or DPO). Like the stations on an assembly line, each layer builds on the one below it: the model first learns to follow instructions, then learns what people prefer, then is tuned to actually prefer it. "Alignment" here means getting the model's behavior to match human intent.

All-to-all token routing​

In a Mixture-of-Experts (MoE) model spread across many GPUs, tokens must be sent to the specific GPU that holds the expert they need. "All-to-all" is the massive communication step where every GPU simultaneously sends its tokens to every other GPU and receives tokens in return. Imagine a busy postal sorting center where workers at different tables all throw packages to each other's tables at the exact same time—it requires incredibly fast network connections to prevent a traffic jam.

AllGather​

A collective communication operation where each worker (rank) in a distributed setup starts with its own local data, and everyone sends their data to all other workers. At the end of the operation, every worker holds the exact same concatenated array of all inputs.

Analogy: A group of friends planning a potluck. Each friend brings one dish from their house. They gather at a single table, place all the dishes down, and everyone leaves the table carrying a complete plate containing a portion of every single dish.

Example: In FSDP training, the model's weights are sharded across multiple GPUs. Before performing the forward pass, the GPUs must perform an AllGather to collect and reconstruct the full layer weights so they can compute the layer's output.

AllReduce​

A team operation in distributed computing: every worker (rank) starts with its own array of numbers, and AllReduce adds them all together and hands the same combined result back to everyone. (A tensor here is just a grid of numbers, not a function; "summing tensors" means lining up two equal-shaped grids and adding matching cells — [1,2,3] + [10,20,30] = [11,22,33].) Imagine four friends who each counted part of a crowd: they pool their counts, add them up, and all walk away knowing the same total. In tensor-parallel inference each GPU computes part of a layer, and an AllReduce combines those partial results so every GPU ends up holding the full answer before the next layer runs.

AllToAll​

A collective communication operation in distributed computing where every worker (rank) sends a distinct slice of its data to every other worker, and receives a distinct slice from each of them in return. Unlike AllGather (where everyone receives the exact same combined list), in AllToAll, every worker receives different, customized pieces from its peers.

Analogy: A group of classmates exchanging personalized holiday cards. Each student writes a different, custom card for every single classmate, and then everyone distributes their cards simultaneously. When the exchange is done, every student holds a stack of cards, but each card in their stack came from a different person and has a unique message.

Example: In tensor-parallel transformer layers, or when routing tokens dynamically in Mixture-of-Experts (MoE) architectures across GPUs, an AllToAll collective is used to send specific activation slices or tokens to their target GPUs for processing and retrieve the results.

Aleatoric uncertainty​

The randomness that is genuinely in the thing you are modeling, and which no amount of extra data will remove. A fair coin lands heads half the time; studying it harder will not make the next flip predictable. A dynamics model captures this by predicting a distribution — "the pole will be near here, give or take this much" — rather than a single number. Contrast epistemic uncertainty, which is the model's own ignorance and does shrink with more data. The distinction matters when you plan: aleatoric noise is a fact you must live with, while epistemic doubt is a warning that you have wandered somewhere you have never been and your model's confident-looking prediction may be nonsense.

AlphaZero​

A game-playing agent (DeepMind) that masters board games like chess, shogi, and Go entirely through self-play, with no human game records — given only the rules. It pairs a single neural network (which outputs a policy and a value estimate for a position) with Monte Carlo Tree Search: the search uses the network's suggestions to explore the most promising moves, and the search's improved move choices become training targets for the network, so each side keeps bootstrapping the other. The "Zero" means it starts from zero human knowledge. Analogy: a chess prodigy locked in a room with only the rulebook, who gets world-class purely by playing millions of games against themselves and reviewing each one. Its successor MuZero removes even the need to be told the rules, learning a model of the game instead.

Alt-text​

The short text description attached to an image in a web page's HTML so screen readers can announce it and so the text still shows if the picture fails to load (the "alt" is short for alternative text). Because it sits right next to billions of web images, alt-text is the free, ready-made caption that web-scraped datasets like LAION use as each image's label — which is why it is the raw material the whole multimodal-data pipeline starts from. The catch is that it was written for accessibility, not for training: it is often missing, a bare filename like "IMG_2025.jpg", keyword spam stuffed in for search ranking, or simply unrelated to the picture — which is exactly why pipelines filter it by CLIP score and rewrite it into synthetic captions. Like the one-line label taped to the back of a photo in a shoebox: handy when it is accurate, useless when someone scribbled the wrong date.

AMP​

Automatic Mixed Precision (AMP) is a training technique that accelerates deep learning by automatically switching between different floating-point precisions (usually float32 and float16 or bfloat16) depending on the operation.

  • Why it matters: Neural networks require massive amounts of memory and compute. Storing every number as a highly precise 32-bit float (FP32) is safe but slow. Storing everything in 16-bit (FP16) is much faster and halves memory usage, but can cause training to fail due to mathematical errors (underflow or overflow). AMP provides the best of both worlds: it automatically speed-runs the safe parts in 16-bit while keeping critical calculations in 32-bit to maintain accuracy.
  • How it works: During the forward pass, AMP runs heavy operations like matrix multiplications in 16-bit. However, it keeps a master copy of the model's weights in 32-bit. In the backward pass, it uses a technique called gradient scaling—multiplying the loss by a large number before computing gradients to prevent tiny numbers from rounding down to zero (underflow)—before scaling them back down to update the 32-bit master weights.
  • Analogy: Imagine a math student solving a multi-step homework problem.
    • Doing all the scratchpad arithmetic (like basic addition and multiplication) using rounded, whole numbers (16-bit) is extremely fast.
    • However, when writing the final answer and carrying over fractional remainders to the next major section, they use full, high-precision decimals (32-bit) to prevent rounding errors from ruining the final score.
  • Example: In PyTorch, you can wrap your training loop with torch.amp.autocast() and use torch.cuda.amp.GradScaler. Operations like convolutions and matrix multiplications automatically run in float16 on the GPU's Tensor Cores, while the loss computation and weight updates remain in float32.

AMD Instinct MI300X​

A high-performance data center graphics processing unit (GPU) designed by Advanced Micro Devices (AMD) specifically for large-scale artificial intelligence training and inference workloads. Built on the CDNA 3 architecture, it features a modular design using multiple silicon dies (chiplets) packaged together. The MI300X offers significant competition to NVIDIA's A100 and Hopper chips by packing very large amounts of High-Bandwidth Memory (HBM) capacity and high memory bandwidth, making it particularly suitable for serving massive large language models. It is programmed using AMD's ROCm software platform or via HIP to translate existing CUDA code.

  • Analogy: Imagine a huge cargo truck (MI300X) compared to a slightly smaller one (Hopper). The AMD truck has an exceptionally wide cargo bed (more memory capacity and bandwidth), allowing it to carry very bulky loads of goods (massive model parameters) in a single trip. However, the highway route maps and driver manuals (the software ecosystem) are newer and less familiar than the well-established routes (NVIDIA's CUDA stack), so drivers sometimes have to take extra steps to navigate.
  • Example: Deploying a 70-billion parameter model on a single MI300X GPU for inference, where the model's entire set of weights fits comfortably into its 192 GB memory footprint, avoiding the need to split the model across multiple GPUs and incur communication latency.

AnimateDiff​

A 2023 technique that adds motion to an existing Stable Diffusion image model without retraining it. The trick is a separately trained motion module — a small stack of time-aware (temporal) layers — that you slide in between the frozen image model's blocks: the image model still draws each frame, and the motion module makes consecutive frames move together coherently. Because the module is trained once on generic video and then frozen, you can drop it into almost any community checkpoint (a custom-art-style fine-tune, say) and animate that style for free. It is the most popular concrete instance of temporal inflation packaged as a reusable add-on rather than a full model.

Anomaly detection​

A debugging mode (torch.autograd.set_detect_anomaly(True)) that makes autograd check each operation and raise an error at the exact line that first produces a NaN or infinite gradient.

ASIC​

An Application-Specific Integrated Circuit (ASIC) is a microchip customized for a specific use or application rather than general-purpose computing. In AI, ASICs like Google's TPU or Groq's LPU are designed specifically to run neural network computations (like matrix multiplications) with maximum efficiency.

  • Why it matters: General-purpose processors like CPUs must support a wide range of commands and scheduling tasks, which consumes a lot of physical space and power on the chip. By stripping out all unnecessary logic and building a circuit dedicated only to neural network calculations, an ASIC can achieve far higher throughput, lower latency, and better power efficiency than general-purpose chips.
  • How it works: Instead of loading instructions from memory, decoding them, and routing data through general-purpose registers, an ASIC physically hardwires the math operations into the silicon. For example, a matrix-multiply ASIC might feed numbers directly through a grid of arithmetic units (a systolic array) so that data flows continuously without needing caches.
  • Analogy: Imagine a generic Swiss Army knife (CPU or GPU) that contains a blade, scissors, corkscrew, and screwdriver. It is extremely versatile, but if your only job is to slice thousands of loaves of bread, a dedicated electric bread slicer (an ASIC) will do it much faster, safer, and with less effort, even though it cannot open a bottle of wine.
  • Example: Google's TPU is a custom ASIC that is hardwired to perform massive tensor operations. Because its silicon is specialized for deep learning math, it can train and serve models at scale with lower cost and power usage than general-purpose processors.

Ant​

A MuJoCo continuous-control task: a four-legged "ant" robot (a torso on four two-joint legs, 8 motors in all) that must learn to walk forward without flipping over. Its state reports joint angles and velocities, and the action sets a continuous torque for each of the 8 joints. It is a mid-difficulty rung on the standard MuJoCo ladder — harder than HalfCheetah and Walker2d because the four legs can interfere and tip it, easier than Humanoid. Provided as Ant-v4 in Gymnasium.

AOTInductor​

Ahead-of-Time Inductor — a deployment path built on torch.export that compiles a captured model graph into a standalone shared library (.so) ahead of time, enabling C++-only inference without a Python runtime.

AnyGrasp​

A pretrained, data-driven deep network designed for general 3D 6-DoF pose estimation of grasps from point clouds. Unlike classical methods that need an exact 3D computer model of the object, AnyGrasp takes a 3D point cloud of a scene, detects graspable regions, and outputs coordinate poses where a parallel-jaw gripper can successfully pick up objects, even in cluttered piles.

  • Analogy: Imagine a blindfolded person who is handed an unknown object. Instead of trying to guess exactly what it is, they run their hands over it to find flat, parallel surfaces that their fingers can pinch securely. AnyGrasp does the same visually with point clouds: it skims the 3D surface data to find optimal squeeze points.
  • How it is used: In a robotics pipeline, a depth camera (which generates a depth map) captures the workspace, generates a point cloud, and AnyGrasp identifies candidate grasp poses. The robot's planning then drives the arm to align the gripper to the best-scoring pose to lift the object.

AnyRes​

A way to feed a VLM images of any size and shape instead of squashing every picture to one fixed square. AnyRes splits the image into a grid of tiles at its native aspect ratio, runs the image encoder on each tile separately, and concatenates all the resulting image tokens — usually alongside one extra down-scaled copy of the whole image for global context. Analogy: rather than shrinking a newspaper page until the text is an unreadable blur, you photograph it column by column at full zoom and lay the close-ups side by side. Example: a tall 768×1536 screenshot might be cut into a 1×2 grid of two 768×768 tiles, doubling the tokens but keeping small text legible — which is why AnyRes (used by Qwen2-VL and InternVL2) sharply improves OCR-heavy and dense-chart benchmarks, at the cost of a longer, slower token sequence.

Application​

The specific real-world job a model is being built to do — for example, "answer customer-support questions about our refund policy," "summarize internal engineering tickets," or "write product descriptions in our brand voice." A model that scores high on a generic public benchmark can still flop on your application if the two don't match, the way a chef who aces a fine-dining contest may still be the wrong hire for your taco truck. That mismatch is why teams build a small targeted eval shaped like their application instead of trusting a famous leaderboard number.

6-DoF pose​

The complete description of where a rigid object is and how it is turned, using six numbers: three for position (x, y, z) and three for orientation (e.g., roll, pitch, yaw). "DoF" is degrees of freedom — the count of independent ways the object can move. A coin lying flat on a table has only 3-DoF (slide in x, slide in y, spin), but a coin floating freely in space has the full 6. Knowing an object's 6-DoF pose is what lets a robot line its gripper up to grasp it from the correct direction, rather than just knowing roughly where it is.

6-DoF pose estimation​

The perception task of predicting an object's full 6-DoF pose — position and orientation — from a camera image or a point cloud. It goes a step beyond detection: a detector says "there is a mug, somewhere in this box of pixels," while pose estimation says "the mug is here and tilted this way," which is what a robot actually needs to grab it. Modern systems (e.g. FoundationPose, dense-correspondence networks) learn this from data, and are usually fine-tuned on the specific objects a robot will handle. Accuracy is scored with ADD-S.

Antipodal grasp​

A two-fingered grip that pinches an object at two points lying directly opposite each other, so the two contact forces point straight at each other along a single line through the object. This "straddle it from both sides" geometry is the simplest grasp that can hold an object securely: as long as that line passes through the object and the friction is enough, the fingers cannot squeeze the object out. Analogy: Imagine picking up a slippery round plastic bottle with just two fingers. You instinctively place your thumb on one side and your index finger on the exact opposite side, squeezing directly toward each other. If you placed your fingers at a slant (not directly opposite), the squeeze would push the bottle sideways, causing it to slip out of your hand. Squeezing from directly opposite sides is an antipodal grasp, and it keeps the object perfectly balanced between your fingers. A top-down antipodal grasp restricts the robot so the gripper must come straight down from above, similar to a claw machine at an arcade. This constraint makes finding a grasp much easier: instead of searching in 3D space from all directions, the robot only has to search in 2D to decide where to position the fingers and how to rotate them to align with the sides of the object.

AprilTag​

A fiducial marker: a printed black-and-white square whose blocky pattern encodes a numeric ID, designed so a camera can spot it and read its code reliably even at an angle, in poor light, or partly blurred. Because the tag's printed size and flat square shape are known in advance, detecting its four corners is enough to recover its full 6-DoF pose relative to the camera (via Perspective-n-Point) — which is why AprilTags are the cheapest, most reliable way to give a robot a known reference point, and why developers "tape one on everything" during bring-up. Like a QR code stripped down to the bare minimum so it stays readable as a pose target rather than a data carrier.

Apple Silicon​

A family of system-on-a-chip (SoC) processors designed by Apple Inc. that use the ARM architecture, featured in modern Mac computers, iPads, and iPhones. A defining feature of Apple Silicon is its unified memory architecture, where the central processing unit (CPU), graphics processing unit (GPU), and Apple Neural Engine (ANE) share a single pool of high-speed memory on the same package. This design eliminates the slow step of copying data over a PCIe bus between CPU and GPU memory, making Apple Silicon max-tier chips highly efficient for running large machine learning models locally.

  • Analogy: Imagine a chef (CPU) and a baker (GPU) working in the same kitchen. In a traditional setup, they work in separate rooms and have to carry heavy trays of ingredients back and forth through a narrow hallway (the PCIe bus). In Apple Silicon, they share the exact same prep table (unified memory) and can both work on the ingredients without moving them, which saves massive time and energy.
  • Example: Loading a 30-billion parameter large language model directly into the unified memory of a Mac Studio with an M-series chip, enabling fast local inference (using MLX or llama.cpp) that would normally require expensive, enterprise-grade server GPUs.

Arena​

A way to rank chat models by having them go head-to-head: two models answer the same prompt, a human or LLM judge picks the winner, and many such duels are turned into Elo ratings — the scoring system used for chess players. The public LMSys Chatbot Arena is the best-known example.

argmax​

The "which one is biggest?" operation: given a list of scores it returns the position of the largest one, not the value itself. The name is short for argument of the maximum — in math the "argument" is the input you hand a function, so argmax answers "which input gives the biggest output?" and returns that input's position. If the logits are [1.2, 4.8, 0.3], argmax is 1 — the index of 4.8 — which the model reads as "pick token #1." Like scanning a class's test scores and naming the top student rather than reading out their mark. Greedy decoding is just argmax applied to the logits at every step, so it always makes the same choice and never gambles.

Amdahl's law​

A formula that states the speedup of a program from parallelization is strictly limited by its sequential (serial) fraction, no matter how many processors or GPU cores you add. Analogy: A team of construction workers building a house. If painting the house (a task that can be parallelized among 10 workers) takes 2 hours, but laying the foundation (a task that must be done sequentially by 1 person) takes 10 hours, then even if you hire 100 painters to paint the house instantly, the entire construction will still take at least 10 hours. The sequential bottleneck (the foundation) limits the maximum speedup.

Artifact​

An unwanted distortion that a process adds to a signal — something that was not in the original but appears in the output, usually because detail was thrown away to save space. In lossy audio or image compression (an MP3, a JPEG, or a neural codec), squeezing the data too hard leaves audible smears, metallic ringing, or muffled detail in sound, and blocky squares or halos around edges in images. Like a photocopy of a photocopy — each pass loses fidelity and adds smudges the original never had. Example: encoding music at a very low bitrate can make cymbals sound watery or add a faint "underwater" warble — those are compression artifacts.

Aspect-ratio bucketing​

A training trick for image generators: instead of forcing every training image into one square shape by cropping (which slices off the edges of tall portraits and wide landscapes, so the model never learns to compose anything but squares), you sort images into a handful of "buckets" by their shape — tall, wide, square — and build each batch from a single bucket so every image in it shares one resolution. This is necessary because a batch must be a single tensor of one shape, so images of different sizes cannot share a batch unless they are grouped first. The model then learns to generate at many aspect ratios, not just 1:1. Like a photo lab that sorts prints into 4×6, 5×7, and 8×8 trays before processing, so each tray runs through the machine at its own size instead of every photo being trimmed square. Example: a 1280×720 photo goes in the 16:9 bucket; afterward you can ask for a 16:9 image and get a properly-framed one instead of a cropped square.

ASR (Automatic Speech Recognition)​

Automatic Speech Recognition — the task of turning recorded speech into written text, what your phone does when it transcribes a voice message. A modern ASR model such as Whisper reads a mel spectrogram of the audio and emits the words one piece at a time, just as a person listens and types along. Concrete example: feed it a clip of someone saying "turn on the lights" and it returns the string "turn on the lights". Because the model must map a long, wobbly sound wave onto a short line of text, the hard parts are accents, background noise, and rare words — which is why fine-tuning on a specific domain or language helps so much.

Atari​

A family of simple 1970s–80s arcade and home-console video games (Pong, Breakout, Space Invaders, and dozens more) that has become a standard testing ground in reinforcement learning. Each game offers a small screen of pixels, a handful of joystick actions, and a score, which makes the set a cheap, convenient benchmark for agents and world models — hard enough to be interesting, small enough to train on a single GPU. "The Atari benchmark" usually refers to the Arcade Learning Environment, a software package that runs ~57 of these games behind one common interface.

ATen​

The C++ tensor library underneath PyTorch's Python frontend

Attention​

The core mechanism of the transformer architecture that allows a model to dynamically focus on different parts of the input sequence when processing each word or token. It enables the model to understand the context and relationships between words, even if they are far apart in a sentence.

  • Why it matters: Traditional sequence models (like RNNs) process text word-by-word and tend to forget earlier information in long sentences. Attention solves this by letting every word look at and connect directly with every other word in the sequence, making it the foundation of modern Large Language Models.
  • How it works: Attention calculates a weighted relationship between three representations of the tokens:
    • Queries (Q): What a token is currently looking for.
    • Keys (K): What information a token contains or offers to others.
    • Values (V): The actual content or meaning of the token. The model matches the Query of the current token against the Keys of all tokens to calculate similarity scores. It passes these scores through a softmax function to turn them into weights between 0 and 1, and then multiplies these weights by the Values of the tokens to produce a weighted blend of information. Mathematically, it is represented as: softmax(*Q**K*ᵀ / √*d*) *V* (where d is the key dimension size).
  • Analogy: Imagine you are attending a networking event looking for experts in different fields:
    • Your search criteria (e.g., "I need a Python developer") is the Query.
    • Everyone at the event has a name tag listing their expertise (e.g., "Web Designer", "Python Developer"); these name tags are the Keys.
    • You compare your search criteria (Query) against everyone's name tag (Keys). The person with the matching tag gets your full attention.
    • The actual knowledge and advice they share with you once you start talking is the Value.
  • Example: In the sentence "The bank of the river was muddy, so I did not deposit my money there," the word "bank" appears twice. When processing the first "bank," the attention mechanism links it strongly to "river" and "muddy," helping the model determine it refers to a slope of land. When processing the second "bank" (implied by "deposit" and "money"), attention links those words to resolve the financial context.

Attention sink​

The first few tokens of a sequence, which attention heads keep putting weight on no matter what those tokens actually say. They are called a sink in the plumbing sense — a drain where leftover water collects: on every step the softmax has to spread a full 100% of attention across the tokens, so when a head has nothing important to look at, that spare attention drains into these first tokens. Because the model leans on them, KV cache eviction schemes deliberately keep these tokens even when they look unimportant, which keeps quality stable in long-context serving.

AudioSet​

A large public dataset from Google of about two million 10-second clips taken from YouTube, each tagged with the kinds of sound it contains (dog bark, guitar, rain, speech) drawn from a vocabulary of 527 labels. Think of it as "ImageNet for sound" — a big, broadly labeled collection people use to teach models what everyday audio events sound like. Because each clip comes with short descriptive tags, AudioSet is also a handy source of (audio, caption) pairs for training a model to describe what it hears.

Autoencoder​

A neural network that learns to copy its input to its output through a narrow middle layer. It has two halves: an encoder that squeezes the input down to a small set of numbers, and a decoder that rebuilds the original from those numbers. Because the middle is much smaller than the input, the network cannot simply memorize — it is forced to keep only the most important features, like writing a short summary of a long article and then reconstructing the article from the summary. That small middle representation is called the latent space.

autograd​

In deep learning frameworks like PyTorch, autograd is the automatic differentiation engine that calculates gradients (derivatives) for mathematical expressions. It is the core mechanism that makes training neural networks possible.

When a neural network makes a prediction (the forward pass) and calculates a loss function (measuring how incorrect its prediction was), we must figure out how to adjust every weight in the network to minimize that loss (using gradient descent). This adjustment requires computing the gradient of the loss with respect to each weight. Since modern networks contain millions or billions of weights, calculating these derivatives by hand using calculus (the chain rule) is practically impossible and error-prone.

Autograd automates this entire process:

  1. Forward Pass: As operations (like addition, multiplication, or activation functions) are performed on input tensors, autograd records them, building a directed graph of all the computations.
  2. Backward Pass: When .backward() is called on the loss tensor, autograd traverses this graph in reverse (using reverse-mode automatic differentiation), applying the chain rule of calculus to automatically compute and store the gradients for all the weights.

Analogy: Imagine a complex Rube Goldberg machine.

  • Forward Pass: You roll a marble (the input data) through a sequence of ramps, levers, and gears (the layers of the neural network), resulting in a weight dropping onto a scale (the loss).
  • Autograd's Record: While this was happening, a sensor-laden camera tracked every movement, collision, and rotation.
  • Backward Pass: You want to know: "If I nudge a specific lever at the start, how much will it change the weight's impact on the scale?" Instead of rebuilding the machine and guessing, you play the recording backward. By analyzing the recorded interactions step-by-step in reverse (the backward pass), you can calculate exactly how a small change to any component propagates to the final output.

Automatic temperature tuning​

The trick that lets SAC set its entropy temperature α automatically instead of by hand. In maximum-entropy RL the temperature α weighs the entropy bonus against reward — high α makes the policy act almost randomly, low α makes it collapse to a brittle near-deterministic habit — and SAC is painfully sensitive to its exact value, which differs from task to task. The fix: pick a target entropy (a single number saying how random you want the policy to be on average, often set to roughly −(number of action dimensions)) and treat α as a learnable value adjusted by gradient descent so the policy's actual entropy is pushed toward that target. When the policy is too deterministic, α rises and pays it to explore more; when it is too random, α falls. Analogy: a thermostat for randomness — you set the temperature you want (the target entropy) and the controller (the α update) keeps turning the exploration knob until the room matches. This one change is why SAC can use a single configuration across wildly different robots instead of re-tuning α for each.

Autoregressive model​

A model that generates a sequence one piece at a time, where each new piece is predicted from all the pieces produced so far — like writing a sentence word by word, with every word depending on the words already on the page. The name says what it does: auto means "self" and regression means "predicting a value from earlier values," so the model predicts each new piece by regressing on its own previous outputs — it feeds on itself. For images, an autoregressive model (such as PixelCNN) draws pixel by pixel in a fixed order. This makes the math clean and the samples sharp, but generation is slow because each step has to wait for the previous one, with no way to compute them all at once.

Average Distance of Model Points for Symmetric Objects​

Average Distance of Model Points for Symmetric Objects (ADD-S) is a metric used to evaluate how accurately a computer vision system has estimated the 3D position and orientation (6-DoF pose) of a symmetric object.

  • Why it matters: For objects that look the same from multiple angles (like a cylinder, a sphere, or a blank coffee mug), there are many different rotations that look identical. A standard accuracy metric (like ADD) would penalize a system for predicting a rotated pose, even if the object looks perfectly aligned. ADD-S solves this by accommodating symmetry so the robot isn't penalized for correct, indistinguishable matches.
  • How it works: It takes the 3D points of the object model in the predicted pose and calculates the distance from each point to the nearest point on the model in the true pose, rather than comparing it to its exact original index twin. It then averages these minimum distances. If the average distance is below a threshold (like 10% of the object's size), the prediction is counted as correct.
  • Analogy: Imagine trying to place a completely plain, round dinner plate back onto its marked spot on a table. If you rotate the plate by 90 degrees, it still fits perfectly and looks exactly the same, even though individual microscopic ceramic particles have moved. You shouldn't be penalized for rotating it, because the plate's overall shape is perfectly aligned with the spot.
  • Example: In a robotic warehouse, when a robot arm needs to pick up a cylindrical can of soup, the vision system uses ADD-S to measure how accurately it estimated the can's pose. Because the can is symmetric around its vertical axis, any rotation around that axis is equally good for the gripper to grab it.

AV1​

A modern, royalty-free video codec — the rules for squeezing video into a small file. AV1 compresses noticeably better than older codecs like H.264 — often 30–50% smaller files at the same quality — but it is much slower to decode, so reading AV1 video back into frames costs more CPU time. Analogy: it is like a denser ZIP format that saves disk space but takes longer to unzip. Example: storing 100 clips as AV1 .webm files might use a third of the disk of the same clips as H.264 .mp4, but take several times longer to decode each clip during training.

AVX-512​

Advanced Vector Extensions 512 — a set of SIMD instructions for x86 CPUs that can operate on 512-bit wide vector registers. It allows the CPU to perform mathematical operations (like addition or multiplication) on up to 16 single-precision (32-bit) or 8 double-precision (64-bit) floating-point numbers simultaneously in a single clock cycle. Analogy: A wide-plow snowblower. Instead of shoveling a driveway one scoop at a time (scalar operations), or using a standard small snowblower (older 128-bit or 256-bit SIMD), the wide-plow snowblower clears a massive 512-inch path in a single pass, clearing the snow (processing data) much faster.

AWQ​

Short for Activation-aware Weight Quantization — a post-training quantization (PTQ) method that compresses LLM weights to low-precision formats (like int4) while preserving model accuracy by identifying and protecting the most important weights.

  • Why it matters: Not all weights in a neural network are equally important. AWQ observes the network's activations on a small set of calibration data and finds that a small fraction (around 1%) of the weights process exceptionally large activation values. Quantizing these "salient" weights to 4 bits introduces massive error that degrades model performance. Instead of quantizing everything uniformly, AWQ keeps these salient weights in higher precision (or scales them up to minimize quantization error) while quantizing the remaining 99% of weights to 4 bits.
  • Analogy: Imagine packing a suitcase for a trip. If you compress everything randomly, you might damage fragile items like your glasses or laptop. AWQ is like identifying the fragile 1% of your items and packing them carefully in bubble wrap, while vacuum-sealing the other 99% of your clothes. The suitcase closes easily (is compressed) but your important items remain intact.
  • Example: Serving an LLM using AWQ-quantized 4-bit weights achieves nearly identical text-generation quality (perplexity) to the original float16 version, but runs much faster and uses a fraction of the GPU memory.

Axis-angle​

A way to describe a 3D rotation with one axis (a direction you spin around) and one angle (how far you spin), packed into a single three-number vector whose direction is the axis and whose length is the angle in radians. It is the most compact honest description of a rotation — three numbers for three degrees of freedom, with none of the redundancy of a nine-number rotation matrix — which is exactly why optimizers and learned models prefer it as the variable they adjust. Analogy: to point a friend at a rotation, you say "tilt your head to the left (the axis) by 30 degrees (the angle)"; that one instruction is the axis-angle form. The version used in code is often called a rotation vector: the same idea, where the vector's length directly carries the angle.

Backend​

A device- or library-specific implementation that actually executes an operation's kernel — for example the CPU, CUDA, or MPS backend. The dispatcher routes each call to the correct backend based on the tensor's device and dtype.

Backlash​

The mechanical play, clearance, or "dead zone" that occurs when the direction of motion in a joint or gear system is reversed. Because mating gears must have a tiny gap between their teeth to prevent jamming, there is a brief moment where the motor rotates but the robot joint does not move until the gear teeth touch again.

  • Why it matters: Backlash is a major source of friction and imprecision in physical robot joints, but it is extremely difficult to model accurately in physics simulators. If a control policy is trained in simulation assuming perfect joints, it might fail in the real world (reality gap) because it doesn't account for the lag and wobbliness caused by backlash.
  • Analogy: Imagine steering an old car where you can turn the steering wheel slightly to the left or right before the tires actually start to turn. That loose play in the steering wheel is backlash.
  • Example: In a robotic hand performing in-hand manipulation (like rotating a cube), a joint needs to switch directions rapidly to adjust its grip. If the gears have backlash, there will be a tiny delay every time a finger switches directions, causing the cube to slip unless the control system is robust to these delays.

Bank conflict​

An efficiency bottleneck that occurs when multiple threads in a warp attempt to access different memory addresses within the same bank of shared memory simultaneously. Shared memory is divided into 32 equal-sized memory banks that can be accessed in parallel; however, if two or more threads request data from the same bank at the same time, the hardware must serialize the requests, causing a stall (slowdown).

  • Analogy: Imagine a bank with 32 teller windows (banks) and a line of 32 customers (threads). If each customer goes to a different teller window, all 32 can be served at the same time. But if 4 customers all try to crowd around teller window #5, they must wait in line and be served one after another (serialized), slowing down the process.
  • Example: In a custom matrix multiplication kernel, if thread i loads elements with a stride that causes thread 0 and thread 16 to both land on bank 0, the GPU must run those loads one by one instead of in parallel, resulting in a measurable performance drop in the memory-bound stage.

Backward pass​

The process of going through the network in reverse — from the output back to the first layer — to compute gradients: how much each weight should change to lower the error. It is used only during training, right after each forward pass: the forward pass makes a prediction, the loss measures how wrong it was, and the backward pass traces that error back to assign blame to each weight (using the chain rule). Like a chef tasting a dish that came out too salty and working backwards through the recipe to figure out which step added too much. A model that is only serving answers (inference) never runs the backward pass — that is why serving is cheaper than training.

Base model​

A model fresh out of pretraining that only continues text and has not yet been taught to follow instructions — a brilliant autocomplete, not yet an assistant.

Baseline​

In policy-gradient methods, a number subtracted from the return before it is used to weight an action, in order to cut the variance of the gradient estimate. The standard choice is the value function V(s) — the average return expected from the current state — which turns the raw return into the advantage return − V(s), answering "was this action better or worse than usual here?" instead of "was the whole episode good?". The key fact is that this subtraction lowers variance without introducing bias: because the baseline depends only on the state and not on which action was chosen, it cancels out in expectation, so the average gradient is unchanged while its jitter shrinks. Analogy: judging a student's answer against the class average for that question rather than against zero — you learn far more from "above or below average" than from the raw score, and grading on the curve does not change who actually did well. A baseline that is itself a learned critic is what makes a method actor-critic.

Basin of attraction​

The set of starting states from which a controller (or any dynamical system) actually converges to its goal instead of running away. A controller designed by linearizing about a setpoint — like an LQR balancing an upright pole — is only trustworthy near that point, because the straight-line approximation it relies on drifts further from the truth the farther you get. The basin of attraction is the safe neighborhood where the approximation still holds well enough to pull the system home; start outside it and the controller pushes the wrong way and the system diverges (the pole falls). Analogy: a ball in a valley rolls back to the bottom from anywhere on the inner slopes, but nudge it past the ridge and it rolls down the other side — the inside of the valley is the basin of attraction, and the ridge is its boundary.

Batch​

A small group of examples (sentences, images, prompts) that the model processes together in a single forward pass instead of one at a time. Like a chef who slices a whole basket of onions at once rather than picking up the knife for each onion separately — the GPU pays a fixed startup cost per pass, so doing 32 examples in one shot is far faster than 32 single passes. In training, the batch size sets how many examples contribute to each gradient update; in quantization methods like GPTQ, a small calibration batch of representative inputs is run through the model to estimate which weights matter most. See also Batching, which is the same idea applied to grouping inference requests on a serving stack.

Batching​

Grouping several inference requests so the GPU runs them together in one forward pass instead of one at a time. Like an elevator that waits a moment to gather a few people and carry them up in a single trip rather than going up and down for each person separately — every rider's start is a touch slower, but far more people move per minute. That is the trade-off batching makes: higher throughput (requests finished per second) at a small cost in latency (how long one request waits). Production servers like Triton Inference Server do this grouping for you automatically; see continuous batching for the version that lets riders hop on and off mid-trip.

Bayes' filter​

Bayes' filter is the fundamental mathematical algorithm used in robotics for estimating the hidden state of a system over time from a sequence of noisy control inputs and sensor measurements. It is a recursive algorithm, meaning it calculates the current state estimate (the posterior probability) using only the estimate from the previous step and the latest data. The filter operates in a continuous two-step cycle:

  1. Predict (or Control Update): Uses a motion model to project the previous state estimate forward in time, accounting for the uncertainty of the robot's actions (e.g., wheel slippage). This widens the probability distribution, making the estimate less certain.
  2. Update (or Measurement Update): Uses a sensor model to adjust the prediction based on new sensor measurements, scaling the probability by how likely the sensor readings are given the predicted state. This contracts the probability distribution, making the estimate more certain.

Analogy: Finding a lost dog in a yard. If you last saw the dog in the center of the yard, and you know dogs run around randomly, you predict the dog has probably drifted away from the center, widening your search area (predict step). Then, you hear a bark coming from the bushes near the fence. You combine your prior guess with this noisy sound clue, narrowing your search area to the bushes (update step).

BC​

Behavior Cloning — the simplest way to learn from a dataset of demonstrations: treat it as plain supervised learning where each input is a state and the label is the action the demonstrator took, and train the policy to copy those actions. There is no reward and no planning — it never asks whether an action was good, only which action was taken — so it is a form of imitation learning rather than reward-driven RL. Analogy: learning to drive by memorizing exactly what a good driver did at each moment, without ever being told the goal. Despite its simplicity it is a strong baseline in offline RL: when the dataset comes from an expert, copying it often beats fancier methods, so you always run BC first to know what you are competing against.

A heuristic search algorithm used to find the most likely or highest-scoring sequence of choices (like words in a sentence, or actions in a trajectory) by keeping track of only a fixed number of the best options at each step. Instead of exploring every possible path (which grows exponentially) or just choosing the single best choice at the very next step (which is greedy and short-sighted), beam search maintains a "beam" of the top K candidates (the beam width). At each step, it expands all K candidates to all possible next options, scores them, and then filters back down to the top K overall candidates. Analogy for beam search: Imagine a group of K explorers mapping a maze together. At every intersection, each explorer sends scouts down all available paths. The group gathers the reports, identifies the K most promising overall paths, and the explorers move to those positions, abandoning the other options. This keeps the exploration focused on the best paths without getting overwhelmed by the sheer size of the maze. In offline RL, the Trajectory Transformer uses beam search to plan ahead: it generates several potential action-state sequences and uses beam search to identify and execute the sequence that leads to the highest predicted reward.

Behavior policy​

The policy that actually generated a dataset of experience — the one whose actions you have recorded — as opposed to the target policy you are trying to learn. This distinction is the heart of off-policy and offline RL: you want to improve on the behavior policy using only the data it left behind, but the moment your learned policy prefers an action the behavior policy rarely took, you are in out-of-distribution territory where value estimates are unreliable. Analogy: studying recordings of an average chess player (the behavior policy) to become a grandmaster (the target policy) — you can learn a lot, but only about positions the average player actually reached. In D4RL the random, medium, and expert datasets are simply three behavior policies of different skill.

Belief state​

A probability distribution over which underlying state the agent is probably in, maintained when it cannot observe that state directly — the standard cure for a POMDP. After each action and observation the belief is updated by Bayes' rule ("I saw a wall on my left, so I'm probably in one of these three cells"), and the remarkable fact is that the belief itself satisfies the Markov property even though the raw observations don't: a policy that acts on the belief state can be optimal, while one that acts on the current observation alone generally cannot. Analogy: a ship's navigator in fog does not know the ship's position, but keeps a chart of where it could be, sharpened by every lighthouse glimpse — and steers by the chart, not by the last glimpse. The catch is cost: the belief is a distribution over all states, so exact belief tracking is only tractable in small worlds; deep RL usually approximates it with a recurrent network's hidden state instead.

Bellman equation​

The rule that ties the value of a state to the value of the states it leads to: V(s) = E[r + γV(s')], read as "the worth of being here equals the reward I get now plus the discounted worth of wherever I land next." It is called recursive because a state's value is defined in terms of the same quantity one step later, and a consistency condition because the true value function is the one set of numbers that makes both sides agree in every state at once. Almost every RL algorithm is some way of forcing this equation to hold when you cannot compute the right-hand side exactly. Named after Richard Bellman, who developed this style of recursive optimization in the 1950s.

Bellman operator​

One application of the Bellman equation's right-hand side, used as an update rule: take your current guess of the value function and, for every state, replace it with "reward now plus discounted value of the next state." One such sweep over all states is called a Bellman backup. Repeating backups is the engine of value iteration and policy evaluation, and it is guaranteed to converge because the operator is a contraction mapping — each backup pulls every guess strictly closer to the true answer. Think of it as one round of "update each cell from its neighbors," repeated until nothing changes.

Benchmark​

A fixed, shared test set used to measure and compare models on a task — like a standardized exam everyone sits so scores line up side by side. MMLU tests knowledge and GSM8K tests math; a benchmark is only meaningful while models have not already seen its answers (see contamination).

Best-of-N​

An inference trick that samples N candidate answers to the same prompt and keeps the single one a scorer — usually a reward model or verifier — rates highest, like writing several drafts of an email and sending only the best.

bfloat16​

16-bit float with fp32's exponent range — the modern default for training (also written bf16, BF16)

Bias correction​

An adjustment applied in the Adam family of optimizers to counteract the zero-initialization of moment estimates; without it, early steps would be artificially small

Bias-variance tradeoff​

A way of describing the two different reasons an estimate can be wrong. Bias is being consistently off in the same direction — like a bathroom scale that always reads 2 kg heavy; taking more readings never fixes it. Variance is being noisy — readings that scatter widely, so any single one might land far from the truth even though they average out correctly. In RL this is the Monte Carlo vs temporal-difference choice: a Monte Carlo return is unbiased (it is a real sampled outcome) but high-variance (it depends on every random step of a whole episode), while a one-step TD target is low-variance (only one step's randomness) yet biased (it leans on a current, still-wrong value estimate). Most modern methods accept a little bias to buy a large drop in variance, which is why bootstrapped targets dominate.

Biases​

The smaller, additive group of learned parameters in a layer — the b in y = xW + b. After the weights combine the inputs, each output neuron adds its own bias: a fixed offset that shifts the result up or down no matter what the input was. Like the + b that lets a line y = mx + b sit above or below the origin, or a starting balance in a bank account before any transactions — it gives each neuron a baseline to lean toward.

BigGAN​

A large class-conditional GAN from 2018 that was the first to make GAN-generated images look convincingly realistic at high resolution across the thousand everyday categories of ImageNet (dogs, mushrooms, coffee mugs, and so on). Its recipe was mostly "make everything bigger and steadier": much larger batches, more parameters, and a projection discriminator to feed in the class label cleanly. Like discovering that a decent home cake recipe just needed a bigger oven, more eggs, and a steadier hand to reach bakery quality. Its best-known trick is the truncation trade-off: drawing the input noise closer to the average gives cleaner, more typical images at the cost of variety, so you can dial between "safe and pretty" and "wild and diverse."

Bitrate​

How many bits are used to represent one second of sound (or video) — the data budget per second. A higher bitrate keeps more detail and sounds closer to the original; a lower bitrate saves storage and bandwidth but blurs detail and adds artifacts. For a neural codec it maps directly to how many tokens per second the audio is turned into — fewer tokens, lower bitrate. Like the quality slider when you export a photo: more data per image captures finer detail at the cost of a bigger file. Example: MP3 music is often 128–320 kbps (kilobits per second), while EnCodec can compress speech down to about 1.5 kbps — far smaller, but with more audible loss.

Blackjack​

A simple card-game environment in Gymnasium (Blackjack-v1) used to teach Monte Carlo methods. The state is small and easy to read — your hand's total, whether you hold a usable ace, and the dealer's one visible card — and each game ends quickly in a win, loss, or draw, so you can collect many complete episodes fast. Because the deck's odds are known, the true value of some situations can be worked out exactly, giving you a ground-truth answer to check a sampled estimate against.

Blackwell​

NVIDIA's 2024 GPU architecture (B100, B200, B200 Ultra) and the successor to Hopper. Like swapping a sports car engine for a more powerful one of the same shape, it keeps the same overall design as Hopper but doubles down on low-precision math — better FP8 throughput and brand-new FP4 Tensor Cores — which is what makes it the preferred chip for the largest 2025-era training and serving runs.

Block​

In GPU computing (like CUDA and Triton), a block (or thread block) is a group of threads that execute the same program (or kernel) together on the same processor unit (an SM).

  • Why it matters: A GPU has thousands of individual cores, but they cannot all talk to each other easily. To make coordination manageable, threads are grouped into blocks. Threads within the same block can share extremely fast local memory (shared memory) and synchronize their work, making blocks the basic units of organization for parallel GPU tasks.
  • How it works: When you launch a GPU kernel, you specify a grid of blocks. The GPU scheduler distributes these blocks across the available SMs. Once a block is assigned to an SM, its threads run side-by-side. However, threads in different blocks cannot easily communicate or synchronize with each other, which ensures the GPU can run blocks in any order.
  • Analogy: Imagine a large construction site. A thread is an individual worker. A block is a team of workers assigned to build one specific room of a house. The workers within that team (the block) can easily talk to each other, share a toolbox on their local workbench (shared memory), and coordinate their steps. But they don't coordinate directly with workers building other rooms (other blocks) across the site.
  • Example: In a Triton softmax kernel, a single block is often responsible for computing the softmax function for one entire row of a matrix. The threads within that block load the row's data, cooperate to find the maximum value, compute the sum of exponentials, and normalize the values.

BLOOM​

BigScience Large Open-science Open-access Multilingual Model (BLOOM) is a massive, open-access autoregressive large language model (LLM) launched in 2022 by the BigScience collaboration, designed to be highly multilingual.

  • Why it matters: Most early LLMs (like GPT-2) were trained primarily on English text. When these models process other languages, their tokenizers split non-English words into many tiny pieces (sometimes character-by-character), which increases token counts, costs, and latency for non-English users. BLOOM was trained on 46 natural languages and 13 programming languages, using a massive vocabulary of 250,000 tokens. This allows it to process diverse languages efficiently without a high "language tax."
  • How it works: Instead of relying on a vocabulary optimized only for English, BLOOM's tokenizer (using the byte-level BPE algorithm) was trained on a balanced multilingual corpus. Because its vocabulary is so large, it has dedicated tokens for common words and phrases in many languages (e.g., Arabic, Hindi, Bengali, Spanish). When it reads a Bengali sentence, it can encode it using whole words or large subwords rather than breaking it down into individual bytes.
  • Analogy: Imagine a dictionary designed by a speaker who only knows English. If they see the Spanish word desafortunadamente, they might have to spell it out letter-by-letter because they don't recognize the word or its parts. A multilingual dictionary, however, has the full word or large parts of it (like desafortunadamente or afortunadamente) already defined. BLOOM is like that multilingual dictionary: it has the right "keys" on its keyboard for many different languages, saving time and space.
  • Example: In a tokenizer comparison, encoding a standard Bengali sentence might take 338 tokens using GPT-2's standard tokenizer, but only 35 tokens using BLOOM's tokenizer. For a Bengali user, this means BLOOM is nearly 10× cheaper to query and fits far more information into the model's context window.

BM25 (Best Matching 25)​

A classic keyword-search ranking — short for Best Matching 25 — that scores a document by how often the query's words appear in it, weighting rare words more heavily. Think of it like a librarian scanning pages for your exact search words and ranking pages where those words appear most often (especially unusual words) higher on the list. It is the sparse (exact-word) counterpart to dense embedding search.

Bookkeeping numbers​

The metadata associated with a tensor (such as shape, stride, offset, and dtype) that describes how to interpret the raw, one-dimensional block of memory where the actual data is stored.

  • Why it matters: Modifying a tensor's structure (like transposing or slicing it) by copying all its data is computationally expensive and wastes memory. Instead, deep learning frameworks like PyTorch simply change these bookkeeping numbers. This allows the system to view the same memory in a new way instantly, making operations like reshaping virtually free.
  • How it works: Under the hood, a tensor is split into two parts: the raw data (a continuous 1-D array called the storage buffer) and the bookkeeping numbers. The bookkeeping numbers tell the computer where the first element starts, how many elements are in each dimension (shape), and how many memory slots to jump over to move to the next row or column (stride).
  • Analogy: Imagine a long line of numbered boxes in a warehouse (the storage buffer). Instead of physically rearranging the boxes into a grid when you want a grid layout (which is exhausting), you just write a label on a clipboard (the bookkeeping numbers) saying "read every 3rd box as a new row." You've changed the layout on paper instantly without moving a single box.
  • Example: If you have a 1-D tensor of [1, 2, 3, 4] and you reshape it into a 2x2 matrix, the raw numbers in memory remain 1, 2, 3, 4. PyTorch simply updates the bookkeeping numbers to say the shape is (2, 2) and the stride is (2, 1).

Bootstrapping​

Updating a value estimate using another current estimate rather than waiting for the true, fully observed outcome — you "pull yourself up by your bootstraps" by leaning partly on a guess. In temporal-difference learning the target r + γ V(s′) bootstraps because V(s′) is itself a still-imperfect estimate, not a real measured return. Bootstrapping is what lets an agent learn from a single step instead of a whole episode; it cuts variance but adds bias, and combined with function approximation and off-policy data it forms the deadly triad.

Bootstrapped DQN​

A DQN with several Q-heads, each trained on a different random slice of the agent's experience, used to explore by committing to one head for a whole episode. Because the heads saw different data they disagree about unfamiliar states, and that disagreement is a cheap stand-in for "how unsure am I?" — acting greedily under one randomly chosen head therefore produces a consistent plan built on one plausible opinion, instead of the incoherent per-step jitter of ε-greedy. This is what makes it capable of "deep exploration": following a hunch for a hundred steps to see where it leads. It is a practical approximation of Thompson sampling with neural networks. Analogy: a committee of five forecasters trained on different years of weather data — pick one at random each morning and follow their whole plan for the day, rather than re-rolling the dice at every street corner.

Bottleneck​

The single slowest stage in a pipeline, which caps the overall speed; in training this is often the data loader rather than the model.

bpd (bits per dimension)​

The standard likelihood metric for image models: the average number of bits needed to store each number (dimension) in an image, computed as -log₂ p(x) / D. Think of it as how "surprised" the model is per pixel-value — a better model predicts the data more confidently and so needs fewer bits, like a good compressor that zips a file smaller. A model that thinks all 256 pixel values are equally likely scores exactly 8 bits per dimension, so any real model must come in under 8 to show it learned something.

BPE​

Byte-Pair Encoding — subword tokenization by greedy frequent-pair merges. It starts from raw bytes and repeatedly glues together the neighboring pair that appears most often, building up reusable chunks. For example, on lots of English text BPE notices t and h sit side by side constantly and merges them into th; a later round merges th + e into the. So a common word like the ends up as a single token, while a rarer word like tokenizer is left as familiar pieces such as token + izer. "Greedy" means each round simply takes the single most-frequent merge available, never looking ahead to see whether a different choice would pay off later.

Bradley-Terry​

A simple statistical model for turning pairwise comparisons into scores: if item A has a hidden strength s_A and B has s_B, the probability a judge prefers A is σ(s_A − s_B), where σ is the logistic squashing function that maps any score gap into a probability between 0 and 1 (a big positive gap → near 1, a big negative gap → near 0). Training a reward model, and the math behind DPO, both fit exactly this model to (chosen, rejected) preference pairs — maximizing the probability assigned to the human-preferred answer, which amounts to pushing the chosen item's score above the rejected one's. Named after R. A. Bradley and M. E. Terry, who introduced it in 1952 for ranking from paired contests. Analogy: rating chess players from match results — you never need an absolute "skill number," only a record of who beat whom, and consistent ratings fall out of the wins and losses.

Breakout​

A classic Atari 2600 game where a player controls a paddle to bounce a ball and destroy a wall of bricks. In reinforcement learning, Breakout is a standard benchmark environment used to test an agent's ability to learn from raw pixels and sparse rewards. A famous milestone for DQN on Breakout was its discovery of the "tunneling" strategy — digging a channel through the side of the wall so the ball gets trapped behind the bricks, scoring points automatically.

Broadcast​

A collective communication operation in distributed computing where a single worker (often the master rank, or rank 0) sends its data to all other workers in the job. At the end of the operation, every worker holds an identical copy of the master worker's data.

Analogy: A teacher at the front of a classroom reading a paragraph aloud. The teacher reads the text once, and every student in the room writes it down, so that everyone now holds the exact same piece of information.

Example: At the start of a distributed training run, the master rank broadcasts its initial model weights to all other GPUs to ensure that every GPU begins training from the exact same starting point.

Broadcasting​

A tensor operation trick where a smaller tensor is automatically stretched to match the shape of a larger one without actually copying data in memory. Like painting a stripe down a wall: you only load the stripe pattern once, but apply it everywhere as you roll.

C++ extension​

A custom operation written in C++ (optionally with CUDA), compiled and loaded so it can be called from Python like a built-in PyTorch op.

Camera intrinsics​

The fixed inner properties of a single camera that decide how a 3D point in front of the lens lands on a particular pixel: the focal length (how strongly the lens magnifies, expressed in pixels) and the principal point (the pixel where the optical axis pierces the sensor, usually near the image center). They are packed into a 3×3 matrix K and stay constant no matter where the camera is placed or which way it points — that "where and which way" is the camera's extrinsics, a separate thing. You recover the intrinsics once through camera calibration. Analogy: intrinsics are the camera's built-in prescription glasses — a property of the lens itself — while where you stand and where you face is recorded separately.

C-space (configuration space)​

The abstract space representing all possible positions or configurations a robot can take. For a robot arm with N rotatable joints, its C-space is an N-dimensional space where each point is a list of N joint angles. While obstacles are easy to define in the physical 3D workspace, collision checking is done by mapping the robot's physical shape to C-space, turning physical obstacles into complex shapes called C-obstacles. Analogy: Imagine a game of chess. The physical workspace is the board and the pieces, but the configuration space is the abstract set of all valid coordinate positions of all pieces. Finding a sequence of moves is searching this abstract space of board states rather than thinking about how your hand moves the wooden pieces through the air. Example: A 7-DoF robotic arm. Its C-space has 7 dimensions. A single point in this 7D space represents the entire pose of the arm. By representing the robot as a single point, path planning simplifies from moving a complex 3D shape around obstacles to finding a line for a single point to travel through the free areas of C-space.

c10​

PyTorch's core C++ library (the "core ten[sor]" library)

C51​

The original distributional RL agent (2017), built by bolting a return distribution onto DQN. Instead of predicting one expected value per action, it predicts a probability for each of 51 fixed, evenly spaced return levels called atoms — that count is where the name comes from, and 51 was found to be enough resolution to capture the distribution's shape without spreading the training data too thinly across bins. Each Bellman update shifts the atoms by the reward and discount factor, then projects the shifted probabilities back onto the fixed atom grid, and the network is trained with a cross-entropy loss to match that target. Later distributional agents (QR-DQN, IQN) drop the fixed grid and instead learn the return quantiles directly, which removes the projection step.

Calibration​

Running a few representative batches of data through a model to learn how big its activations typically get — their usual smallest and largest values — before quantizing it. Knowing that range lets static quantization choose one fixed int8 "scale": the conversion factor that maps the real numbers onto the 256 slots an int8 can hold. It is like measuring the tallest guest you expect before setting a doorframe height — check the real range once, then size the fixed scale so almost nothing gets clipped.

Camera calibration​

The one-time procedure that measures a camera's intrinsics (focal length, principal point) and its lens distortion, so that pixels can be turned into accurate 3D directions. You photograph a target with a known pattern — typically a checkerboard of known square size — from many angles, detect its corners in each image, and adjust the camera parameters until the model best reproduces where those corners actually landed (minimizing the reprojection error). Every downstream geometric step (stereo depth, pose estimation, hand-eye calibration) inherits this calibration's accuracy, so it is the foundation laid before any measurement. Analogy: zeroing a kitchen scale before weighing — a quick setup step whose error silently taints everything you measure afterward.

Camera control​

Steering not just what moves in a generated video but where the virtual camera goes — pan, zoom, orbit, dolly — by feeding the model an explicit camera path. The dominant method represents each frame's camera as Plücker coordinates (a six-number description of the ray every pixel looks along) and adds those as an extra input, so the model can keep objects placed consistently as the viewpoint moves. Named systems that do this include CameraCtrl and MotionCtrl. It is one of the control surfaces — alongside the motion score and depth- or pose-conditioning — that turn a raw video generator into a directable tool.

Canny edge detector​

A classic (non-neural) algorithm that reduces a photo to a clean black-and-white map of its outlines — the lines where brightness changes sharply, such as the border between a face and the background. It works by measuring the gradient (how fast pixel brightness changes) at every point, keeping only the local peaks so the edges come out one pixel thin, and then linking those peaks into continuous contours. Picture tracing all the hard boundaries of a photo with a fine pen and throwing away the shading in between. ControlNet uses such an edge map as a conditioning signal: the outline says where shapes must go while the prompt decides what fills them. Named after its inventor, John Canny.

CartPole​

The classic "hello world" control task: a pole is hinged upright on a cart, and the agent pushes the cart left or right to keep the pole from toppling. The entire state is just four numbers — cart position, cart velocity, pole angle, and pole angular velocity — and there are only two actions, which makes it the standard first environment for checking that a new algorithm works at all before scaling up. An episode "succeeds" when the pole stays balanced for a long stretch (e.g. 500 steps). Like balancing a broomstick on your palm: the physics is simple, but you still have to react to which way it is already tipping. Provided as CartPole-v1 in Gymnasium. The same physical system is also the textbook example in classical control, where the push is treated as a continuous force and the balancing law is designed by hand — for instance with an LQR controller built from a linearization about the upright point — rather than learned from reward.

CasADi​

A free open-source software framework for symbolic automatic differentiation and numerical optimization, used heavily to build model predictive controllers and solve trajectory-optimization problems (finding the best path-plus-controls over time). You write your robot's dynamics and cost as ordinary math expressions in Python or C++; CasADi then computes their exact derivatives for you (no hand-derived gradients, no finite-difference error) and feeds the whole problem to a numerical solver such as IPOPT to find the controls that minimize the cost while respecting constraints like torque limits. Analogy: a graphing calculator that not only evaluates your formula but also hands you its slope at every point and then hunts down the formula's best setting — the tedious calculus and search are done for you. The name is short for Computer Algebra System with Automatic Differentiation.

Catastrophic forgetting​

When training a model on new data erases skills it had already learned, because the new gradients overwrite the old weights.

Cascaded diffusion​

A way to generate high-resolution images or video by chaining several diffusion models in sequence rather than asking one model to do everything at once: the first model produces a small, coarse result, and each later model takes that output and adds detail or resolution, conditioned on the previous stage's blurry version. "Cascade" is the picture of water falling down a series of steps — each pool feeds the next. Splitting the work this way lets each model specialize (rough layout at low resolution, fine texture at high resolution) and was the dominant recipe for high-resolution video before latent models; Imagen Video and Make-A-Video both built super-resolution cascades. Modern latent-diffusion systems mostly dropped it because compressing into a small latent space up front already makes full-resolution generation affordable inside a single model.

Causal 3D VAE​

A 3D VAE built so that each frame is encoded using only itself and earlier frames, never later ones. In machine learning, "causal" means "respecting the flow of time" — just as an effect cannot precede its cause, a causal model cannot look into the future to process the present. This is the same "look only backward" rule a causal mask enforces in language models. A plain 3D VAE merges a fixed block of frames together (e.g., always merging 4 frames into 1), so it has no clean way to handle a lone still image (because it expects a full block, a lone image has no later neighboring frames to merge with). The causal version sidesteps this: because the very first frame depends on nothing after it, a single-image input (T=1) compresses to a single latent frame (T'=1), and the one model can encode both still images and full video. This is what lets frontier video systems train a single shared compressor on a mix of images and clips (see joint image-video training) instead of maintaining separate image and video encoders.

Causal mask​

A mask applied to attention scores that hides future positions, so each token can attend only to itself and the tokens before it. In machine learning, "causal" means "respecting the flow of time" — just as an effect cannot precede its cause, a causal model cannot look into the future to process the present.

CausVid​

A method for fast, streaming video generation that distills a slow, high-quality diffusion "teacher" — which denoises a whole clip at once and needs dozens of steps — into a causal autoregressive "student" that emits frames in time order using only a few steps each. "Caus" is short for causal: like the causal mask in a language model, each frame may look only at frames already generated, never future ones, which is what lets the model run indefinitely and start showing output immediately. It is one of the current recipes (alongside Self-Forcing) for real-time and infinite-length video.

CBF​

Control Barrier Function (CBF) is a mathematical tool used in robotics to guarantee physical safety by acting as a runtime safety filter on the control commands sent to a robot.

  • Why it matters: While advanced control policies (like those trained with deep reinforcement learning) are excellent at complex tasks, they can propose actions that lead to collision or damage. A CBF safety filter acts as a shield, monitoring the system state and modifying commands only when they threaten safety, ensuring mathematically guaranteed safe operation.
  • How it works: A safety barrier is defined by a function h(x) that is positive in safe states, zero on the boundary, and negative in unsafe states (like colliding with a wall). The filter enforces a constraint on the time derivative (rate of change) of this function, written as ḣ(x) ≥ −α(h(x)), where α is a scaling function. This inequality states that as the robot approaches the unsafe boundary (where h gets close to zero), its velocity toward the boundary must slow down to zero, ensuring it can never cross the barrier.
  • Analogy: Imagine a student driver steering a car. The student (the learned policy) makes all the steering and acceleration decisions. However, the driving instructor (the CBF safety filter) sits next to them with a dual set of pedals. If the student drives normally, the instructor does nothing. But if the student is about to run into a wall, the instructor steps on the brakes just enough to prevent the crash, overriding the student's input at the very last second.
  • Example: A robotic manipulator arm is controlled by a policy to pick up objects. A CBF filter defines the surface of the table as a barrier. If the policy commands the arm to plunge downward too fast, the CBF filter overrides the joint torques, slowing the arm down so it just grazes the surface instead of crashing into it.

CDNA / RDNA​

AMD's datacenter / consumer GPU architectures

CelebA​

A dataset of about 200,000 photos of celebrity faces, each labeled with attributes such as "smiling," "wearing glasses," or "blond hair." Because every image is a face, it is a favorite for studying generative models of a single, well-defined kind of picture — you can easily judge whether a generated face looks real, and the attribute labels let you check whether the model learned to control features like hair or expression.

Centroidal dynamics​

A simplified representation of a legged or humanoid robot's motion that focuses entirely on the linear and angular momentum of the robot's overall center of mass (CoM).

  • Why it matters: Planning trajectories using a robot's full multi-body dynamics (which involves all the joint angles, link masses, and inertia matrices) is computationally expensive and too slow for real-time control. Centroidal dynamics simplifies the equations by projecting the forces acting on the robot's feet directly to the center of mass, making online planning feasible.
  • How it works: It aggregates the mass of all links into a single point at the center of mass. Instead of tracking how each individual leg segment moves, it only calculates how the forces applied at the contact points (the feet on the ground) affect the overall acceleration, rotation, and balance of the body.
  • Analogy: Imagine trying to balance a broomstick on your hand. You don't need to know the exact molecular layout or every tiny vibration along the stick. You only need to focus on where its center of gravity is and where your hand is applying force at the bottom to keep it upright.
  • Example: Quadruped controllers use centroidal dynamics in their Model Predictive Control (MPC) loops to quickly calculate what contact forces each foot should exert on the ground over the next second to maintain balance while trotting.

CEM (Cross-Entropy Method)​

The Cross-Entropy Method (CEM) is a simple optimization algorithm for finding the input that maximizes some score, used in model-based RL to search for good action sequences during planning. It works in rounds: sample a batch of candidates from a Gaussian distribution, score each one (here, by rolling it through a dynamics model and summing predicted reward), keep the top fraction (the "elites"), refit the Gaussian's mean and variance to those elites, and repeat — so the distribution marches toward the high-scoring region. Analogy: finding the warmest spot in a room by scattering thermometers, walking toward the warmest few, scattering again more tightly, and repeating until they converge. On the shared name: CEM minimizes the cross-entropy between its sampling distribution and an idealized distribution placed on the best samples — the same statistical quantity behind the cross-entropy loss used to train classifiers — but it optimizes which actions to try, not network weights, so the two are relatives rather than the same tool. CEM is the action search inside planners like TD-MPC2 and is a direct upgrade over uniform random shooting.

CFG (classifier-free guidance)​

Classifier-free guidance — the standard inference trick for making a diffusion model follow its prompt more closely. The model is trained to run both with the condition (the prompt or label) and without it; at sampling time you take the difference between the two predictions and amplify it, pushing the output away from "generic" and toward "matches the prompt." Unlike classifier guidance, it needs no separate classifier — the same generator provides both signals — which is why it became universal in text-to-image models. A guidance-scale knob trades diversity for prompt adherence.

CFG fusion​

A diffusion-serving optimization for classifier-free guidance, which normally needs two model passes per denoising step — one conditioned on the prompt, one unconditioned. CFG fusion runs both in a single batched forward pass (stacking them as a batch of two) instead of two separate calls, so the GPU is launched once per step rather than twice. Like cooking two portions in one pan instead of washing up between them — same result, far less overhead.

Chain MDP​

A deliberately bare-bones MDP whose states are arranged in a single line, like beads on a string: from each state the agent can step left or right, and the only reward sits at one far end. Because the agent must take a long, unbroken run of correct steps before it ever sees a payoff, a chain is the standard stress-test for exploration: the longer the chain, the harder it is for undirected methods like ε-greedy to stumble onto the goal. Think of a row of light switches where only flipping the very last one turns on a lamp — random flipping almost never reaches it.

Chain rule​

A calculus principle used to compute the derivative of a composite function by multiplying the derivatives of its parts.

Chameleon​

Meta's family of native multimodal models that treat text and images as one single stream of tokens: pictures are turned into image tokens by a VQ-VAE, mixed in with ordinary text tokens, and a single transformer is trained from scratch over the combined sequence with one plain next-token-prediction objective. This is the early-fusion recipe taken to its extreme — there is no separate vision encoder bolted on, so the model can read and write any interleaving of words and image patches. Analogy: instead of a writer and an illustrator passing a notebook back and forth, one person who was taught from the start to "write" in both words and pictures sketches and types along the same flowing line. Example: handed a recipe that is half text and half photos, Chameleon continues it by emitting the next word or the next patch of an image, whichever comes next; the name nods to the lizard that blends seamlessly into any surroundings — here, any mix of modalities.

Character consistency​

Keeping the same person or object looking like themselves across the many separate shots that make up a long video — the same face, hairstyle, and clothing in shot 5 as in shot 1. The failure mode is called drift: because each shot is generated somewhat independently, small differences pile up until the character slowly morphs into someone else. Fixes pin the identity to a fixed reference — an IP-Adapter that feeds a reference image's appearance into every shot, or a character LoRA fine-tuned on a few pictures of that character. You measure the leftover drift by turning each generated face into an embedding and checking how far it travels from shot to shot. Like a film's continuity supervisor making sure an actor's costume and haircut match between takes shot weeks apart. Related named systems include DreamBooth-Video and ID-Animator.

Chat template​

The structured format (system/user/assistant) the model is fine-tuned on

Checkpoint​

A saved snapshot of a model's weights (and optimizer state) at a point in training, so a run can be resumed or rolled back to it after a failure.

Chinchilla​

The scaling law showing compute-optimal training uses ~20 tokens per parameter

Chirp​

A signal or control input whose frequency changes—usually increasing (an "up-chirp") or decreasing (a "down-chirp")—continuously over a set duration. In control systems and robotics, a chirp signal is often used as an excitation trajectory to feed into system identification. By sweeping through a wide range of frequencies, it excites the physical dynamics of the system (such as inertia and joint friction) across a broad spectrum all at once, rather than requiring separate tests for each individual frequency.

  • Analogy: Imagine trying to find loose parts or rattles in a car. Instead of driving at one speed for an hour, then another speed for another hour, you sit in park and slowly rev the engine from a low idle up to a high RPM. This sweep of frequencies causes the car to vibrate at many different rates, triggering any rattle that responds to any frequency in that range and letting you diagnose the whole car in a single test.
  • Example: During the calibration of a robotic joint, the controller commands the joint to swing back and forth, beginning with slow, wide sways (low frequency) and gradually transitioning into fast, tight vibrations (high frequency). By measuring the input motor torques and the resulting joint motions, engineers can fit parameters like joint friction and link mass to build an accurate model of the robot's physical behavior.

CHOMP​

Covariant Hamiltonian Optimization for Motion Planning (CHOMP) is an optimization-based motion planning algorithm that reformulates path planning as a continuous trajectory optimization problem. It starts with an initial trajectory (which can be in collision) and uses functional gradient descent to optimize it. The cost function has two terms: a smoothness cost (penalizing high velocities and accelerations) and a collision cost (using a Signed Distance Field (SDF) to repel the trajectory from obstacles). Analogy: Imagine a rubber band stretched between a start peg and a goal peg, lying across a table with several obstacles. The rubber band naturally wants to contract to become as short and smooth as possible (smoothness cost). If you also imagine the obstacles are magnetic and repel the rubber band (collision cost), the rubber band will slide and bend around the obstacles until it settles in a smooth, collision-free path. Example: Planning a smooth arm movement to pick up a cup. CHOMP takes a straight line in joint space, which might clip the table, and deforms it away from the table surface using the gradient of the table's SDF, outputting a smooth, collision-free trajectory.

Chunked prefill​

Splitting long prompts across multiple iterations to interleave with decode steps

Chunking​

Splitting documents into smaller passages (often a few hundred tokens each) before indexing them for retrieval, so a search returns a focused snippet instead of a whole book.

CIFAR-10​

A classic dataset of 60,000 tiny 32×32 color photos sorted into 10 everyday categories (airplane, cat, dog, ship, truck, and so on). Because the images are small and the whole set downloads in seconds, it is a go-to "hello world" for image models — big enough to be interesting, small enough to train on a laptop. The name stands for "Canadian Institute For Advanced Research, 10 classes." See also MNIST, its even simpler grayscale cousin.

Class conditioning​

Telling a generative model which category to produce instead of leaving it to chance. You feed the model a label (for example, the digit "7" or the class "cat") alongside its usual input, so at generation time you can ask for exactly that class. Without it, the model draws a random sample from everything it learned; with it, you steer the output — like ordering a specific flavor instead of accepting whatever scoop you are handed.

Classifier guidance​

An early technique for steering a diffusion model toward a chosen class or label: you train a separate image classifier that can read noisy images, then at each denoising step add a nudge in the direction of its gradient — the direction that makes the target class more likely. Like a critic standing over a painter and pointing "more toward a cat" at every brushstroke, it trades a little sample diversity for much stronger adherence to the condition. Its drawback is the extra cost of training and running that dedicated noisy classifier, which classifier-free guidance (CFG) later eliminated by getting the same steering from the generator itself.

Cliff Walking​

A small gridworld environment (Gymnasium's CliffWalking-v0, from Sutton & Barto) built to expose the gap between on-policy and off-policy learning. The agent walks from start to goal along the top of a cliff; every step costs −1, and stepping into the cliff costs −100 and snaps it back to the start. The shortest route hugs the cliff edge, but any random exploratory move there is disastrous — so SARSA, which accounts for its own exploration, learns a cautious path one row up, while Q-learning, which assumes greedy future behavior, learns the risky edge path. Reproducing its two learning curves is Sutton & Barto's Figure 6.5.

CLIP​

Contrastive Language-Image Pretraining — a model that learns to match pictures with the words that describe them. It has two separate encoders: one reads an image, the other reads text, and both map their input into the same shared space of embeddings, so a photo of a dog and the caption "a dog" land near each other while the caption "a bicycle" lands far away. It is trained on hundreds of millions of image–caption pairs scraped from the web with a contrastive objective: pull each true image–caption pair together and push every mismatched pair apart. Think of it as teaching two translators — one who only speaks "image" and one who only speaks "text" — to agree on a common language, so any picture and its description end up pointing at the same spot. Once trained you can measure how well a caption fits an image (a CLIP score), classify images with no extra training by comparing them to label phrases (zero-shot), or extract just the text encoder and feed it into a generator. Why add a text encoder to a generator when CLIP is already trained to match text and images? Because the generator itself doesn't inherently understand words; by borrowing CLIP's text encoder—which has already learned an excellent representation of language—you give the generator a deep understanding of descriptive language so it can follow prompts. It is the text encoder inside early Stable Diffusion.

CLIP-L​

A specific, larger version of the CLIP model. The "L" stands for Large (compared to the standard or Base versions), meaning it has more parameters and can grasp more nuanced relationships between text and images. Because of its stronger understanding, it is often used as the text encoder in powerful generators to ensure they follow complex prompts accurately.

Clipping loss​

A modified objective function in Proximal Policy Optimization (PPO) that prevents the policy from changing too much in a single update step.

  • Why it matters: In policy gradient methods, updating the policy using a noisy batch of data can lead to a disastrously large change, ruining the policy's performance. The clipping loss solves this by removing the incentive to move the policy beyond a safe boundary, ensuring training stability.
  • How it works: PPO calculates the policy ratio (or importance ratio), which is the probability of choosing an action under the new policy divided by the probability under the old policy. It then multiplies this ratio by the advantage (a score of how much better an action was than expected). If the ratio wanders outside a narrow band (typically 1 − ε to 1 + ε, where ε is a small hyperparameter like 0.2), the ratio is "clipped" to the boundary value. The final loss takes the minimum of the unclipped and clipped values, meaning the agent gets no extra credit for updates that push the policy beyond the safe limit.
  • Analogy: Imagine a hiking guide who keeps you on a leash: if you find a nice viewpoint ahead, you can walk toward it, but if you try to leap 10 feet forward at once, the guide pulls the leash to keep your step at a safe 2-foot stride. If you step backward to avoid an obstacle, the guide similarly limits how far you can plunge. The clipping loss is this leash: it allows the policy to adjust, but caps the maximum change in either direction to a safe, narrow band.
  • Example: In the CartPole task, if a cart-pole agent takes an action that keeps the pole balanced and receives a high positive advantage, the optimization will try to make that action more likely. However, if the update attempts to make that action 50% more likely (a policy ratio of 1.5), and ε is 0.2, the clipping loss will clip the ratio at 1.2. The optimizer is not rewarded for pushing the action's probability any higher, keeping the policy update small and stable.

Closed-form​

A solution you can write down and compute directly with a fixed formula, instead of reaching it through many rounds of trial-and-error. Solving 2x = 10 by writing x = 5 is closed-form; nudging x up and down until both sides match is not. In DPO a closed-form objective lets the model learn straight from preference pairs with one training loss, skipping the slow reward model-plus-PPO loop of classic RLHF.

CLS token​

A special extra "summary" token (short for classification) glued to the front of a transformer's input sequence whose only job is to soak up information from all the real tokens, so its final output vector can stand in for the whole input. In a ViT it has no patch of its own — it starts as a learned placeholder and, through attention, gathers a single image-wide description you then hand to a classifier. Like a meeting secretary who owns none of the agenda items but listens to every speaker and writes the one-line summary everyone refers to afterward. (Many models instead average all token outputs — mean pooling — which often works just as well.)

CNN​

Convolutional Neural Network — a neural network built mainly from convolution layers; the standard architecture for image tasks. Instead of staring at the whole picture at once, a CNN slides a small magnifying glass across the image, checking one little patch at a time for simple features — an edge here, a splash of color there. Early layers spot these tiny patterns; deeper layers stitch them into bigger ideas (edges become a whisker, whiskers become a cat). Because the same magnifying glass is reused over every patch, a CNN needs far fewer parameters than a network that wired up every pixel separately — and it can recognize a cat whether it sits in the corner or the center of the photo.

COCO​

Common Objects in Context — a widely used image dataset of roughly 120,000 everyday photos, each paired with five short human-written captions plus labeled object outlines, so it serves as a shared benchmark for both captioning and object detection. Think of it as the field's standard "practice set," the way students everywhere drill the same well-known textbook problems so their results can be compared. In this guide, COCO's image-caption pairs are the convenient small-scale fuel for training toy CLIP, Q-Former, and captioning models — big enough to be realistic, small enough to fit a weekend.

Codebook​

The fixed list of code vectors a VQ-VAE is allowed to use to describe an image — think of it as a numbered paint set, where every patch of the picture must be painted using one of the colors on the palette rather than any color imaginable. The encoder looks at a patch, finds the closest entry in this list, and stores just that entry's index, which is what makes the latent code discrete. A bigger codebook offers more "colors" (finer detail) but is harder to use fully — see codebook collapse.

Codebook collapse​

A failure where a VQ-VAE ends up using only a few entries of its codebook and ignores the rest — like owning a 64-color crayon box but only ever drawing with three. The unused entries are wasted capacity, so the model stores less detail than its codebook size suggests and reconstructions stay blurry. Common fixes are EMA codebook updates, re-initializing dead (never-chosen) entries near popular ones, and k-means warmup. It is the discrete-latent cousin of mode collapse in GANs.

Collate function​

The function a DataLoader uses to combine a list of individual samples into one batched tensor; a custom one can pad variable-length data.

Collective operation​

A communication step that all processes (ranks) in a distributed job perform together — such as AllReduce; if one rank skips it, the others wait forever.

Collision mesh​

Simplified geometry used for collision tests, distinct from visual mesh

Column-wise partitioning​

Splitting a weight matrix along its column (output) dimension so that each GPU holds a vertical slice and computes part of the output independently — the standard first step in Megatron-style tensor parallelism.

Compliance​

How much a robot yields—gives way—when something pushes on it, the opposite of being rigidly stiff. A compliant arm bends out of the way under contact instead of fighting back with unbounded force, which is what makes contact tasks safe and robust. Compliance comes in two forms that are often combined: mechanical compliance built into the hardware (soft fingertips, rubber pads, springs in the joints) and software compliance produced by the controller, most commonly impedance control, which makes a stiff motor behave like a soft spring by sensing displacement and easing off.

Analogy: Catching a raw egg. If you catch it with a rigid, flat frying pan, the egg will instantly smash upon impact. If you catch it with your hand, your hand naturally yields and pulls back slightly to cushion the blow, spreading the force over time so the egg survives.

Example: A robot writing with a pencil. If the robot is rigidly stiff and the table is slightly higher than expected, it will press down too hard and instantly snap the pencil lead. A compliant robot will yield slightly when it feels the table pushing back, allowing it to write smoothly even if the surface is uneven. Similarly, a compliant robot can easily wiggle a key into a misaligned keyhole, whereas a rigid robot would jam.

Compute capability​

An integer version number assigned by NVIDIA to define the hardware features, instruction sets, and numerical formats supported by a specific GPU architecture. It dictates which low-precision formats (like FP8 or FP4) and specialized operations (like hardware Tensor Cores) are available on that chip.

Analogy: A phone's operating system version (e.g., iOS 17). A newer version supports all the features of older versions but introduces new capabilities, APIs, and performance optimizations.

Example: An A100 GPU has compute capability 8.0 (Ampere architecture), which supports BF16 and TF32. An H100 GPU has compute capability 9.0 (Hopper architecture), adding support for FP8 and dynamic scaling in its TransformerEngine. Code targeting compute capability 9.0 features cannot run on an 8.0 GPU.

Computed-torque control​

A control method for robot arms that uses the arm's own inverse dynamics as a feedforward term to cancel its nonlinear physics, so what is left over behaves like a simple, predictable system. Concretely, you take the desired acceleration, add a small PID-style correction for tracking error, and then plug that through the manipulator equation M(q)q̈ + C(q,q̇)q̇ + g(q) to work out exactly the torques that produce it — gravity, inertia, and velocity effects all accounted for in advance. Because the model has already absorbed the hard nonlinear part, each joint then responds like an independent, well-behaved unit that a light feedback gain can steer precisely. It is the natural baseline controller for any torque-controlled arm whose dynamics model is trustworthy. Analogy: an archer who already knows the wind and the arrow's drop and pre-aims for them, so only a tiny last-moment adjustment is needed — versus one who fires straight at the target and fights every gust after the fact.

Concatenation​

The most basic way to fuse two modalities: just stick their feature vectors end to end into one longer vector and hand that to the next layer. If an image embedding has 512 numbers and a text embedding has 512, concatenation glues them into one 1024-number vector — like taping two index cards side by side and reading them as a single wider card. It adds almost no parameters and is a surprisingly strong baseline, but the two streams never actually look at each other the way cross-attention lets them; they only get combined once the next layer mixes the stacked numbers, which is why richer fusion often wins when the task needs the modalities to interact.

Conditional GAN (cGAN)​

A GAN that is told which kind of image to make instead of producing a random one. The class label (for example, the digit "7") is fed to both the generator and the discriminator, so generation becomes class-conditioned — you ask for a category and get it. Like a vending machine where you press a button for the snack you want rather than taking whatever drops. See also projection discriminator, an efficient way to feed the label to the critic.

Conjugate gradient​

An iterative method for solving a linear system A·x = b that never needs the matrix A itself — only the ability to compute the product A·v for a vector v you choose. Each iteration refines the guess along a direction "conjugate" to (roughly: non-overlapping with) all previous ones, so it makes progress without ever revisiting ground it has covered, and it gets a good answer in far fewer iterations than the size of the system.

  • Why it matters in RL: It is what makes TRPO computable. TRPO needs to solve F·x = g, where F is the Fisher information matrix of the policy — for even a small network, a square matrix with millions of entries that would be ruinous to build, store, or invert. Conjugate gradient only ever asks for F·v, and that product can be computed directly by differentiating through a KL divergence twice, at the price of one extra backward pass. A matrix nobody can afford to write down is thereby used as though it were available.
  • Analogy: Finding the bottom of a valley when you are only allowed to ask "if I walk in this direction, how does the slope change?" — never to see a map. Ten well-chosen questions get you close enough.

Connect Four​

A two-player board game where you drop colored discs into a vertical grid and try to line up four in a row — horizontally, vertically, or diagonally — before your opponent does. The "4×4" variant used as a toy RL testbed shrinks the board so the whole game is small enough to learn and debug quickly. Like Tic-Tac-Toe with gravity: pieces fall to the lowest free slot in a column, so you choose a column, not an exact cell. It is a common minimal environment for MuZero- and AlphaZero-style search agents because it has clear win/lose rewards and a tiny action space.

Consistency model​

A diffusion-derived model trained so that every point along a noisy-to-clean denoising path maps directly to the same final clean image — so at sampling time you can jump from pure noise to a finished picture in one (or a handful of) steps instead of the usual dozens. It is built by consistency distillation: a student learns to agree with itself at neighboring noise levels along a teacher's ODE trajectory. Like a winding park path where every bench has a sign pointing straight to the exit — wherever you start, one glance gets you to the end. Trade-off: a huge speedup (1–4 steps vs ~50) for a modest dip in quality. The latent-space version is the LCM.

Constant-velocity model​

Constant-velocity model is a mathematical model of motion used in state estimation and tracking that assumes an object moves in a straight line at a constant speed, unless acted upon by random accelerations (modeled as process noise). In state estimation, the system's state vector includes both the object's position and velocity, and the motion equations project the position forward based on the current velocity while keeping the velocity constant (plus noise) for the next step.

Analogy: A hockey puck sliding across smooth ice. You expect the puck to keep moving in the same direction and at the same speed (constant velocity). However, minor bumps in the ice or air resistance (process noise) will cause tiny, unpredictable changes in its speed and direction over time.

Constitutional AI​

An alignment recipe (introduced by Anthropic) where some or all human preference labels are replaced by an AI judge that grades responses against a written "constitution" — a short list of principles like "be helpful," "refuse to assist with harm," "don't pretend to be human." Like running a debate club with a published rulebook instead of asking the audience to vote: cheaper, more consistent, and easier to update than collecting fresh human labels for every new behavior. The technique is the foundation of RLAIF.

Constrained generation​

A decoding-time technique that masks out any next-token choices that would break a target structure — a regex, a JSON schema, a grammar — so the model is only allowed to pick valid continuations. Like a Mad Libs game whose blanks accept only nouns or only numbers: the writer can be creative inside each blank but cannot break the form. Libraries such as Outlines and sglang are common implementations, and the technique is what makes reliable function calling and tool-using agents possible.

Contamination​

When items from an evaluation benchmark accidentally end up in a model's training data, so its score reflects memorization rather than skill — like a student who studied from a leaked copy of the exam. Also called train-test contamination, it is a leading reason a high benchmark number can mislead.

Content-addressable token mixing​

The routing and retrieval of information between tokens based on their query-key similarity (as in attention) rather than their positions

Context parallelism​

Splitting one very long prompt across several GPUs by sequence position, so each GPU holds and processes a different slice of the tokens. Like handing each of four friends one chapter of the same long book to read at the same time, instead of one person reading all four chapters alone. It is how engines serve 100k–1M-token contexts whose KV cache would never fit on a single GPU.

Context window​

The maximum number of tokens the model can attend over in one forward pass

Continued pretraining​

Taking an already-pretrained model and training it further on a new corpus to add domain knowledge, rather than starting from random weights.

Continuous batching​

A serving trick where the GPU adds new requests into the running batch — and drops finished ones — at every decode step, instead of waiting for the whole batch to finish together. Like a hotel shuttle that can pick up and drop off passengers anywhere along its loop rather than only at the start and end: far fewer empty seats overall, so throughput goes up dramatically. It is the single largest speedup in modern LLM serving and is the default in vLLM and TGI.

Continuous control​

Reinforcement learning where the action is one or more real numbers you dial smoothly — a joint torque, a steering angle, a thruster setting — rather than a choice from a short menu (left/right/jump). This breaks the value-based playbook: Q-learning and DQN pick actions with maxₐ Q(s, a), scanning every action to find the best, which is impossible when there are infinitely many. The standard answer is to learn a policy that outputs an action directly — either a deterministic one (DDPG, TD3) or a probability distribution to sample from (SAC, PPO) — so no exhaustive search is needed. Analogy: discrete control is choosing a dish off a menu; continuous control is seasoning a sauce, where you can add any amount of salt, not just "salt: yes/no." The classic testbeds are the MuJoCo robots like Pendulum, HalfCheetah, and Humanoid.

Continuous integration​

Continuous integration (CI) is a software development practice where developers frequently merge their code changes into a central repository, after which automated builds and tests are run to verify correctness.

  • Why it matters: In large codebase projects, having multiple developers make changes simultaneously can lead to conflicts, broken builds, and hidden bugs. By automating the build and test process, CI ensures that every commit is verified immediately. This catches integration errors, failing tests, and performance regressions early, preventing broken code from accumulating and blocking development.
  • How it works: Whenever a developer commits code or opens a pull request, the CI system (such as GitHub Actions, GitLab CI, or Jenkins) detects the event, spins up a clean container or virtual machine, installs dependencies, builds the project, and runs the entire suite of automated unit and regression tests. The build passes only if all tests succeed and all checks (like linters or formatting) clear.
  • Analogy: Imagine a busy restaurant kitchen. If each chef prepared their ingredients in isolation and only combined them on a plate right before serving the customer, they would frequently find that ingredients didn't match or the dish tasted bad. Continuous integration is like having a sous-chef who constantly tastes small spoonfuls of the sauces and checks the ingredients as they are prepared, ensuring any issues are fixed long before the final plate is assembled.
  • Example: Setting up a GitHub Actions workflow that automatically runs a linter, builds the Docusaurus site, and runs unit tests for custom CUDA/Triton kernels every time code is pushed to the repository.

Contraction mapping​

A function that, every time you apply it, pulls any two inputs closer together by at least a fixed ratio — so repeating it drags everything toward a single unmovable point (its fixed point). This matters in RL because the Bellman operator is a contraction with ratio γ (the discount factor): each backup shrinks the gap between your current estimate and the true value function by a factor of γ, so the error falls like γ, γ², γ³, … and the estimate is guaranteed to converge. Analogy: photocopying a copy of a copy at 90% size each time — the images shrink toward one point no matter what you started with. This single property justifies every iterative value-based algorithm in RL.

ControlNet​

An add-on that gives a frozen diffusion model precise spatial control. It clones the U-Net's encoder into a parallel branch that reads an auxiliary conditioning image — a depth map, a pose skeleton, a Canny edge map, a segmentation map — and feeds that branch's features back into the original network so the output follows the supplied structure. The base model stays untouched (so its quality and prompt-following are preserved) and only the new branch is trained; the connections use zero-convolutions so the branch contributes nothing at first and is learned gradually. Like laying tracing paper with an outline over a painter's canvas: the prompt still chooses colors and texture, but every shape must follow the lines you drew. Adapting ControlNet to video (sometimes called ControlNet-Video) feeds a per-frame conditioning map — e.g. a depth map for every frame — into a video diffusion model; the extra challenge is keeping the control temporally consistent so the result does not flicker frame to frame.

ConvLSTM​

An LSTM that swaps its internal matrix multiplications for convolutions, so it can carry memory across time and keep the 2D spatial layout of each frame instead of flattening it into a single vector. A plain LSTM treats its input as a flat list of numbers, which throws away which pixel sat next to which; a ConvLSTM keeps the grid intact, so a local fact like "this corner is getting brighter" stays local. That makes it a natural fit for future frame prediction, where both what changes and where it changes matter. It was the standard baseline for video prediction before transformers and diffusion took over, and later recurrent variants such as PredRNN refined the same idea with extra memory paths between layers.

Convolution Layers​

These are the foundational building blocks of a Convolutional Neural Network (CNN). Their job is to scan an image and hunt for specific patterns.

Each layer uses a small grid of numbers—called a filter or kernel—that acts like a tiny pattern detector. The network slides this filter systematically, step by step, across the entire image. At every pause, the filter looks exclusively at the small patch of the image directly underneath it, checks how well that patch matches the pattern it is hunting for, and spits out a single "match score." As the filter sweeps over the whole image, it records these scores onto a new, blank grid called a feature map.

Picture a small, transparent stencil painted with red-and-white stripes. You drag this stencil step by step over a crowded "Where's Waldo?" poster:

  • When the stencil is underneath a patch of blue sky or a green tree, the patterns don't match, so it leaves a "0" (a dark mark) on your feature map.
  • But when you slide the stencil directly over Waldo's shirt, the stripes align perfectly, leaving a high score (a bright mark) on your feature map.

By the end of the sweep, your feature map acts as a glowing treasure map, lighting up exactly where Waldo's shirt is located.

In a real network, one filter might hunt for stripes, another for glasses, and another for the curve of a beanie cap. By stacking many of these convolution layers together, the network pieces together simple clues to eventually recognize a complex object like Waldo himself. Because the network reuses the same tiny filter across the entire poster, the process stays incredibly efficient—and ensures that the pattern is found no matter where it is hiding in the picture.

copy​

A tensor that owns its own storage, independent of any source tensor; created by .clone(), or automatically by operations like .contiguous() and reshape when a view is not possible

Coriolis​

The Coriolis effect (often grouped with centrifugal forces) is a velocity-dependent force that acts on moving objects within a rotating system. In robotics, as a robot's joints rotate, the individual links are moving relative to one another in rotating frames of reference, which generates Coriolis forces that pull or push the joints.

  • Why it matters: Because Coriolis forces grow quadratically with velocity (how fast the joints are spinning), they can become very strong when a robot is moving quickly. If a robot controller does not calculate and compensate for Coriolis forces (using the manipulator equation), the robot will veer off its path during fast movements.
  • Analogy: Walking on a spinning carousel. If you try to walk in a straight line from the center to the edge of a spinning playground carousel, you will feel a mysterious force pushing you sideways. To keep walking straight, you have to lean and push back against this force. That sideways push is the Coriolis force.
  • Example: In a two-joint robotic arm, if joint 1 is spinning quickly and joint 2 suddenly extends outward, joint 1 will experience an extra torque (resistance) due to the Coriolis effect. The robot's control system must predict this resistance and apply extra motor torque to keep joint 1 spinning at the correct speed.

Cosine decay​

A learning-rate schedule that, after warmup, lowers the rate along the smooth downward half of a cosine curve until it reaches near zero by the end of training. The step size starts large and eases off gently — like braking smoothly as you coast up to a stop sign instead of slamming the pedal at the last moment — which helps the model settle into a good solution. It is the long-standing default schedule, before newer recipes like WSD.

Cosine similarity​

A score from −1 to +1 for how closely two vectors point in the same direction, ignoring how long they are. You get it by taking the dot product of the two vectors and dividing by both of their lengths — which is the same as first L2-normalizing each vector (rescaling it to length 1 so it sits on the unit sphere) and then taking a plain dot product. Worked example: for a = [3, 4] (length 5) and b = [4, 3] (length 5), the dot product is 3·4 + 4·3 = 24, so cosine similarity is 24 / (5·5) = 0.96 — nearly 1, meaning they point almost the same way. A value of 1 means identical direction, 0 means unrelated (at right angles), and −1 means exactly opposite. Analogy: two people pointing at the night sky — cosine similarity asks only "are your arms aimed at the same star?", not "whose arm is longer." This is the standard way to compare embeddings, because in most models meaning lives in a vector's direction, not its magnitude; it is the score inside CLIP matching and the building block of InfoNCE.

Cost per million tokens​

The standard price unit for running a model in production: how many dollars it costs to generate one million tokens of output. You get it by dividing the hardware's hourly cost by how many tokens it produces per hour — like working out a car's cost per mile from its fuel bill and the distance it covers. Almost every serving optimization, from batching to quantization, is ultimately a way to push this one number down.

Costmap​

A grid-based representation of space where each cell is assigned a numeric value (a "cost") representing how dangerous, difficult, or undesirable it is for a robot to travel through that cell. Unlike a simple binary occupancy grid (which only marks cells as either free or blocked), a costmap uses a range of costs to represent different degrees of traversability—such as a low cost for flat pavement, a medium cost for rough gravel or areas near obstacles, and a maximum cost for solid walls. In navigation, path planners (like A* search) search this grid to find the path that minimizes the total accumulated cost, naturally steering the robot away from obstacles and preferred zones (such as keeping a safe distance from walls via costmap inflation). Analogy: A hiking map where paths are color-coded: green trails are smooth and easy (low cost), yellow areas are steep or rocky (higher cost), and red cliffs are impassable (maximum cost); a smart hiker uses this map to find the easiest route rather than the shortest straight line.

CoT​

Chain of Thought — prompting or training a model to write out its reasoning step by step before giving a final answer, the way a student shows their work on a math problem instead of blurting out just the result.

Count-based exploration​

An intrinsic-motivation method that rewards the agent for visiting unfamiliar states. You keep a counter N(s) of how often each state has been seen and add a bonus of 1/√N(s) to the environment's reward: a brand-new state (count 1) gives a full bonus, while a state seen a thousand times gives almost none. The 1/√N shape is borrowed from statistics — the uncertainty of an average shrinks like one over the square root of the sample count — so the bonus fades at exactly the rate the agent's uncertainty about a state does. It is the simplest and most reliable exploration bonus in small, discrete worlds, but it breaks down with images or continuous states, where no two observations are ever byte-for-byte identical so every raw count stays stuck at one (the fix is to estimate pseudo-counts from a density model). Like a hiker who prefers the trails with the fewest footprints, steering toward wherever the map is still blank.

Covariance​

A measure of how two quantities move together: when one is above its average, does the other tend to be above too (positive covariance), below (negative), or neither (near zero)? Stacked up for many quantities at once it becomes a covariance matrix, which describes the overall shape and spread of a cloud of points — how wide it is in each direction and how tilted. Picture a scatter of darts on a board: the covariance tells you whether the cloud is a tight circle, a wide oval, or a diagonal streak. FID compares the covariances of real and generated image features to check that the two clouds have the same shape, not just the same center.

Covariate shift​

A specific type of distribution shift where the input data distribution P(x) changes between training and testing, but the underlying conditional probability P(y|x) (the true labeling function) remains constant. In behavior cloning, this occurs because a small execution error by the policy leads to a new physical state s that lies outside the training dataset, changing the distribution of states the robot visits (P(s) drifts) even though the correct action for any given state (P(a|s)) is unchanged.

  • Why it matters: Because behavior cloning is trained on a static expert dataset, the policy never learns how to correct for drift. When a small error shifts the robot off the expert's path, the policy encounters unfamiliar states, makes larger errors, and eventually experiences catastrophic failure.
  • Analogy: Imagine trying to walk along a tightrope. An expert demonstration shows you perfectly balanced in the middle of the rope. If you step slightly to the left, you are now in a state you never saw the expert in. Since you do not know the correct action to recover from leaning left, you continue to fall further left.
  • Example: A self-driving car policy is trained only on perfect human driving down the center of the lane. If a gust of wind pushes the car near the lane marker, this new state is outside the training distribution (covariate shift), and the model fails to steer back. DAgger solves this by collecting expert labels on these drifted states.

CPU​

Central Processing Unit — the primary processor of a computer, designed as a general-purpose brain. It consists of a few highly powerful cores optimized for low latency on irregular, sequential tasks. It uses deep pipelines, large caches, branch prediction, and out-of-order execution to run single-threaded programs as fast as possible. Analogy: A master chef who can handle complex, delicate, and changing recipes one step at a time. The chef is extremely fast and smart at making decisions (like deciding what to cook next based on what ingredients are left), but can only work on a few dishes at once.

CQL​

Conservative Q-Learning — an offline RL method that adds one extra penalty term to the standard Q-learning loss to cure the out-of-distribution value blow-up. The penalty pushes the predicted value down for actions the network is tempted to overrate and pulls it up only for the actions actually present in the dataset, so the learned Q is deliberately pessimistic about anything unfamiliar. With unseen actions no longer looking artificially attractive, the greedy policy stays near the behavior policy's data and avoids the fantasy high-value actions that make naive offline Q-learning collapse. Analogy: a cautious appraiser who marks down any house they have not personally inspected, so you never overpay for one sight-unseen. CQL is the value-pessimism branch of offline RL; IQL is the simpler modern alternative, and keeping the policy close to the data instead is policy constraint.

Credit assignment​

The problem of working out which of the many actions an agent took deserves the blame or the praise for an outcome that arrived much later. An agent that lands a spacecraft fires its thrusters hundreds of times; the single scalar "you landed successfully" has to be distributed back over all of them, and nothing in the reward signal says which firings were the good ones.

  • Why it matters: It is arguably the problem of reinforcement learning, and most of the algorithmic machinery exists to attack it. Reward-to-go is a credit-assignment fix (an action cannot have caused a reward that preceded it, so do not credit it with one). The baseline and advantage are credit-assignment fixes (judge an action against what was expected in that state, not against the whole episode). GAE is a knob controlling how far into the future credit is allowed to propagate before a learned estimate takes over.
  • Analogy: A company has a profitable year. Which of the ten thousand decisions made by its staff caused it? The annual profit figure is a real signal, and a nearly useless one for telling any individual employee whether their own call was right.

Cross-attention​

A form of attention that lets one stream of data look at and pull in information from a different source. In ordinary self-attention a sequence attends to itself; in cross-attention the queries come from one place (say, the image being denoised) while the keys and values come from another (say, the text prompt's embeddings) — often a different modality entirely. Picture a painter who keeps glancing at a written description while working: each patch of canvas asks the words "which of you matters to me?" and pulls in the answer to decide what to paint. This is exactly how diffusion models inject a text prompt into the image: inside the U-Net the image patches are the queries and the text tokens are the keys and values, so every region of the picture can attend to the words most relevant to it.

Cross-embodiment​

The ability of a robotic control policy to be trained on data collected from different physical robots (different "embodiments") and successfully transfer its capabilities to a new target robot, even if the robots differ in their geometry, joint configurations, actuators, or sensors.

  • Why it matters: Collecting high-quality robot demonstration data is slow and expensive. Cross-embodiment allows researchers to pool data from hundreds of different robots worldwide, creating a large-scale dataset to train a single foundational policy that can then be deployed on a new robot with very little fine-tuning.
  • Analogy: Imagine a human who knows how to open doors. If they are handed a new tool, or if they have to use their non-dominant hand, or even if they have to open a door using their foot, they can still figure it out because they understand the concept of door opening, rather than just memorizing a specific sequence of muscle movements. Cross-embodiment aims to give robots this same general conceptual understanding.
  • Example: A policy trained on manipulation data from a Franka Emika arm with a parallel gripper can be deployed on a UR5 arm with a three-finger gripper, using a shared vision encoder and a translation layer that maps the policy's general spatial commands to the specific joint torques of the UR5.

Cross-encoder​

A model that reads a query and one candidate document together in a single pass and outputs one relevance score — far more accurate than comparing their separate embeddings, but too slow to run over a whole corpus, so it is used to rerank a short candidate list.

Cross-entropy​

A loss function that scores how surprised a model is by the correct answer: it stays small when the model gave the true next word a high probability and grows large when it was confidently wrong. Like grading a weather forecaster on confidence and not just on being right — announcing "90% chance of sun" and then getting rain costs far more points than a hedged "50%." Training an LLM means adjusting the weights to push this surprise as low as it will go.

Cross-Entropy Method Model Predictive Control (CEM-MPC)​

An agent control system that plans its next action by combining Model Predictive Control with the Cross-Entropy Method. Instead of picking actions randomly or using a fixed policy, the agent uses a learned dynamics model to simulate several steps into the future. It uses CEM to iteratively search for the best sequence of actions by sampling a batch of options, selecting the top-performing sequences (the "elites"), and narrowing its sampling distribution around them. Once it finds the best simulated sequence, it executes only the first action, observes the new environment state, and repeats the entire search process for the next step. Analogy: Imagine playing mini-golf with a physics simulator on your phone. Before taking a shot, you simulate 100 random swings. You take the 10 best-looking swings, adjust your aim to be close to them, and simulate 100 more. After a few rounds of narrowing it down, you execute just the first swing of the best path. Wherever the ball actually lands, you pull out your simulator and repeat the entire process for the next shot.

Curse of dimensionality​

The fact that a space gets emptier astonishingly fast as you add dimensions to it, so any method that works by covering the space with samples collapses. Concretely: suppose you need to get each of d numbers roughly right, with a 1-in-3 chance of guessing each one well. For d = 5 that is (1/3)^5 ≈ 1 in 243 — a few hundred random guesses will turn up a good one. For d = 60 it is (1/3)^60 ≈ 1 in 10^28, and you could guess until the sun burns out without success. The cost grew exponentially in d, not linearly. This is why random shooting works fine planning a 1-joint Pendulum and is hopeless planning a 20-joint humanoid, and why iterative searches like CEM — which never have to hit the target in one throw, but walk toward it — take over as soon as the action space grows.

Cross-modal retrieval​

Searching with one modality to find matches in another — typing a caption to pull up the right photo, or handing in an image to find the text that describes it. It works by mapping both modalities into one shared space (for instance with CLIP), so that a query and its true match land near each other; you then keep the few stored items with the highest cosine similarity to the query (a top-k nearest-neighbor lookup). Because every item is encoded just once, answering a query is only a batch of dot products — one matmul — which is why it scales to huge collections. Analogy: a library where books and their summaries are shelved by meaning instead of by title, so a summary in your hand leads you straight to the shelf holding the matching book. It is the first of the four canonical multimodal tasks and the thing dual encoders are best at.

Cross product​

A way to combine two 3D vectors that — unlike the dot product, which boils them down to a single number — returns a third vector. That new vector points at a right angle (perpendicular) to both of the originals. For a = [a₁, a₂, a₃] and b = [b₁, b₂, b₃] it is computed slot by slot as a × b = [a₂·b₃ − a₃·b₂, a₃·b₁ − a₁·b₃, a₁·b₂ − a₂·b₁]. For example, [1, 0, 0] × [0, 1, 0] = [0, 0, 1]: two arrows lying flat on a table (one pointing "east", one "north") produce one pointing straight up, out of the table.

What it does (the effect). Where the dot product measures how aligned two vectors are, the cross product hands you the axis they are not aligned along. Analogy: Imagine laying two pens flat on a desk so their ends touch, forming a "V" shape. Now, imagine taking a pencil and standing it perfectly upright exactly where the two pens meet, pointing directly at the ceiling. That standing pencil is the cross product. Furthermore, the length of that pencil depends on how wide you open the "V" shape. If you open the pens to a perfect 90-degree corner, the pencil grows to its maximum height. If you close the pens together so they overlap and point the same way, the pencil vanishes entirely (its length shrinks to zero).

Why Plücker coordinates use it. If you just say "a line pointing North," you haven't given enough information to pin it down — imagine two parallel train tracks that both point North, but sit in completely different places. You need a way to tell them apart.

This is where the cross product comes in. By taking the cross product of a position vector (pointing to any spot on the track) and the track's direction, you create a new vector called the line's moment. Think of this moment as a unique fingerprint for the line's exact location in space. The magic of this math is that no matter which spot you pick along that specific track, the cross product always spits out the exact same fingerprint. But if you do the math on the other parallel track, you get a completely different fingerprint. So, by keeping just two things — the direction and this fingerprint — you perfectly lock down exactly which line you are talking about.

cuBLAS​

NVIDIA's highly optimized library of dense linear-algebra kernels; PyTorch calls it under the hood to perform fast matrix multiplication on CUDA-enabled GPUs. Analogy: A specialized calculator programmed with the fastest possible shortcuts for doing matrix multiplication. Example: Calling torch.matmul(A, B) on a CUDA tensor uses cuBLAS, bypassing the need for developers to write custom CUDA kernels for standard matrix operations.

CUDA​

Compute Unified Device Architecture (CUDA) is a software platform and programming model created by NVIDIA that allows developers to run general-purpose calculations on NVIDIA graphics cards (GPUs).

  • Why it matters: Historically, GPUs were designed only to render 3D graphics for video games. CUDA unlocked their massive computational power for general math and AI. Instead of tricking the graphics card into treating math as pixels, developers can write C-like code to run code directly on the GPU.
  • Analogy: Imagine a massive warehouse filled with thousands of assembly-line workers (GPU cores). Without CUDA, you can only send instructions to them by drawing pictures (the old graphics language). CUDA is like a translation system that lets you write standard instruction lists in a language similar to C++ or Python, which are then distributed to all the workers to execute in parallel.
  • Example: In PyTorch, moving a tensor to the GPU with x = x.to('cuda') registers the data with the CUDA driver. When you run x + 1, PyTorch uses CUDA under the hood to launch thousands of tiny addition tasks—one for each number in the tensor—across the GPU's hardware.

CUDA core​

The basic arithmetic worker inside an NVIDIA GPU — a tiny unit that performs one simple math operation (such as a single multiply-add) for one thread at a time. A modern GPU has thousands of them, bundled into SMs, and they all crunch numbers side by side, which is what makes a GPU so fast at the massively repetitive math behind deep learning. Think of one CUDA core as a single cashier: not especially fast on its own, but put thousands of them in one store and they ring up a huge crowd in the time a single super-fast cashier would clear one line. CUDA cores handle everyday general-purpose (FP32) math, while the heaviest matrix multiplications are handed off to the specialized Tensor Cores that sit right alongside them.

CUDA Graphs​

A way to record a whole sequence of GPU kernel launches once and then replay them all with a single command, instead of telling the GPU what to do step by step every time. Like pressing "play" on a saved macro instead of retyping the same keystrokes — it skips the per-launch bookkeeping. What has to stay fixed is the list of steps, not the data they run on: every decode step runs the exact same kernels in the exact same order, just on a different token, so it can be recorded once and replayed each step while the actual tokens keep changing. (It only stops helping if the steps themselves change — say a different model path on every call.) This saves a noticeable 5–20% on small models, where launching dozens of tiny kernels per token is itself a real cost.

CUDA stream​

A queue of GPU work that runs in order, but independently of other streams — so the GPU can be doing one stream's job while the CPU prepares the next, or two streams can overlap. Like separate checkout lanes at a store: putting independent tasks in different lanes lets them progress at the same time instead of waiting in one long line, which is how a serving stack overlaps detokenization or KV transfer with the next forward pass.

Curiosity-driven exploration​

An intrinsic-motivation approach in reinforcement learning where the agent generates its own reward signal based on how surprised it is by the outcomes of its actions. Instead of only chasing external rewards from the environment (which might be very sparse), the agent is driven to seek out states or transitions where its internal predictive model has a high prediction error. Analogy: A scientist who spends their life investigating unexpected experimental results is driven by curiosity. They don't just repeat experiments with known outcomes; they seek out the ones they cannot yet predict, using those surprises to build a better model of the universe. Why it matters: It helps agents learn to navigate complex environments when success is rare (e.g. Montezuma's Revenge), but it can fail when confronted with unlearnable, random noise (the noisy-TV problem), prompting methods like ICM to predict surprise only in a controllable feature space.

Custom op​

A user-defined operation registered with PyTorch (e.g. via torch.library.custom_op) so it behaves like a built-in operator. Registering a custom operator is essential when writing custom CUDA or Triton kernels because it ensures that PyTorch's auto-differentiation and compiler engines can trace the operation without breaking. Analogy: Adding a new recipe directly to a restaurant's standard kitchen menu. Instead of having to explain the unique recipe to the cooks every time, registering it as a custom op allows the kitchen's automated system to track, order, and cook it just like any other standard dish. Example: Registering a custom Triton softmax kernel with a custom backward formula ensures that torch.compile can compile the entire forward and backward training pass without falling back to slow eager mode execution.

CUTLASS​

NVIDIA's open template library for matmul kernels

DAgger (Dataset Aggregation)​

Dataset Aggregation (DAgger) is an online imitation learning algorithm designed to solve the covariate-shift problem in behavior cloning. Instead of training purely on a static dataset of expert demonstrations, DAgger dynamically collects new data by running the current policy in the environment, querying an expert to provide the correct actions for the states the policy actually visited, aggregating this new data into the training set, and retraining the policy.

  • Why it matters: In standard behavior cloning, a policy is trained only on states visited by an expert. At test time, any tiny execution error shifts the robot into unfamiliar states where it does not know how to recover, leading to compounding errors. DAgger forces the expert to label these recovery states so the policy learns how to steer back onto the correct path.
  • Analogy: Imagine learning to drive a car with an instructor. Behavior cloning is like watching a video of the instructor driving perfectly. If you take the wheel and drift slightly off the road, you panic because you never saw the instructor in that position. DAgger is like you actually driving while the instructor sits in the passenger seat: whenever you steer off-course, the instructor yells the correct steering adjustment, teaching you how to recover.
  • Example: In a robotic manipulation task, if a policy-controlled gripper misses an object and slips to the side, DAgger queries the expert policy to label the correction (e.g., "move left 5cm") from that failed state, preventing the gripper from drifting further away.

DALL·E 3​

OpenAI's text-to-image model, best known for following long, detailed prompts faithfully — it reliably places the right objects, counts, and spatial relationships you asked for. Its standout trick was training on synthetic captions: instead of messy web alt-text, the team rewrote the training captions to richly describe each image, so the model learned exactly which words map to which pictures. Like a student who finally aces reading comprehension once their textbook is rewritten in clear, complete sentences. It is the proprietary counterpart to open models like Stable Diffusion, and the public demonstration that better captions can beat a bigger model.

Damped least-squares​

A robust way to invert a matrix that may be near-singular (close to having no clean inverse), used in inverse kinematics to turn a desired hand motion into joint motions. A plain inverse of the Jacobian blows up near a singularity — it asks for near-infinite joint speed to make a tiny hand motion — so damped least-squares adds a small number λ² along the diagonal before inverting, which caps how large the solution can grow. The cost is a little accuracy: the hand tracks slightly behind its target, but the joints move at sane speeds instead of exploding. It is the same idea as L2-regularized least squares (ridge regression) in statistics — adding a small penalty to keep an ill-posed answer bounded. The resulting step, Jᵀ(JJᵀ + λ²I)⁻¹ e, is also known as the Levenberg-Marquardt update.

Damper​

A mechanical device or virtual controller component that resists motion and absorbs energy in proportion to velocity (how fast something is moving). It acts as a brake on speed, damping out oscillations (vibrations) to prevent bouncing.

  • How it works: While a spring pushes back based on distance (displacement), a damper pushes back based on speed (velocity). The faster you try to push it, the harder it resists; when you stop moving, it stops resisting.
  • Analogy: A hydraulic screen door closer. If you try to swing the door shut very quickly, the cylinder resists heavily and slows it down. If you push it very slowly, it offers almost no resistance, letting it close gently.
  • Example: Car shock absorbers are physical dampers that stop the car from bouncing repeatedly after hitting a pothole. In robotics, a virtual damper is programmed into a controller (such as in impedance control) to smooth out joint movements and prevent the arm from shaking or oscillating.

Data parallelism​

The default way to train across many GPUs: put a full copy of the model on each GPU, feed each one a different slice of the batch, then average their gradients so all copies stay identical — like several graders each marking part of an exam pile and then pooling the scores. (See DDP.)

DataLoader​

PyTorch's iterator that pulls samples from a Dataset, groups them into batches, and can load them in parallel using worker processes.

Data-wrangling​

The process of cleaning, transforming, and organizing raw, messy data into a structured format suitable for training machine learning models or performing analysis. Analogy: Imagine buying raw, unwashed vegetables straight from a farm. You cannot just throw them whole into a cooking pot. You have to wash off the dirt, peel the skins, chop them into even pieces, and sort them. Data-wrangling is this preparation step — taking raw inputs and getting them ready to be "cooked" (trained on). Example: In reinforcement learning, data-wrangling involves collecting the raw transitions (states, actions, rewards) from gameplay, aligning them into tidy batches, normalizing the advantages so they have a mean of 0 and standard deviation of 1, and flattening the tensors so they can be fed into the neural network.

D4RL​

Datasets for Deep Data-Driven Reinforcement Learning — the standard benchmark suite for offline RL. It ships fixed datasets of recorded transitions for common MuJoCo control tasks (like HalfCheetah and Walker2d) plus mazes and robotic tasks, each collected by a behavior policy of a stated skill level — random, medium, expert, and mixtures like medium-replay — so everyone trains on the same data and reports comparable numbers. A dataset name like walker2d-medium-v2 therefore pins down both the task and exactly how good the data-collecting agent was. D4RL plays the role for offline RL that ImageNet plays for image classification: a shared yardstick that makes papers' results directly comparable.

DCGAN​

Deep Convolutional GAN — the 2015 recipe that first made GAN training reliable, by building both the generator and discriminator out of convolution layers with a few simple rules (batch normalization, no pooling layers, specific activations). Before it, GANs often fell apart mid-training; DCGAN's architecture became the default starting point that almost every later image GAN built on.

DDIM​

Denoising Diffusion Implicit Models — a way to sample from an already-trained DDPM far faster. The word "Implicit" means that instead of relying on a strict, random step-by-step chain to add noise, the math implicitly defines a non-random shortcut that reaches the same noisy result, allowing us to skip many steps when generating an image backward. Where DDPM's reverse process is stochastic (it injects fresh randomness at every step and may need ~1000 steps), DDIM makes the path deterministic: the same starting noise always yields the same image, and the smooth path lets you skip most steps, so ~50 steps match 1000-step quality. Crucially it reuses the same trained network — DDIM changes only how you sample, not how you train. Like taking a few long, confident strides across a room instead of many tiny shuffles.

DDIM inversion​

Running the deterministic DDIM sampler in reverse to find the starting noise that would regenerate a given real image. Normal sampling goes noise → image by removing a little noise each step; inversion walks the same path backward, image → noise, adding the noise the model would have removed. Once you hold that noise you can change the prompt and denoise forward again, and because the path is largely reused the edit keeps the original's layout and pose. The catch is drift: each backward step is only approximate, so the recovered noise does not reconstruct the photo perfectly. A follow-up technique called null-text inversion fixes this by tweaking the empty-prompt embedding — the baseline "blank canvas" signal the model uses when given no text — until the reconstruction perfectly matches the original photo. Think of it like reverse-engineering the exact base batter recipe (the empty prompt) for a finished cake. Once you tweak that base batter so the cake bakes perfectly, you can swap in one new ingredient (the new text prompt) to change just the flavor, while the shape and texture come out exactly the same.

DDP​

Distributed Data Parallel — replicate model, split batch, all-reduce gradients

DDPG​

Deep Deterministic Policy Gradient — the algorithm that carried DQN's off-policy, replay-buffer recipe into continuous control. With continuous actions you cannot take maxₐ Q(s, a) — there is no finite list of actions to scan — so DDPG learns a deterministic actor μ(s) that simply outputs the action it thinks is best, plus a critic Q(s, a) that scores it. The actor is trained by the deterministic policy gradient: push the critic's gradient back through the chosen action into the actor's weights, nudging the actor toward actions the critic rates higher. Because the policy is deterministic, exploration must be injected by hand — usually Ornstein-Uhlenbeck or plain Gaussian noise added to the action. Analogy: a student (actor) always gives one definite answer, and a grader (critic) says how good it was; the student keeps adjusting in whatever direction raises the grade. DDPG works but is famously twitchy — a single critic tends to overestimate values, and small errors snowball — which is exactly what TD3 and SAC were built to fix; treat it as a pedagogical stepping stone, not something to ship.

DDPM​

Denoising Diffusion Probabilistic Models — the foundational 2020 paper and recipe that kicked off the modern diffusion era. Training is disarmingly simple: take a clean image, add a known amount of random (Gaussian) noise, and teach a network (usually a U-Net) to predict that noise so it can be subtracted back off; the loss is just mean squared error on the noise. To generate, start from pure static and repeat the learned "remove a little noise" step many times (classically 1000) until an image appears. Because there is no adversarial game, it sidesteps the mode collapse that plagues GANs.

Deadly triad​

The combination of three core reinforcement learning ingredients that together make value function training unstable and prone to divergence (value estimates blowing up to infinity):

  1. Function approximation: Using a parameterized model like a neural network to estimate values instead of a lookup table (allowing the agent to generalize to unseen states).
  2. Bootstrapping: Updating value estimates using other current value estimates (like in temporal-difference learning) instead of waiting for the actual final return of an episode.
  3. Off-policy training: Learning from data collected by a different policy than the one currently being improved (e.g., from an experience replay buffer).

When all three are combined, errors in function approximation can be amplified by bootstrapping, while off-policy training fails to correct these updates, causing a dangerous positive feedback loop. Algorithms like DQN stabilize this triad by using a target network to freeze bootstrapping targets and experience replay to stabilize the data.

Decision Transformer​

An offline RL method that drops value functions and Bellman updates and instead treats control as sequence modeling. It feeds a transformer a timeline of tokens — desired return-to-go, state, action, repeating — and trains it, exactly like a language model predicting the next word, to predict the next action. At test time you prompt it with the return you want and the current state, and it autoregressively emits actions consistent with reaching that return, because in training it learned which action patterns preceded which outcomes. This reframes "find the optimal policy" as "predict what an agent that earned this much reward would do," which works well on large, varied datasets and needs none of the out-of-distribution machinery of CQL or IQL — at the cost that it reliably hits only return levels the data actually demonstrates. Its relative the Trajectory Transformer models states and rewards too and plans with beam search.

Decode​

The token-by-token half of LLM inference: after prefill digests the prompt, the model generates one new token per forward pass, each step reading the whole KV cache before producing the next logits. Like writing a sentence one word at a time while glancing back over every word already written — fast per step, but the constant re-reading of the page is what bounds speed. Decode is memory-bandwidth-bound on a GPU, the opposite of prefill, and is what most serving optimizations target.

decord​

A fast video-reading library that decodes frames straight into tensors, built for deep-learning data loaders. Its key trick is efficient random access: you can ask for "frames 0, 30, and 90" and it jumps to them without decoding everything in between, which is exactly what frame sampling needs. Analogy: a regular video player reads a movie front to back like a cassette tape, while decord works like a book with an index — it flips straight to the page you want. Example: vr.get_batch([0, 30, 90]) returns just those three frames as a single tensor, ready for the model.

Decoupled​

A training technique where two effects that are mathematically equivalent in standard SGD are separated into independent operations. In AdamW, weight decay is decoupled from the gradient update so that the regularization strength is not scaled by the adaptive learning rate.

Deduplication​

Removing repeated or near-repeated documents from a training corpus; one of the highest-return cleaning steps in pretraining.

Deformable manipulation​

A subfield of robotic manipulation focused on objects that change shape under force, such as cloth, ropes, wires, bags, or soft food. Because these objects have infinite degrees of freedom and can fold, stretch, or wrinkle, classical geometric models (which work well for rigid-body objects) fail, requiring either highly flexible physics simulators or policies trained via imitation learning or reinforcement learning.

  • Analogy: Picking up and packing a cardboard box is rigid-body manipulation: it keeps its shape, and you can easily plan your grip. Folding a wet t-shirt or wrapping a wire around a spool is deformable manipulation: the item twists and flops in your hands, meaning you must constantly adjust your motion based on how the material reacts.
  • Example: Robotic cloth folding, where a robot uses a camera to track the corners of a towel, executes pick-and-place motions, and measures its success by calculating the IoU (Intersection over Union) between the folded cloth and a target shape.

Dexterous manipulation​

Dexterous manipulation is the capability of a robot to control and reorient objects within its hand's grasp using multi-fingered end-effectors, relying on complex coordination of contacts, forces, and movements.

  • Why it matters: Standard robotic grippers (like simple two-finger parallel grippers) can only pick up and place objects. Dexterous manipulation allows a robot to perform in-hand reorientation (like rotating a tool in its hand to use it) or handle highly deformable objects, which is essential for human-like daily tasks.
  • How it works: This is typically modeled as a contact-rich control problem involving dynamic force balance, friction constraints, and tactile sensing. Since mathematically modeling all contact transitions is extremely difficult, dexterous manipulation is frequently solved using deep reinforcement learning in simulation followed by sim-to-real transfer.
  • Analogy: Imagine trying to write with a pencil. You do not just pick up the pencil and move your entire arm to write. Instead, you use your fingers to roll and slide the pencil within your hand until it is positioned at the perfect angle. That precise, finger-level adjustment is dexterous manipulation.
  • Example: Training a multi-fingered Shadow Hand model using PPO in a physics simulator to rotate a Rubik's cube in its palm, then successfully transferring that policy to a physical robot hand.

Deep network​

A neural network with many layers stacked one after another, so the input passes through a long chain of transformations before reaching the output — "deep" literally refers to that depth (the number of stacked layers), in contrast to a "shallow" network of just one or two. Each layer builds on the features the previous one produced: in an image model the early layers might pick out edges, the middle layers shapes, and the later layers whole objects — like an assembly line where every station adds a little more refinement. Depth is what lets these models learn rich, abstract patterns, but it also makes them hard to train, because the learning signal (gradients) has to travel back through every layer and tends to fade or blow up along the way — which is precisely the problem residual connections and normalization were invented to tame.

Deep reinforcement learning​

A subfield of artificial intelligence that combines deep learning (using deep neural networks to process complex data like pixels or sound) with reinforcement learning (learning by trial and error from rewards).

  • Why it matters: Traditional reinforcement learning is like teaching a dog tricks in a clean, quiet room where the rules are simple. It works well if the environment has only a few possible states (like a grid-world game). But if you want a robot to navigate a messy real-world kitchen or play a complex video game from raw screen pixels, the state space is too vast. Deep reinforcement learning uses deep networks to compress these high-dimensional sensory inputs into useful patterns so the agent can learn to make decisions.
  • Analogy: Imagine trying to teach someone how to drive a car. Traditional RL is like giving a driver instructions on a closed track where they only need to worry about speed and steering angle. Deep RL is like putting them in a real city: they have to look at the street through their eyes (processing millions of pixels of visual data), recognize cars, pedestrians, and traffic lights (deep learning's job), and then decide when to accelerate, brake, or steer to safely reach their destination (reinforcement learning's job).
  • Example: DeepMind's AlphaZero playing Go, or a robotic hand learning to manipulate objects using a camera feed as input. The network processes the image to understand the object's position and then calculates the joint torques to move it.

Deepfake​

A fake but convincingly realistic image, video, or audio clip of a real person — produced by an AI model — that shows them doing or saying something they never did. The name blends "deep learning" with "fake." Like a forged signature, but for someone's face and voice: the danger is that a viewer cannot tell it apart from genuine footage, which is why deepfakes drive misinformation, fraud, and non-consensual imagery. They are the central threat that watermarking and detection tools like SynthID try to counter, by tagging or spotting synthetic media so a clip's origin can be checked.

DeepMind Control Suite (DMC)​

A standard set of continuous-control benchmark tasks built on the MuJoCo physics engine by DeepMind — balancing a cartpole, swinging up a pendulum, and making simulated cheetahs, walkers, and humanoids run. Every task shares one reward convention (each episode's score falls in a fixed 0–1000 range), which makes it easy to compare algorithms across many tasks on the same scale. It is the go-to proving ground for modern continuous-control methods such as SAC, DreamerV3, and TD-MPC2. Think of it as a decathlon for control agents: a fixed slate of varied physical events, all scored the same way.

DeepSpeed​

Microsoft's open-source library for training very large models efficiently. It is best known for ZeRO, which shards a model's parameters, gradients, and optimizer state across GPUs so no single GPU has to hold the whole model — the same idea as PyTorch's FSDP. Think of it as a moving company that splits one giant load across several trucks instead of trying to cram everything into one.

Delayed policy updates​

One of TD3's three stability fixes: update the critic several times (typically twice) for every single update of the actor. The reasoning is that the actor is only as good as the critic guiding it; if the actor chases a critic whose value estimates are still bouncing around, it learns to exploit transient errors. Letting the critic settle between actor updates gives the actor a steadier target to climb. Analogy: don't keep re-drawing your route while the map is still being printed — wait for the ink to dry, then plan. Compare target policy smoothing and twin critics, the other two TD3 fixes.

Degrees of freedom​

The number of independent values you must specify to fully describe a system's configuration — equivalently, the number of independent ways it can move. A robot arm with six motors has six degrees of freedom (DoF), because six joint angles pin down every part of it; a rigid body floating freely in 3D also has six (three for position, three for orientation). Analogy: a train has one degree of freedom (forward/back along the track), a car on a flat lot has three (x, y, and heading), and your shoulder-to-fingertip arm has seven. The count matters because it sets how many independent goals the robot can satisfy at once: a 6-DoF arm can hit a full 3D position-and-orientation target exactly, while a 7-DoF arm has a spare DoF it can spend on extra goals like dodging obstacles.

Dead reckoning​

Working out where you are by starting from a known spot and adding up every small motion since — direction and distance, step by step — without any outside reference to check against. An IMU does this by summing (integrating) its accelerometer and gyroscope rates over time to track position and orientation. Its fatal flaw is drift: because each new estimate is built on top of the previous one, tiny measurement errors never cancel out — they pile up, so the estimate wanders further from the truth the longer you go with no correction. Analogy: crossing a dark room with your eyes closed, counting your steps — you can guess your position for a few seconds, but with no glance at the walls you are soon lost. The name comes from ship navigation, where sailors logged heading and speed to estimate position out of sight of land.

Depth map​

A grayscale image that records how far away each pixel is rather than its color — near things are drawn light and far things dark (or the reverse), like a black-and-white fog where closer objects glow brighter. It throws away texture and color and keeps only the 3D shape of a scene: which parts stick out toward the camera and which recede into the distance. You can estimate one from an ordinary photo with a depth-prediction network, or capture it directly with a depth sensor. ControlNet uses a depth map as a conditioning signal so a generated image keeps the same sense of near-and-far layout — the prompt repaints the surfaces, but a person standing in front of a wall stays in front of the wall.

Detached tensor​

A tensor that has been removed from the dynamic computation graph via the .detach() method, meaning operations performed on it will not be tracked for autograd.

Derivative​

The instantaneous rate of change of a function with respect to its input. In deep learning, derivatives are computed via the chain rule during backpropagation to produce gradients used to update model parameters.

Deterministic algorithms​

Operations that produce bit-identical outputs for identical inputs every time; enabled in PyTorch via torch.use_deterministic_algorithms(True) at the cost of some performance

Deterministic policy gradient​

The rule for training a deterministic actor μ_θ(s) — one that outputs a single action rather than a probability distribution — used by DDPG and TD3. Ordinary policy gradients work on a stochastic policy by reweighting sampled actions; with a deterministic policy there is nothing to sample, so instead you ask the critic directly: "if I nudge the action a little, does my Q-value go up?" Formally ∇_θ J = E_s[ ∇_a Q(s, a)|_{a=μ_θ(s)} · ∇_θ μ_θ(s) ] — the chain rule run through the critic into the actor's weights. In plain terms: the critic tells you which direction in action-space raises value, and you adjust the actor to move its output that way. Analogy: a chef (actor) plates one dish, a judge (critic) says "a touch more salt would score higher," and the chef shifts the recipe exactly in the direction the judge points. This only works because the critic is differentiable in the action, which is why continuous, smooth action spaces are required.

Detokenization​

Turning a sequence of token IDs back into a UTF-8 string — the reverse of what the tokenizer did on the way in. The tricky part for streaming servers is that a single visible character (like an emoji or a Chinese character) is often spread across several BPE pieces, so emitting each token's text the moment it arrives can produce broken bytes; a correct streaming detokenizer buffers the partial bytes until they form a complete character.

DH parameters​

Denavit-Hartenberg parameters — textbook arm-geometry description

Dial​

Used throughout these guides as a verb: to dial a value up or down means to adjust it smoothly anywhere along a continuous range — the way you turn a knob — instead of flipping a switch that is only ever fully on or fully off (the same idea as a soft gate). The word borrows from a physical dial: a round gauge you rotate, so its setting is an angle. That rotating-angle picture is also why a dial is an apt image for RoPE, which marks a token's position by rotating its vectors through an angle. Examples elsewhere in this glossary: a temperature dial that trades safe answers for creative ones, or a motion score that dials how much a generated video moves.

DIAYN​

DIAYN ("Diversity Is All You Need") is an unsupervised skill-discovery method that learns a set of distinct behaviors without any environment reward. It feeds the policy a randomly chosen "skill" code as an extra input and gives an intrinsic reward for making the visited states easy to guess from the skill yet different between skills — formally, for maximizing the mutual information between the skill code and the states it produces. The result is that each code carves out its own behavior (one skill walks forward, another spins, another stays put), giving a ready-made library of moves that a later reward-driven task can reuse. It is closely related to maximum-entropy RL, which likewise pushes for a varied, non-collapsed policy. Like handing several actors the same empty stage and paying each to invent a routine no one would mistake for another's.

Dijkstra's algorithm​

An algorithm for finding the shortest paths between nodes in a weighted graph (where edges have different costs or distances). It starts at a source node and iteratively visits the unvisited node with the smallest tentative distance, updating the distances to its neighbors. It is guaranteed to find the absolute shortest path if all edge weights are non-negative. Analogy: Imagine pouring water onto a starting spot on a network of pipes. The water flows through all pipes at the same speed, naturally filling the closest junctions first. The order in which the water reaches each junction tells you the shortest piping distance from the start to that junction. Example: Finding the fastest driving route between two cities on a map where each road has a travel time. Dijkstra's algorithm systematically searches outward from the start city, checking all intersections in order of travel time until it reaches the destination.

Differential drive​

A method of steering a mobile robot using two independently powered drive wheels on a shared axle.

  • Why it matters: It is the simplest and most common wheel configuration for indoor mobile robots (such as robot vacuums). It allows the robot to rotate in place with a zero turning radius, making it highly maneuverable in tight spaces.
  • How it works: Steering is achieved by varying the relative speed of the two wheels. If both wheels turn at the exact same speed forward, the robot moves in a straight line. If they spin in opposite directions, the robot pivots around its center point.
  • Analogy: Rowboats steer using differential drive. If you pull both oars together, the boat goes straight. If you pull the left oar harder, the boat turns right.

Differential flatness​

A property of some dynamical systems where all the system's states (like velocity, tilt, and angle) and control inputs (like motor forces) can be computed directly from a small set of special outputs (called flat outputs) and their derivatives over time, without solving any differential equations.

  • Why it matters: In general, planning a path for a complex machine (like a drone or rocket) is hard because you have to solve complex physics equations to ensure the engines can actually execute the path. If a system is differentially flat, you can plan the path purely in the simple output space (like 3D position) as a smooth curve, and then directly calculate the exact motor forces needed to follow it.
  • How it works: For a quadrotor, the flat outputs are its 3D position (x, y, z) and its yaw angle. If you write down a smooth 3D curve for the position and yaw over time, the physics of a quadrotor uniquely determines its tilt (roll and pitch), velocity, acceleration, and the speed each of the four rotors must spin to stay on that curve.
  • Analogy: Imagine pulling a heavy cart along a path with a rope. If the cart's position is a flat output, just knowing the path the cart took (and how fast it moved) tells you exactly where the rope had to be pointed and how hard you had to pull at every second, without needing to simulate the wheel friction or cable tension step-by-step.
  • Example: Drones plan flight paths as minimum-snap polynomial curves in (x, y, z, yaw) space because the drone's differential flatness allows it to bypass full rotational dynamics during the planning phase.

Diffusion Forcing​

A training trick that gives each frame its own independent noise level instead of noising the whole clip to the same amount. Because the model learns to denoise a sequence whose frames sit at different stages of cleanup, at generation time it can hold earlier frames clean while denoising later ones — letting one model both denoise a full clip at once and roll out frame-by-frame like an autoregressive model. It is the bridge between full-sequence diffusion and next-frame prediction, and underlies several long-form and streaming video methods. The name captures the idea that each frame is independently "forced" to a chosen noise level.

Diffusion model​

A generative model that learns to un-noise an image (or video, or audio). The Key Intuition: The model only learns the reverse process. The forward process (adding noise) is a fixed, mathematical destruction (like randomly shuffling a puzzle) that requires no learning. The reverse process is the actual learning phase: the network is handed a scrambled image along with a strict label of how much noise is currently present (often measured by a timestep t or standard deviation σ). This noise level acts as a critical condition, physically injected into the network via mechanisms like AdaGN so the model knows whether to focus on forming broad outlines (high noise) or tweaking fine details (low noise).

Diffusion policy​

A robotic control policy that represents its action distribution using a denoising diffusion model conditioned on visual and physical observations. Instead of directly predicting a single action vector via an MLP (which struggle with multimodal distributions), a diffusion policy starts with random noise and iteratively refines it into a set of smooth, continuous actions.

  • Why it matters: Real-world robotic tasks are highly multimodal—for instance, when avoiding an obstacle, a robot could go left or right, but averaging those options leads to hitting the obstacle head-on. By treating action generation as a denoising process, a diffusion policy can generate complex, multi-step trajectories that naturally capture these distinct choices without averaging them.
  • Analogy: Imagine trying to write a signature. An MLP policy tries to draw the entire signature in one single stamp (and if it has seen both "John" and "Jane," it might stamp a blurry average of both). A diffusion policy is like sketching the signature: it starts with a rough pencil scribble (noise) and gradually rubs out mistakes and sharpens the lines over several quick passes until it forms a crisp, clean name.
  • Example: In a bin-picking task where a robot must choose between two identical cups, a diffusion policy will cleanly generate a trajectory towards either the left cup or the right cup, whereas a standard MLP policy might command the gripper to move to the empty space directly between them.

DINOv2​

A strong, off-the-shelf image encoder from Meta trained in a self-supervised way via self-distillation — it learns purely from images, with no human labels, by teaching the network to give two different crops of the same photo matching internal descriptions. The result is a general-purpose ViT backbone whose features work well for many tasks (classification, segmentation, depth) right out of the box, often beating label-trained encoders on a linear probe. Like a student who learns to recognize objects just by looking at millions of pictures and noticing what stays the same when an object is moved or cropped, never being told any object's name. The "v2" marks the second, larger and cleaner-data version; the name comes from self-distillation with no labels.

Direct collocation​

A method for solving optimal control and trajectory optimization problems by discretizing both the state trajectory x(t) and the control trajectory u(t) at a set of time points (collocation points). The system's differential equations (dynamics) are enforced as algebraic constraints at these points (typically using polynomial interpolation like trapezoidal or Hermite-Simpson rules). This converts a continuous-time optimal control problem into a standard nonlinear programming (NLP) problem that can be solved with numerical solvers like IPOPT. Analogy: Imagine you want to plan a rocket's flight path. Instead of trying to find a continuous mathematical formula for the engine thrust over the whole flight, you break the flight into 100 segments. For each segment, you treat the position, speed, and thrust as variables, and write algebraic equations ensuring that the change in speed matches the thrust at that segment. Example: Planning a cart-pole swing-up maneuver. The states (cart position/velocity, pole angle/velocity) and control forces are discretized. The dynamics equations are written as constraints at each step, and a solver like IPOPT finds the sequence of forces that swings the pole up from hanging to upright in minimum time.

Disaggregated serving​

Running prefill and decode on separate GPU pools with KV cache transfer between them

Discount factor​

The number γ (gamma), between 0 and 1, that says how much a reward is worth per step of delay: a reward k steps in the future counts as γᵏ times its face value. With γ close to 0 the agent is short-sighted and chases only immediate reward; with γ close to 1 it is patient and plans far ahead. Two reasons it exists: it keeps the infinite sum of future rewards from blowing up (the geometric series 1 + γ + γ² + … = 1/(1−γ) is finite only when γ < 1), and it encodes a real preference for sooner over later, the way money today is worth more than money next year. Its size sets the effective horizon — roughly how many steps ahead the agent effectively cares about. It is the γ in the Bellman equation and one of the five parts of an MDP.

Discriminator​

The "critic" half of a GAN: a network that looks at an image and outputs how likely it is to be real rather than made by the generator. It is trained like a detective spotting fakes, and its verdicts are the only teaching signal the generator ever gets — as the discriminator sharpens, the generator is forced to make more convincing images. In Wasserstein GANs it outputs an unbounded score instead of a 0–1 probability and is usually called a critic.

Dispatcher​

The PyTorch component that routes torch.foo(...) calls to the right backend/dtype kernel

Distillation​

Training a smaller "student" model to copy the output of a larger, more capable "teacher" so the student inherits most of the teacher's behavior at a fraction of the cost. Like a junior cook shadowing a head chef and learning each recipe by mimicking the dish — they may never match the master, but they can plate most of the menu for far less money. Distillation works for skills the teacher already has but cannot conjure new abilities the teacher lacks.

Disparity​

The sideways shift of the same scene point between the left and right images of a stereo camera pair. Hold up a finger and blink each eye in turn: it jumps sideways a lot when near and barely moves when far — that jump is disparity, and it shrinks with distance. Measuring it for every pixel is the heart of stereo depth, because distance equals (focal length × baseline) ÷ disparity — a large shift means the object is close, a tiny shift means it is far. Objects with no texture (a blank white wall) offer no distinctive pattern to match between the two images, so their disparity, and hence their depth, cannot be measured.

Distribution drift​

When the kind of data a model sees in production slowly changes away from the data it was tuned on — like a store whose regular customers gradually change their tastes, so last year's best-selling stock starts to sit on the shelf. For a quantized model it matters because calibration was fitted to the old traffic, so quality can quietly slip as the new traffic drifts further away.

Distribution shift​

When the data a model is used on differs from the data it was trained on, so its outputs become unreliable. It is the central obstacle in offline RL: training data comes from the behavior policy, but as learning improves the policy it starts preferring different actions and visiting different states than the dataset ever showed — exactly the out-of-distribution regions where a Q-function was never corrected and can hallucinate wildly wrong values. Analogy: a student who only ever practiced on last year's exam paper and then freezes when this year's questions are phrased differently. Offline-RL algorithms fight it either by keeping the policy close to the data (policy constraint) or by being pessimistic about unfamiliar actions (CQL, IQL).

Distributional RL​

A family of value-based methods that predict the entire distribution of possible returns from a state-action pair, rather than collapsing it to the single expected number that ordinary Q-learning tracks. Two actions can share the same average return while one is a safe bet and the other is feast-or-famine; modeling the spread, not just the mean, gives the network a richer signal to learn from and in practice makes training more stable — even though you usually still act by picking the action with the highest mean. The best-known members are C51 (a fixed grid of return levels), QR-DQN, and IQN (which learn return quantiles directly). Analogy: instead of "this restaurant averages 3 stars," it gives the full shape — mostly 4-star nights with the occasional disaster — which is far more useful when you are deciding whether to risk it.

DiT​

Diffusion Transformer — Peebles & Xie's diffusion backbone that replaces the U-Net with a pure transformer. It chops the noisy image (really its VAE latent) into a grid of small patches, turns each patch into a token, and lets attention mix them — the same recipe that took over language modeling, now pointed at denoising. The name simply joins "diffusion" (the denoising task) with "transformer" (the architecture). Its big draw is scaling: make it wider or deeper and quality improves along a predictable curve, the way a bigger language model reliably gets better. Like swapping a custom-built, image-shaped machine (the U-Net) for a general-purpose assembly line you can just make longer to produce more. Sizes are named DiT-S (small), DiT-B (base), DiT-L (large), and a suffix like "/2" gives the patch size — DiT-S/2 is the small model with 2×2 patches.

Divergence​

In GPU computing, divergence (often called branch divergence or warp divergence) occurs when different threads within the same warp (a group of 32 threads executing in lockstep) need to execute different paths of a conditional branch (like an if-else statement) because their data differs.

Because GPUs are built on the SIMT model, all threads in a warp must execute the same instruction at the same time. When they diverge:

  1. The hardware runs the if path first, executing the threads that took that branch while masking out (holding idle) the threads that need to run the else path.
  2. The hardware then runs the else path, executing the remaining threads while masking out the first group.

This serializes execution, meaning the total time taken is the sum of both paths, which can significantly slow down parallel performance.

Analogy: Imagine a tour guide (the instruction scheduler) leading a group of 32 tourists (the threads in a warp).

  • No divergence: The guide says "Everyone take a photo of this monument" (a single instruction). Everyone does it at once (full parallel efficiency).
  • Divergence: The guide says "If you want to buy souvenirs, go to the left door; otherwise, go to the right door." Because tourists want different things, the guide cannot be in two places at once. The guide first takes the souvenir group to the left door while the others stand around waiting, and then takes the second group to the right door while the first group stands around. The tour takes twice as long because the group split up, even though the tourists themselves are capable of walking independently.

Dolly​

A camera move where the whole camera physically travels toward or away from the subject — the name comes from the wheeled cart (a "dolly") that camera operators roll along a track. Unlike a zoom, which only magnifies the image from a fixed spot, a dolly actually changes the camera's position, so the background shifts relative to the foreground and you get a real sense of moving through the scene. It is one of the moves a video model can be directed through with camera control.

Domain adaptation​

A machine learning technique where a model trained on one data distribution (the source domain, like simulation) is adjusted to perform well on a different but related data distribution (the target domain, like the real world). In robotics, this involves updating the policy or features learned in simulation so they map correctly to the sensors and physics of real hardware.

  • Analogy: Imagine you are a skilled driver who has only ever driven left-hand drive cars in the US (source domain). When you travel to the UK and have to drive a right-hand drive car on the left side of the road (target domain), you do not relearn how to steer or brake from scratch. Instead, you adapt your existing driving skills to the new rules and layout.
  • Example: A visuomotor policy trained on simulated images uses a neural network to translate real-world camera images into simulated-style images before feeding them to the controller, allowing the robot to use its simulation-trained logic directly.

Domain randomization​

A technique used to bridge the reality gap by randomly varying physical and visual properties of the simulator (such as friction, mass, object sizes, lighting, and sensor noise) during training. By exposing the robotic control policy to a wide variety of simulated worlds, the policy learns to be robust and adaptable, so that it can succeed on physical hardware without needing an exact model of the real world.

  • Analogy: If you want to train someone to catch a ball under any condition, you don't just throw a red tennis ball at them in a well-lit gym. You throw balls of different sizes, weights, and colors under bright sun, dim light, and rain. By training on all these variations, they can catch a ball anywhere.
  • Example: During training of an in-hand manipulation policy in MuJoCo, the simulator randomly changes the weight of the cube, the friction of the fingers, and the visual background in every training episode.

Dot product​

A way to boil two equal-length lists of numbers (two vectors) down to a single number: multiply them position by position, then add up all the products. For [1, 2, 3] · [4, 5, 6] you compute 1·4 + 2·5 + 3·6 = 4 + 10 + 18 = 32.

What it does (the effect). The dot product works as a similarity score between two vectors. It comes out large and positive when the two lists "point the same way" — their big numbers sit in the same slots — near zero when they are unrelated, and negative when they pull in opposite directions. Analogy: imagine two friends each rate ten movies from −5 to +5. Multiply their scores movie by movie and add them up: if they both loved and both hated the same films the total is a big positive number (very alike); if their tastes are unrelated the pluses and minuses cancel out near zero; if one loved what the other hated it goes negative. The dot product is exactly that "how aligned are we?" number.

How to calculate it with vectors (and matmul). Doing one dot product is the multiply-and-sum above. Doing many at once is precisely what matrix multiplication (matmul, written A @ B) is built from: each number in the output grid is the dot product of one row of the left matrix with one column of the right matrix. So a single matmul is just a big batch of dot products computed together. This is why, in a projection discriminator, "taking a dot product between the image's features and a learned class vector" is simply measuring how much the image lines up with that class — a high dot product means "this really looks like that category."

Double backward​

Computing the gradient of a gradient by tracking the backward pass operations in a new computation graph.

Double DQN​

A one-line fix for the overestimation bias baked into ordinary DQN. Plain DQN builds its target with maxₐ Q_target(s′, a), letting the same network both choose the best next action and score it — so whenever noise makes some action look too good, that inflated value is taken at face value. Double DQN splits the two jobs: the online network chooses the next action (the argmax of its own Q-values), and the target network evaluates that chosen action, so a lucky overestimate in one network is unlikely to be confirmed by the other. Analogy: don't let the same student both write an exam answer and grade it — a second marker catches the wishful thinking.

Downstream​

The later, real-world tasks a model is eventually judged on — such as question answering or coding — as opposed to the pretraining objective it was trained on. "Downstream scores" measure how much a change (cleaner data, a better learning rate) actually pays off on those end tasks, the way a river's health downstream reflects what happened upstream at the source.

DPM-Solver​

Short for Diffusion Probabilistic Model Solver — a fast sampler for diffusion models. While the baseline Euler method blindly takes small straight-line steps based on the immediate slope (requiring hundreds of tiny steps to avoid wandering off-path), DPM-Solver exploits the known mathematical curvature of the ODE. By calculating the exact linear parts ahead of time, it can take large, confident strides, effectively reaching the same quality in just 10–20 steps. DPM-Solver++: An upgraded version optimized for high-CFG environments. High CFG makes the predicted noise direction (ε) oscillate wildly, which confuses the standard DPM-Solver. DPM-Solver++ fixes this by mathematically pivoting the equation to predict the stable final clean image (x₀) instead of the wavering noise direction. By anchoring its steps to this unmoving final destination, it safely prevents image burn and artifacts — the oversaturated, blown-out colors and blotchy fake textures that show up when too-high guidance pushes pixel values past their valid range, like overexposing a photo until the bright spots turn into harsh white patches — even under aggressive guidance.

DPO​

Direct Preference Optimization — a way to do RLHF-style alignment that skips the usual two-step machinery (first train a separate reward model, then optimize against it with PPO) and instead tunes the model directly on pairs of (chosen, rejected) answers to the same prompt. The trick is a math shortcut — a closed-form result, meaning an exact formula you can write down directly instead of searching for the answer by trial and error. It proves you never have to build a separate reward model at all: the score that reward model would have given is already hidden as an implicit reward inside the language model's own answer probabilities (how likely the model thinks each answer is) compared to a frozen reference model. So, one simple training step does the whole job — nudge the model to make the human-preferred answer a little more likely and the rejected answer a little less likely. To stop it from over-correcting and wandering off into nonsense, this nudge is measured relative to the reference model — a saved, unchanging copy of the model from before tuning — so the tuned model can only drift a small distance from where it started, like a climber clipped to a fixed anchor who can move around but not fall far. Like teaching a cook by repeatedly showing them two plates and saying "this one, not that one," instead of first writing a detailed scoring rubric (the reward model) and then training against the rubric. Example: given a prompt and two responses a human marked better/worse, one DPO step nudges the model toward the better one — no reward network and no RL rollouts required, which makes it far simpler and cheaper to run than PPO.

DQN​

Deep Q-Network — a reinforcement learning algorithm that combines Q-learning with a deep neural network, enabling an agent to learn actions in complex environments with large state spaces (like raw game screen pixels). Rather than keeping a giant, impractical lookup table of action values for every possible situation, DQN uses a neural network to predict the values of different actions based on the current state. To keep training stable, it relies on two key tricks: experience replay (training on random mixes of past experiences rather than only the most recent moves) and a target network (a frozen, slow-to-update copy of the model used to compute the learning target). As a value-based method, it contrasts with policy-gradient methods like REINFORCE which directly optimize the action probabilities without estimating their values. Analogy: learning to drive. Using a lookup table is like memorizing exactly what to do at every coordinate on Earth—impossible. DQN is like learning rules of thumb (when a car appears ahead, slow down) that generalize to new roads. The experience replay is like studying a mixed set of video logs from previous driving sessions at night, so you do not forget how to drive in the dark just because you spent the afternoon driving in the sun.

Draft model​

In speculative decoding, a small, fast model that guesses the next few tokens so the big target model can check them all at once. Like a quick assistant who scribbles a rough draft for the expert to approve or correct — cheap to run, and most of its guesses turn out right, so the slow expert is consulted far less often.

DreamBooth​

A personalization recipe that fine-tunes the whole diffusion model on just 3–5 photos of one subject and binds it to a rare trigger word, so you can afterwards prompt "a photo of [V] dog surfing." Because every weight is updated the likeness is excellent, but the saved model is full-sized — the opposite trade-off from a lightweight LoRA. To stop the model from catastrophically forgetting what other dogs look like, it adds a prior-preservation loss that keeps training on the model's own generic class images. Like memorizing one specific face in such detail that you must consciously remind yourself other faces still exist.

DreamerV3​

The third version of Dreamer (Hafner et al.), a reinforcement-learning agent that first learns a compact world model of its environment and then trains its policy almost entirely inside that learned model — "imagining" thousands of future rollouts rather than acting in the real (slow, expensive) environment. This guide borrows only the generative half: predict the next latent state given the current one and an action; the policy-learning loop that sits on top is reinforcement learning's job. DreamerV3 is famous for clearing a wide range of very different tasks — from Atari to collecting diamonds in Minecraft — with a single fixed set of hyperparameters, which had been an open challenge in RL. The "Dreamer" name captures the core idea: the agent learns by dreaming up experience in its head instead of living through all of it.

Drift​

The slow accumulation of small errors across a sequence — frame after frame, shot after shot, or step after step — until the result wanders noticeably away from where it began. Each individual step is only slightly off, but because every new piece is built on top of the previous (already-imperfect) one, those tiny mistakes pile up and compound. In long video generation it shows up as colors creeping, a character's face or outfit slowly morphing into someone else's, or a scene losing its original layout the further you travel from the opening frame. The same compounding plagues robot state estimation: an IMU doing dead reckoning, or a visual odometry system, sums tiny per-step motion errors until its estimate of where the robot is has wandered meters from the truth. Like photocopying a photocopy of a photocopy: any single copy looks fine, but after a hundred generations the image has visibly degraded. Fixes work by repeatedly re-anchoring to a fixed reference so the errors cannot keep stacking up — for example pinning a character's identity with an IP-Adapter or character LoRA, or overlapping and blending clips in sliding-window generation; in robot odometry the same re-anchoring is done by loop closure, which snaps the path back whenever a known place is recognized.

dtype​

A tensor's element data type — e.g. float32, float16, bfloat16, int8, bool

Dual encoder​

An architecture with two separate encoders — one per modality, e.g. an image tower and a text tower — that each map their input into the same shared space, where the two are compared by cosine similarity. The key trait is that the modalities never mix until the very end (late fusion): each side is encoded entirely on its own. CLIP is the classic example. Analogy: two translators who never talk to each other but have both been trained to render anything into one common interlingua, so their outputs can be lined up afterward. This separation is exactly what makes dual encoders fast for cross-modal retrieval — you encode the whole image collection once, in advance, and a new text query only has to be encoded and compared — but it also means they can only match or score, never reason over or generate the other modality the way a VLM can.

Dueling DQN​

A change to the shape of the DQN network, not its training rule. Instead of outputting one action-value Q(s, a) per action directly, the network splits into two streams after its shared body: one estimates the state's overall value V(s) ("how good is it to be here at all?"), the other estimates each action's advantage A(s, a) ("how much better than average is this action?"), and they recombine as Q(s, a) = V(s) + A(s, a). The payoff is that the agent can learn a state is good or bad without having to separately try every action there — valuable in the many states where the chosen action barely changes the outcome. To keep the split well-defined (many V/A pairs sum to the same Q), the advantages are centered by subtracting their mean before adding V.

Dyna​

Sutton's classic blueprint for model-based RL: learn a dynamics model from real experience, then use that model to generate extra imagined transitions and train the same value or policy on the mix of real and imagined data. The point is to squeeze more learning out of each costly real step by "daydreaming" plausible variations of it. Analogy: a chess student who plays a few real games, then spends the evening replaying imagined continuations in their head to practice more without needing new opponents. MBPO is the modern, carefully-tuned descendant of Dyna.

Dynamic computation graph​

A graph of operations built on-the-fly as code executes, representing the forward pass used for autograd.

Dynamic programming​

In RL, the family of algorithms — policy iteration and value iteration — that solves an MDP exactly using a fully known model: the transition probabilities and the reward function. "Dynamic programming" is Richard Bellman's term for cracking a hard problem by breaking it into smaller overlapping subproblems and reusing their answers; here the subproblem is "what is each state worth?" and the reuse comes from the Bellman backup, which writes a state's value in terms of its neighbors' values. It is the planning baseline that the rest of RL approximates once the model is unknown (meaning the agent does not know the rules of the world, like exactly where a step will take it or what reward it will get) and you can only sample transitions by actually acting in the environment.

Dynamic scaling​

A technique (often used in FP8 training and inference) that calculates a scaling factor for each tensor dynamically at runtime to prevent numerical underflow and overflow. Because low-precision formats have a very narrow dynamic range, dynamic scaling ensures values are scaled to fit perfectly within that range before conversion.

  • Analogy: Imagine trying to map a set of numbers onto a scale from 1 to 10. If your numbers range from 0.1 to 0.9, they will all get rounded to 0 or 1 if you don't scale them (underflow). If your numbers range from 100 to 900, they will all exceed the limit (overflow). Dynamic scaling is like an auto-zoom feature on a camera: it detects the range of numbers in the current frame and automatically multiplies them by a scale factor (like 10 for the small numbers, or 0.01 for the large numbers) so they fit perfectly within the 1-to-10 viewfinder, then records the zoom level so the original values can be recovered.
  • Example: In FP8 training on Hopper GPUs, the TransformerEngine tracks the maximum absolute value (amax) of a layer's activations over a history window. If the values start to shrink, it automatically increases the scaling factor to prevent underflow. If a large spike occurs, it decreases the scaling factor to avoid overflow, updating the scaling factors dynamically at each training step.

Dynamic quantization​

A quantization method that stores weights as int8 ahead of time but computes each layer's activation scale at runtime, just before the layer runs.

Dynamic Window Approach (DWA)​

A local collision-avoidance and path-tracking algorithm for mobile robots that operates directly in the velocity space of the robot (its forward speed v and rotational speed ω).

  • Why it matters: Robots must navigate around obstacles dynamically. DWA ensures the robot chooses safe, achievable speeds that steer it toward its goal without colliding with obstacles, while respecting the robot's physical acceleration limits.
  • How it works: It defines a "dynamic window" of feasible velocities that the robot can safely reach within the next time step given its current speed and acceleration limits. It then simulates the robot's trajectory for each velocity candidate, scores them based on progress toward the goal, speed, and distance from obstacles, and executes the highest-scoring safe velocity command.
  • Analogy: Driving a car in heavy traffic. You don't plan your exact path hours in advance; instead, you look at your steering wheel and pedals. You ask yourself: "Given how fast I can accelerate, brake, and steer, what are my safe speed options right now to get closer to my lane without hitting the car in front of me?"
  • Example: The local planner in a robot vacuum cleaner uses DWA to steer around chair legs and toys while following a global cleaning path.

Dynamics head (MuZero)​

The network inside MuZero that predicts the next abstract latent state and the immediate reward given a current latent state and a proposed action — written g(s, a) → s′, r.

What it does: It acts as the model's internal rules-of-the-game engine. It lets the agent simulate transitions purely in its head without actually executing the action in the real world or drawing the next frame of pixels.

Analogy: When playing a game in your head, you think: "If I move my knight here (action a) from this position (state s), my position will change (state s′) and I will capture their queen (reward r)." The dynamics head is this internal simulation step.

Dynamics model​

A learned stand-in for the environment's rules: given the current state and an action, it predicts the next state (and usually the reward) — written f(s, a) → s′, r.

Why it is called a "dynamics" model: In physics, "dynamics" refers to how forces make things move and change over time. In reinforcement learning, a dynamics model does not just make a static prediction (like a standard machine learning model that classifies an image as a cat or predicts a house price); instead, it models how the environment's state changes over time in response to the agent's actions.

Difference between "model" and "dynamics model": A generic "model" maps any input to an output (e.g., recognizing a face), whereas a "dynamics model" specifically maps a state and action to the future state (state + action → next state). It predicts the consequences of actions over time, letting an agent simulate experience cheaply instead of paying for real environment steps, which is the basis of model-based RL. Analogy: A standard classifier is like looking at a single photo and naming what is in it; a dynamics model is like looking at that photo, knowing someone pressed the "gas pedal" button, and predicting the next frame of the video.

The catch is compounding error: each predicted state feeds the next prediction, so small mistakes snowball over a long rollout — which is why model-based methods keep imagined rollouts short or train an ensemble to sense when the model is unsure. Analogy: a weather model that is accurate one hour out but drifts badly a week out. It can be a plain forward network, an ensemble, or a latent dynamics model that predicts in a compressed feature space rather than over raw pixels.

Eager mode​

PyTorch's default execution, where each operation runs immediately as its Python line is reached — flexible and easy to debug, but without the cross-operation optimizations a compiler can apply.

EAGLE / Medusa​

Self-speculation: extra heads on the target model propose tokens, no separate draft model

Earth Mover's Distance​

A way to measure how far apart two distributions are by the smallest amount of "work" needed to reshape one pile into the other — imagine shovelling a heap of dirt into the shape of a second heap, where work is dirt moved times distance carried. Also called the Wasserstein distance, it gives a smooth, meaningful number even when the two piles barely overlap, which is exactly why Wasserstein GANs use it in place of the original GAN loss that goes flat in that case.

Edge inference​

Running a model directly on the device in front of the user — a phone, laptop, car, or small embedded board — instead of sending the request to a data-center GPU. Like cooking at home rather than ordering delivery: it is private and works without a network, but you are limited to the small "kitchen" the device has, so models are kept small (1–8B), heavily quantized, and tuned to sip battery and fit in shared memory.

EDM​

A cleaned-up reformulation of diffusion from the paper Elucidating the Design Space of Diffusion-Based Generative Models (Karras et al. 2022) that strips away historical baggage and makes training and sampling much easier to tune. Two ideas carry it: index noise by its standard deviation σ rather than a discrete timestep (the σ-schedule), and precondition the network — rescale its input, output, and per-σ loss weight so it always sees roughly unit-variance signals no matter how much noise is present. The result is a flat, forgiving hyperparameter surface. A useful rule of thumb: if a 2020-era diffusion paper feels obscure, restate it in EDM's language and it usually becomes obvious.

Effective horizon​

A rough measure of how many steps into the future an agent's choices actually take into account, set by the discount factor: about 1/(1−γ) steps. The intuition is that rewards beyond this point have been shrunk by so many factors of γ that they barely move the decision. For example γ = 0.9 gives 1/(1−0.9) = 1/0.1 = 10 steps, γ = 0.99 gives 1/0.01 = 100 steps, and γ = 0.999 gives 1,000 — so nudging γ from 0.99 to 0.999 makes the agent plan ten times farther ahead. Like a flashlight beam that γ widens or narrows: it sets how far down the road you can see well enough to steer by.

Effective sample size​

A count of how many independent data points a batch is really worth, once you account for the fact that some rows are near-copies of each other. Consecutive steps from the same running rollout are highly correlated — a robot's position a fortieth of a second later tells you almost nothing new — so a batch of 1,024 such steps behaves like a much smaller batch of actually different observations. With a step-to-step correlation ρ (rho), the standard estimate is effective sample size ≈ (rows in batch) · (1 − ρ)/(1 + ρ): at ρ = 0.85, a 1,024-row batch is worth only about 89 independent rows. Analogy: asking the same person to restate their opinion ten times in a row does not give you ten opinions — you still only really have one, plus nine echoes. Running several environments in parallel helps because each adds a fresh, independent chain rather than a longer echo of the same one — see vectorized environment.

EfficientZero​

A MuZero variant tuned for sample efficiency that was the first to reach roughly human-level Atari performance from only about two hours of gameplay data. It adds a few targeted fixes — most notably a self-supervised consistency loss that forces the learned dynamics model's predicted next state to match the actual next state's encoding — so the model wrings useful structure out of far less data. Think of it as MuZero with better study habits, learning the same lessons from a fraction of the practice. It is a landmark result for model-based RL's core promise of strong sample efficiency.

EKF​

Extended Kalman Filter (EKF) is a version of the Kalman Filter designed to handle nonlinear systems by linearizing the system equations at each step. In the real world, most systems are nonlinear (e.g., a wheel turning converts rotational motion to linear motion using trigonometry). The EKF approximates these nonlinear functions by using a first-order Taylor expansion (taking the derivative, or Jacobian) around the current state estimate.

Analogy: Navigating a winding mountain road on a dark night. The road curves unpredictably (nonlinear dynamics). Instead of trying to model the entire curvy road at once, you look at the short, straight stretch of road directly in front of your headlights (linearizing at the current point). As long as you update this straight-line approximation frequently, you can successfully steer along the curves. However, if the road curves too sharply or your estimate is way off, the straight-line guess will miss the road entirely, causing the filter to diverge.

Example: A self-driving car tracking its position. Its sensors report wheel speeds and steering angle, which relate to its coordinate change via nonlinear trigonometric equations. An EKF linearizes these equations around the car's estimated position at each millisecond, allowing it to fuse wheel encoders and GPS data to keep the tracking stable.

ELBO​

Evidence Lower Bound — a mathematical score used to train generative models like VAEs. It balances two goals: recreating the original input accurately, and keeping the model's internal representation organized. Think of it like packing for a trip: you want to bring everything you need (accurate recreation) but also pack it neatly so the suitcase closes easily (organized representation). The ELBO is the score that measures how well the model balances both tasks.

Elementwise operation​

An operation applied independently to each element of a tensor (e.g. add, multiply, ReLU), where output position i depends only on input position i.

Eligibility traces​

A short-term memory that lets a single TD error update many past states at once, instead of only the state where the error appeared. Each state carries a "trace" — a number that jumps up when the state is visited and then fades by a factor of γλ every step — and when a TD error arrives, every state is nudged in proportion to its current trace, so more recently visited states get more credit.

The decay parameter λ tunes the bias–variance balance continuously: λ = 0 gives plain one-step TD(0), λ = 1 reproduces Monte Carlo, and the method in between is called TD(λ). Think of this like diagnosing food poisoning at the end of the day: if λ = 0 (one-step TD), you only blame the very last thing you ate (a late-night mint); if λ = 1 (Monte Carlo), you blame everything you ate all day equally; and if λ is in between, you mostly blame the recent dinner, less so lunch, and barely blame breakfast.

Replacing traces cap a revisited state's trace at 1 rather than adding to it, which stops a state visited in a loop from earning unrealistically large credit. Picture a fading scent trail behind you: when reward finally appears, the freshest parts of the trail feel the strongest pull, but replacing traces ensures a spot you walked in a circle over doesn't smell overpoweringly strong, just "fresh."

Elo​

A rating system borrowed from chess that turns a series of head-to-head wins and losses into a single number per player: beat a strong opponent and your rating jumps, lose to a weak one and it drops. LLM arenas use it to rank chat models from pairwise comparisons instead of from a fixed-answer benchmark.

EMA weights​

Exponential moving average of model weights; samples better than the live weights

Embedding​

A dense vector that represents a token (or other item) so the model can compute over it; each token ID maps to one row of the embedding matrix

Embedding matrix​

The lookup table E ∈ ℝ^{V×d} that turns each token ID into a dense vector by selecting its row; growing the vocabulary means adding rows

Embedding space​

The shared multi-dimensional space that all embeddings live in, where every item is a point and direction and distance carry meaning — items that mean similar things sit close together and point the same way. Talking about the geometry of the embedding space means asking how those points are arranged: are the true image–caption pairs bunched into tight clusters, spread evenly over the unit sphere, or collapsed into one indistinct blob? Analogy: a city where related shops naturally form neighborhoods — you learn a lot about a model by inspecting the shape of its map, not just whether it gets answers right. In CLIP, a well-chosen temperature pulls matching pairs into tight clusters on the sphere while keeping mismatches pushed apart, so the geometry itself reveals how confidently the model separates right from wrong.

EMO​

Emote Portrait Alive — a specific talking-head model that turns a single portrait photo and an audio clip into a highly expressive, singing or talking video. Unlike older models that only move the mouth, EMO predicts motion directly without relying on intermediate 3D face models or facial landmarks.

Enormous on paper​

Describes a model like a Mixture-of-Experts (MoE) that has a massive total number of parameters (e.g., 100 billion), but only uses a small fraction of them (e.g., 10 billion) for any single word. Like a giant university with 5,000 courses listed in its catalog (enormous on paper)—no single student takes all 5,000 courses. Each student only takes a few classes at a time, so the cost per student remains low, even though the total catalog is huge.

Ensemble​

Training several models on the same task and combining their answers, instead of trusting a single one. The spread between the models is itself a cheap, useful signal: where they agree, the prediction is probably reliable; where they disagree, the input is likely outside what they have seen. In model-based RL an ensemble of dynamics models (as in PETS) is the standard way to estimate model uncertainty, so a planner can avoid actions whose outcomes the models cannot agree on. Analogy: asking five forecasters for tomorrow's weather — when all five say "rain" you pack an umbrella; when they split wildly you know the forecast itself is shaky.

Entropy regularization​

Adding a bonus for keeping the policy uncertain to the RL objective, so the agent does not lock onto one action too early. Entropy is a measure of how spread-out a probability distribution is — high when the policy gives several actions a real chance, near zero when it always picks the same one — and the trick adds β · H(π) to what the agent maximizes, where H(π) is the policy's entropy and β sets how strongly you reward indecision. This keeps the agent exploring instead of prematurely collapsing to a single habit that merely looked good early on. Analogy: a tourist who, for the first few days, deliberately tries a different restaurant each night rather than returning to the first decent one — the bonus pays them to keep sampling until they have really seen the options. PPO adds a small entropy term to its loss for exactly this reason; SAC takes the idea further and makes maximizing entropy a core part of its objective.

End-effector​

The working tip of a robot arm — the part that actually touches the world, such as a gripper, suction cup, welding torch, or camera mount. Almost every robot task is stated as a goal for the end-effector ("put the hand here, pointing this way"), and forward kinematics exists precisely to compute where the end-effector is from the joint angles. Analogy: if the arm is your shoulder-elbow-wrist chain, the end-effector is your hand — the only part the task really cares about; the joints are just how you get it there.

Episode​

A single, complete sequence of interactions between an agent and an environment, starting from an initial state and ending when a terminal state is reached (such as winning a game, dying, or running out of time). Analogy: Think of an episode like a single game or match of a sport. For example, in baseball, a single game from the first pitch to the final out is one episode. Once the game ends, the score is settled, the field is reset, and the next game starts fresh. Example: In training an agent to solve a maze, an episode begins when the robot is placed at the starting line. The episode ends when the robot either reaches the exit (success) or hits a trap (failure). The entire sequence of steps taken in that single attempt is one episode.

Epoch​

One full pass through the entire training dataset. If a dataset has 3,000 examples and the batch size is 64, one epoch is about 47 optimizer steps, and "training for 6 epochs" means the model saw every example 6 times (usually in a different shuffled order each pass). Small datasets are often trained for many epochs; giant pretraining corpora frequently get less than one — there is more text than the compute budget can read.

Epistemic uncertainty​

The part of a model's uncertainty that comes from not having seen enough data — the model's own ignorance, as opposed to noise inherent in the world (aleatoric uncertainty). Unlike aleatoric noise, it shrinks as you collect more data in that region. The standard way to measure it is an ensemble: train several models on different samples of the data and see how much they disagree. Where the data was thick they will agree; where it was thin they will confidently contradict each other, and that spread is the estimate. This is what stops a planner from happily routing a robot through a region where the model has hallucinated free reward — the members do not agree that the reward is there, so the average washes it out. Crucially, it needs no ground truth to compute, which is exactly why it is usable at decision time, when the truth is precisely what you do not have.

Epsilon-greedy​

The simplest way to balance exploration and exploitation, usually written ε-greedy (ε is the Greek letter epsilon, standing for a small probability). With probability 1 − ε the agent acts greedily — it picks the action its current estimates rate best — and with probability ε it instead picks a uniformly random action, just to keep trying options it might be undervaluing. ε is normally annealed (decayed) from near 1 down to a small floor over training, so the agent explores wildly at first and increasingly exploits what it has learned. It is "good enough" when rewards are dense but fails badly when reward is rare, because random jitter almost never stumbles onto a far-off goal.

Error budget​

The small amount of failure an SLO allows. If your target is 99.9% success, the remaining 0.1% — about 43 minutes a month — is your error budget. Like a monthly data allowance on a phone plan: you can "spend" it on risky deploys and experiments, but once it runs out you stop taking risks until it resets. It turns reliability from a vague goal into a balance you can watch.

Euler angles​

A way to describe a 3D orientation as three separate rotation angles — usually called roll, pitch, and yaw (or sometimes labeled with the axes they rotate around, such as ZYX or XYZ, depending on the convention used) — applied one after another around chosen axes. Each angle tells you "how much did the object spin around this axis?" in a specific order: for example, "yaw 30° left, then pitch 20° up, then roll 10° clockwise." Analogy: like giving someone driving directions using three turns in order — "turn left 30°, then tilt up 20°, then spin clockwise 10°" — but if you change the order of the turns you end up facing a completely different direction. That order-dependency is the headline weakness: "pitch then yaw" reaches a different final orientation than "yaw then pitch," making Euler angles easy to mix up in code. The deeper failure is gimbal lock: at certain orientations, two of the three axes collapse onto the same line and one degree of freedom silently vanishes. For these reasons, robotics and 3D graphics store orientations as quaternions or rotation matrices internally, and convert to Euler angles only when displaying them to a human.

Euler method​

The simplest way to numerically solve a differential equation: look at the slope where you currently are, take one straight-line step in that direction, then repeat. It is easy to implement but accumulates error quickly because it ignores how the slope changes during the step, so diffusion samplers built on it need many steps to stay accurate. Like steering a car by only ever looking at the road directly under the bumper. Contrast with Heun's method and DPM-Solver, which correct for the changing slope and so need far fewer steps.

Evaluation harness​

A software framework that automates the testing and scoring of a system against a suite of benchmark tasks under standardized conditions. In language and vision-language modeling, common harnesses (like lmms-eval or VLMEvalKit) run a model through prompts and parse the multiple-choice or text answers to produce comparable accuracy tables across models. In robotics, an evaluation harness runs policy or control code across dozens of simulated tasks (such as a 50-task suite with seeded variations in physics, object positions, and initial conditions) to automatically compute a statistical pass-rate dashboard. Analogy: A standardized testing center that hands every student the same exam paper and grades it with the same rubric, ensuring the results are directly comparable, rather than letting teachers write and grade their own quizzes. This continuous evaluation is crucial for catching behavioral regressions in robotic systems before deploying updates to physical hardware.

ExecuTorch​

PyTorch's lightweight runtime for running models on mobile and edge devices, built on the graph captured by torch.export.

Experience replay​

A buffer (also called a replay buffer) that stores an agent's past (state, action, reward, next-state) transitions, so training can draw random mini-batches from it instead of learning only from the latest step. This does two things at once: it breaks correlation — consecutive steps within an episode are highly similar, and a network trained on them in order tends to overfit to whatever it is doing right now — and it reuses data, letting each expensive real-world transition feed many gradient updates rather than being discarded after one. It is one of the two tricks (with the target network) that made DQN stable enough to learn Atari from pixels; prioritized experience replay extends it by sampling surprising transitions more often. Analogy: a student who reshuffles flashcards from the whole term, rather than only rereading this morning's notes, remembers the material far better.

Expectile regression​

A twist on ordinary least-squares regression that fits an asymmetric target instead of the mean. Normal regression punishes overshooting and undershooting equally, which lands you on the average; expectile regression weights one side more — a τ (tau) close to 1 punishes under-prediction harder, so the fit is pulled up toward the higher values in the data, like a τ = 0.9 line that tracks the "better-than-usual" outcomes rather than the typical one. IQL uses it to estimate the value of the best action seen in a state without ever evaluating an action outside the dataset: by leaning the value function toward the top returns the data already contains, it approximates a max over in-distribution actions while sidestepping the out-of-distribution blow-up. (An expectile is the squared-error cousin of a quantile, which uses absolute error; τ = 0.5 recovers the plain mean.) Analogy for expectile regression: Imagine trying to estimate a runner's typical "good" race time. Instead of taking their absolute fastest record (which might be a fluke) or their overall average (which includes days they were sick), you weigh their better performances much more heavily than their poor ones. This gives you a reliable estimate of their upper potential without being thrown off by a single lucky outlier.

Explained variance​

A single number, from 0 to 1 (sometimes negative), for how well a predictor tracks the true values: 1 − Var[target − prediction] / Var[target]. 1.0 means the predictions perfectly track every up and down in the target; 0.0 means the predictor is no better than always guessing the average; a negative score means it is actively worse than that flat guess. In actor-critic methods it is the standard health check for the critic: does V(s) actually explain the ups and downs in the returns it is trying to predict, or is it just outputting noise? Analogy: a weather forecaster's explained variance is how much of the day-to-day temperature swing their forecast captures — a forecaster who just says "average for this month, every day" scores 0.0; one who is actively misleading (says "hot" on the cold days and vice versa) can score below 0.0.

Expert​

In a Mixture-of-Experts (MoE), one of several parallel MLP sub-networks; a router sends each token to only the top few experts instead of all of them. Like a hospital triage desk that routes each patient to the right specialist rather than making everyone see every doctor — lots of expertise on hand, but only a little used per case.

Expert parallelism (EP)​

For MoE models, distributing experts across GPUs with all-to-all token routing

Exploration vs exploitation​

The core tension in RL: should the agent exploit — take the action it currently believes is best — or explore — try something uncertain that might turn out better? Pure exploitation can lock onto a mediocre habit because the agent never gathers the evidence that a better option exists; pure exploration never cashes in what it has learned. It is like always reordering your favorite dish versus occasionally trying something new on the menu — you need a mix of both. Epsilon-greedy, softmax (Boltzmann) action selection, and UCB (which adds an "optimism" bonus to rarely tried actions) are common ways to strike the balance.

Exponent​

The part of a floating-point number that records its scale — how many places to shift the decimal point. In scientific notation like 3.5 × 10¹², the 12 is the exponent (using base 10 instead of base 2). More exponent bits give a wider range of representable magnitudes, from astronomically large to vanishingly small; fewer exponent bits mean values overflow or underflow more easily. This is why FP8 has two flavors: E5M2 (5 exponent bits) for gradients that can swing wildly in size, and E4M3 (4 exponent bits) for activations that stay in a tighter range. See also mantissa.

Extrapolation​

Extending a learned pattern beyond the range of values seen during training — working out what comes outside the examples, rather than filling a gap between two of them (that in-between case is interpolation). A model that extrapolates gracefully keeps behaving sensibly on inputs that are larger, longer, or differently shaped than anything in its training data. This is the whole reason RoPE encodes position as a rotation: a rotation angle is defined for any position, including ones past the longest sequence the model ever trained on, so a model trained on short clips can still place a token at a never-seen position and generate longer videos at variable resolution. Contrast a fixed lookup table of learned position vectors, which simply has no entry for a position it never saw. Like a tide chart built from one month of readings: interpolating tells you the water level at a moment between two marks, while extrapolating predicts next week's tide — beyond every reading you have.

FCFS​

First-Come, First-Served — the simplest scheduling rule: handle requests in the exact order they arrive, like a single queue at a bakery where nobody can skip ahead. It is fair and easy to build, but it has no sense of deadlines, so one slow request at the front can make everyone behind it late.

FFN​

Feed-Forward Network — the small MLP inside each transformer block. Position-wise means it is applied to each token (each position in the sequence) on its own, reusing the same weights at every position — like one cashier serving each customer in line one at a time at the same till, never letting them interact. That is the opposite of the attention sublayer, where tokens do look at each other; the FFN just lets each token "think" by itself.

Factor graph​

Factor graph is a graphical model used to represent a complex probability distribution by breaking it down into a product of simpler functions called factors. In robotics and SLAM, it is represented as a bipartite graph with two types of nodes: variable nodes (representing the robot's states, such as positions at different time steps) and factor nodes (representing sensor measurements or constraints, such as wheel odometry, GPS readings, or loop closures). Solving the factor graph means finding the set of variables that maximizes the likelihood of all factors, which is mathematically equivalent to solving a sparse nonlinear least-squares optimization problem. This global smoothing approach is more accurate than filtering because it maintains the history of states and can correct past errors when loop closures are detected.

Analogy: A group of people trying to agree on a schedule. Each person represents a "factor" with specific constraints (e.g., "I must meet Alice between 2 PM and 4 PM", "I need 1 hour to travel from Bob's office"). Instead of scheduling meetings one by one and risking a deadlock, you lay out all the people and constraints on a board and solve them together to find the single schedule that satisfies everyone as best as possible.

F/T sensor​

Force/Torque sensor — six-axis force and moment at a wrist or fingertip

Farnebäck optical flow​

A classical (non-neural) algorithm for computing dense optical flow, named after its inventor Gunnar Farnebäck. It estimates motion by approximating the brightness around each pixel with a small quadratic (a smooth curved surface) in both frames and solving for the shift that lines them up. Analogy: it slides a tiny transparent patch of the first frame around the second until it clicks into place, and records how far it had to move. It is fast and needs no training, but it is less accurate on large or blurry motion than a learned model like RAFT. Example: OpenCV's cv2.calcOpticalFlowFarneback returns a (H, W, 2) array giving each pixel's left–right and up–down movement.

Feature​

A distinct, individual pattern, shape, or characteristic extracted from raw data (like an edge in a photo, or the tense of a verb in a sentence) that a model uses to make sense of the input. In deep learning, instead of humans choosing which features to look for, the network learns to detect them automatically. As the signal goes deeper, simple features (like lines and curves) are combined into high-level features (like eyes, wheels, or faces). These learned features are represented mathematically as lists of numbers called embeddings in an embedding space. Analogy for features: When identifying a fruit, you don't look at every microscopic cell; you look at features like color (red), shape (round), and texture (smooth) to recognize it as an apple. Neural networks do the same, converting complex raw pixels into a few key features.

Feature space​

The multi-dimensional space where feature representations of data (such as embeddings or hidden activations) live, rather than raw inputs like pixels. In deep reinforcement learning, a controllable feature space is a feature space specifically learned to retain only the features of the environment that the agent's actions can directly affect or influence, discarding uncontrollable background elements. Analogy: Imagine a self-driving car. Instead of analyzing every raw pixel of the camera feed (like the pattern of leaves on a tree or clouds in the sky), the car's system processes the image into a "feature space" containing only relevant, actionable items: lanes, pedestrians, traffic lights, and other vehicles. Why it matters: It makes learning and prediction much more efficient. In curiosity-driven exploration, predicting in a controllable feature space (as in ICM) prevents the agent from getting trapped by unpredictable, random background details (the noisy-TV problem) because those details are filtered out of the feature space entirely.

Feedback control​

Controlling a system by continuously measuring what it is actually doing, comparing that to what you wanted (the setpoint), and correcting based on the difference — the "closed loop" of measure → compare → act → measure again. Its opposite is open-loop control, which applies a precomputed command and simply hopes the world cooperates: open-loop is fine when nothing disturbs the system (a microwave runs its timer blind) but fails the moment reality drifts from the plan. Feedback control is the entire discipline of classical control, and almost every controller in this guide — PID, LQR, impedance control — is one recipe for turning the measured error into a correcting command. Analogy: steering a car by watching the lane and nudging the wheel (feedback) versus fixing the wheel at one angle, closing your eyes, and praying (open-loop). It is most powerful when paired with feedforward, which predicts most of the needed command in advance so feedback only has to fix the small residual.

Feedforward control​

Computing most of the command a system needs ahead of time from a model of how it works, rather than waiting for an error to appear and reacting to it. A pure feedback controller is always one step behind — it can only correct a deviation after it has already happened — whereas feedforward predicts the bulk of the effort up front (using the known target motion and the system's dynamics) and lets feedback clean up only the small leftover error. In a robot arm this means using inverse dynamics to pre-compute the torque a planned trajectory demands (computed-torque control); in a motor it can mean adding a friction-compensation term that cancels known drag before it slows the joint. Analogy: leaning into a turn before you feel yourself sliding, because you know the corner is coming — instead of catching the slide after it starts. The two are complementary, not rival: feedforward handles the predictable part, feedback handles the surprises.

FID​

Fréchet Inception Distance — the standard sample-quality metric for image generation. ("Fréchet," after the mathematician Maurice Fréchet, names the Fréchet distance: a way to measure how far apart two probability distributions sit.) It runs both real and generated images through a pretrained Inception network to turn each image into a feature vector, then measures how far apart the two clouds of features sit by comparing their means and covariances (their centers and spreads). A lower FID means the generated images look statistically more like the real ones — picture two overlapping clouds of dots: the more they overlap, the smaller the distance. The real images here are only a yardstick, not an ingredient: your model invents brand-new images from random noise and never copies the real ones — FID simply needs a pile of real photos to compare those inventions against so it can score how convincing they are.

Fiducial marker​

A pattern — usually a flat printed tag — placed in a scene on purpose to act as an easy, unambiguous reference point for a camera. Unlike natural objects, it is engineered to be detected reliably and to encode an ID, so the robot knows exactly which marker it sees and, because the marker's real size and shape are known, exactly where it sits. The AprilTag and ArUco families are the common ones in robotics. Analogy: the small registration crosses a printer prints in a page's corners so a machine can line the sheet up perfectly — a mark whose only job is to be found precisely.

FILM​

FILM (Frame Interpolation for Large Motion) is a neural frame-interpolation model from Google that, given two real frames, generates the frames in between — and it is specifically built to cope when objects move a long way between the two shots, the case where older methods smear or tear. It estimates motion at several scales at once (a coarse pass catches big jumps, finer passes catch small ones) and warps both frames toward the middle before blending them. Think of an animation assistant who can fill in the missing "in-between" drawings between two key poses even when the character has leapt clear across the scene. It is a convenient pretrained model for seeing, firsthand, the artifacts that fast motion produces.

Filterbank​

A stack of band-pass filters that each measure how much energy a signal carries in one narrow frequency range — together they split a sound into a set of frequency "buckets." A mel filterbank is the specific set used to build a mel spectrogram: commonly 80 filters, each shaped by triangular weights and spaced on the perceptual mel scale, all stored as one fixed matrix. Applying it is a single matrix multiply that collapses the STFT's hundreds of evenly spaced frequency rows down to a handful of mel bands. Like a row of differently tuned wine glasses, each ringing only for the note near its own pitch: play a chord and you can read off how much of each note is present from how loudly each glass hums. Example: a 1024-point FFT produces ~513 frequency values per frame; multiplying by an 80×513 mel filterbank matrix turns each time frame into just 80 numbers.

Fine-tuning​

Taking a model that was already trained on a huge dataset and training it a little further on a small, specific dataset so it picks up a new skill, subject, or style. The big initial training is expensive and done once; fine-tuning is cheap and reuses all that knowledge — like hiring an experienced cook and teaching them your three house recipes rather than training someone from scratch. In image generation you might fine-tune Stable Diffusion on 20 photos of your pet so it can draw that specific pet. Fine-tuning can update every weight (as in DreamBooth) or just a tiny added piece (as in LoRA); the less you change, the smaller and more shareable the result, at some cost in how much new behavior you can absorb.

FineWeb-Edu​

A large, openly released pretraining dataset built by running a quality filter over crawled web pages and keeping only the educational-looking ones — like skimming a huge pile of internet text and saving just the pages that read like a textbook. Models trained on it often beat models trained on far more unfiltered text, making it a go-to example that data quality can matter more than raw quantity.

Finite difference​

A way to estimate a derivative (a rate of change) without doing any calculus, by nudging the input a tiny amount and watching how the output changes: (f(x + h) − f(x)) / h for a small step h. It answers "if I move this knob a hair, how fast does the result move?" using only the function itself, which is why it is the universal sanity check for any analytic derivative you code by hand — including the Jacobian, where you wiggle each joint by a tiny angle and divide the end-effector's resulting motion by that angle. The trade-off is choosing h: too large and the straight-line approximation is crude; too small and floating-point rounding swamps the tiny difference.

Fixed comb​

In distributional reinforcement learning (like C51), a metaphor for the set of support values (called atoms) used to represent the probability distribution of possible returns. Instead of learning a single expected value, the algorithm uses a fixed, discrete set of evenly spaced return values (resembling the teeth of a comb) and learns a probability for each. When updating, the teeth of the comb are shifted by rewards, and these shifted values must be projected (re-allocated) back onto the original, unshifted teeth.

FK / IK​

Forward / Inverse Kinematics — compute end-effector pose from joints or vice versa

Fisher information​

A matrix that measures how sharply a probability distribution changes when you nudge its parameters — formally, the curvature (second derivative) of the KL divergence between the distribution and itself, in every parameter direction at once. Directions where the Fisher is large are directions where a small parameter change swings the distribution a lot; where it is near zero, the parameters can move freely without the distribution noticing.

  • Why it matters in RL: It supplies the notion of distance that TRPO and the natural gradient are built on. An ordinary gradient step measures "how far did I move" in units of the parameters, which is meaningless — the same policy can be written with wildly different weights. The Fisher instead measures distance in units of behavioural change, which is what actually matters: an update that shifts the weights a great deal but the action probabilities barely at all is a small step, and should be allowed to be large.
  • How it works: For a policy it can be estimated from the same batch of states already collected, and — crucially — it is never formed explicitly. Only its product with a vector is needed, which is obtainable by conjugate gradient plus a double backward pass.

Flamingo​

DeepMind's 2022 vision-language model that pioneered gated cross-attention: it leaves a big pretrained language model entirely frozen and inserts brand-new cross-attention layers between its blocks so the text can look at image features. The clever part is the gate — a learned multiplier that starts at exactly zero, so on the very first training step the new layers contribute nothing and the model behaves identically to the original language model, then the gate slowly opens as training teaches it how much image information to let in. Like adding a new water line to a working house but keeping its valve shut until you have checked every joint, then easing it open. This "don't break what already works, blend the new capability in gradually" trick is why Flamingo could bolt vision onto a frozen LLM without destabilizing it, and it became a template later VLMs copied. The projector-only approach of LLaVA is the simpler rival design.

FlashAttention​

A much faster way to compute attention that never writes the giant token-by-token score table to slow HBM memory. Plain attention builds the full T × T grid of how strongly every token attends to every other token, parks it in HBM, then reads it back — a flood of slow memory traffic. FlashAttention instead works on small tiles inside the chip's fast on-chip memory (SRAM) and keeps a running total, so the huge grid never has to be stored at all. Like adding up a long column of numbers in your head as you go instead of writing every subtotal on paper — same answer, far fewer trips to the slow notebook. Every modern inference engine relies on it.

FlashDecoding​

A version of FlashAttention tuned for the decode step, where there is just one new query token but a long KV cache to read. It splits that long read across many GPU workers so the HBM bandwidth stays fully used instead of one worker plodding through the cache alone — the trick that lets engines like vLLM hit near-peak bandwidth on decode-heavy traffic.

float16​

16-bit floating-point format (fp16); saves memory and can be fast on GPUs, but has a limited range (max ~65,504) that can cause underflow when accumulating very small values

float32​

32-bit floating-point format (fp32); the standard default precision for PyTorch tensors — wide enough range and enough precision for most training and inference tasks

FLOPs​

Floating-Point Operations — a count of the individual arithmetic steps (additions and multiplications on decimal numbers) a model performs, used as a hardware-independent measure of how much compute one forward pass costs. You estimate it by adding up the work in each layer: a matrix multiply of an M×K matrix by a K×N one, for instance, takes about 2·M·K·N FLOPs (each of the M·N outputs needs K multiplies and K adds). Like counting the total pencil strokes a calculation requires, regardless of how fast the person writing them is. More FLOPs means a slower, costlier model — exactly the price you pay when a ViT uses smaller patches. (Note: "FLOPs" = operations; "FLOP/s" with a slash = operations per second, a speed.)

Flow matching​

Training a velocity field — a model that, given a half-noisy image and a time, predicts which direction and how fast to move it toward a clean image — so that following those arrows turns pure noise into data. Concretely, you draw a straight line between a real image x_0 and random noise ε, pick a random point on that line, and train the model to output the line's direction ε - x_0; at generation time you start at noise and repeatedly step along the predicted arrows (solving an ODE) until you arrive at a clean image. It is a simpler, more modern alternative to DDPM: there is no noise schedule to tune, just one clean regression target. Like learning the wind currents over a map so that, dropped anywhere in the fog, you always know which way blows toward home.

Flux​

A family of state-of-the-art open-weight text-to-image models released in 2024 by Black Forest Labs (a team that included original Stable Diffusion researchers). Flux is built on a large MMDiT backbone trained with rectified flow, so text and image tokens share the same attention layers and the model denoises along nearly straight paths — which is why it follows detailed prompts and renders legible text unusually well. It ships in a few flavors: a top-quality "pro" version, an open "dev" version for tinkering, and a distilled "schnell" (German for fast) version that trades a little quality for very few sampling steps. Think of it as the generation of image models that arrived just after SD3 and pushed open-weight quality a notch higher.

FPGA​

A Field-Programmable Gate Array (FPGA) is an integrated circuit designed to be configured by a customer or a designer after manufacturing — hence "field-programmable."

  • Why it matters: Unlike general-purpose CPUs or custom ASICs, which have permanent, unchangeable circuits, an FPGA's hardware structure can be redesigned and flashed in seconds using software. This makes them highly valuable for prototyping custom hardware designs, experimenting with new neural network architectures, or deploying edge accelerators without the massive cost and risk of fabricating custom silicon chips.
  • How it works: An FPGA contains an array of programmable logic blocks (like look-up tables) and a hierarchy of reconfigurable interconnects. The designer writes hardware description code (like Verilog, VHDL, or high-level synthesis HLS in C++) that defines how these blocks should connect. When compiled into a bitstream and loaded onto the FPGA, the chip physically reorganizes its internal electrical connections to mimic the desired hardware.
  • Analogy: Imagine a blueprint for a Lego house. A custom ASIC is like a plastic toy castle that was molded in a factory; it is solid, durable, and highly efficient at being a castle, but you can never change it. An FPGA is like a giant bucket of Lego bricks with an instruction sheet: you can assemble it into a castle, play with it, and if you later decide you need a spaceship instead, you can take it apart and rebuild it using the same bricks.
  • Example: Implementing a small convolutional neural network (CNN) on an FPGA development board like the PYNQ-Z1. The FPGA is programmed to act as a dedicated matrix-multiply pipeline, accelerating the model's inference at the edge with lower latency than a standard CPU.

Footstep planning​

A motion planning paradigm for legged robots (like bipeds or quadrupeds) where, instead of planning continuous body trajectories, the planner first plans a discrete sequence of footstep locations. The planning problem is formulated as a search over a footstep lattice (a graph where nodes are foot positions and edges are valid stepping actions), typically solved using heuristic search algorithms like A* search. Analogy: Imagine crossing a shallow river by stepping on a series of wet stepping stones. You don't think about the exact trajectory of your hips first; you look at the stones and decide a sequence of left-foot and right-foot steps to get to the other side. Example: A humanoid robot walking up a flight of stairs or over rough rubble. The footstep planner searches for a sequence of stable foot placements on the stairs, after which a whole-body controller calculates the joint motions to execute those steps.

Forensics​

Working backward from a training failure to the operation that first caused it, instead of chasing the visible symptom. In PyTorch this means turning on autograd anomaly detection to halt at the first NaN or bad gradient.

FP4​

4-bit floating point — half the bits of FP8 again, so a weight takes a quarter of the space of bfloat16. With only 4 bits there are just 16 possible values, so it sits near the edge of usable precision and needs careful checking; newer Blackwell GPUs accelerate it in hardware, making it attractive for squeezing huge models onto fewer chips.

Force closure​

A fundamental property of a robotic grasp where the contact forces applied by the gripper can resist any arbitrary external force or torque (known as a wrench) acting on the object, assuming no slipping. It is a key metric in analytic grasping and contact mechanics, typically modeled using friction cones at the contact points.

  • Analogy: Think of holding a heavy, slippery book. If you squeeze it tightly between your palms from both sides, the friction is strong enough that if someone tries to pull, push, or twist the book, your hands keep it locked in place. Your grip has achieved force closure because no matter how they yank it, it won't budge. If you held it loosely with just your fingertips, they could easily pull it out of your grip.
  • How it is calculated: A grasp achieves force closure if the convex hull of the contact wrenches spans the entire 6-dimensional wrench space R^6 (or R^3 in 2D space), meaning any external disturbance wrench can be balanced by a combination of feasible contact forces lying within their friction cones.

Force control​

Controlling a robot by regulating the force it applies, rather than the position it reaches. Position control answers "be at this spot" and will push with whatever force it takes to get there — fine in free space, dangerous on contact, because a tiny position error against a hard surface turns into a huge force (the arm tries to drive through the table). Force control instead answers "press with this much force," which is exactly what contact tasks need: keeping a polishing pad, a pen, or a peg in firm but gentle contact with a surface whose exact shape you do not know. In practice it is usually achieved through impedance control, and the common pattern is hybrid — command position in the directions you know the geometry and force in the directions you do not. Analogy: a doctor taking your pulse presses to a feel, not to a fixed depth; pressing to a preset depth would either miss your wrist or bruise it.

Forward hook​

A callback registered on an nn.Module that PyTorch calls automatically after the module's forward pass, receiving the input and output tensors; used for capturing activations and debugging

Forward kinematics​

The calculation that takes a robot's joint angles and returns where its end-effector ends up — position and orientation — in some reference frame. You build it by chaining together one homogeneous transform per joint, multiplying from the base outward, so it is always solvable and always fast. "Forward" because it runs in the natural cause-to-effect direction (joints → hand pose); the reverse problem, hand pose → joints, is inverse kinematics and is much harder. Analogy: forward kinematics is reading the time off a clock from how far each gear has turned.

Forward model​

A learned network that predicts the next state given the current state and action — in other words, the agent's internal simulator of "what happens if I do this?". Forward models power model-based RL (plan by imagining rollouts) and curiosity-style exploration like the ICM, where the error of the forward model — how surprised it is by the real next state — becomes an intrinsic reward. Like a chess player picturing the board after a move before actually making it; the more the real outcome differs from the mental picture, the more there was to learn.

Forward pass​

One complete run of an input through the whole network — every layer in order, from the first to the last — to produce an output (for an LLM, the logits for the next token). It means start-to-finish through all the layers, not a single layer. Like running a part down an entire assembly line once to get the finished product. The reverse direction, used in training to compute gradients, is the backward pass.

Fourier transform​

A math tool that takes a signal that changes over time — like a sound wave — and reveals which pure frequencies (pitches) it is secretly built from, and how much of each. Like a glass prism splitting white light into its rainbow of colors, the Fourier transform splits a messy sound into the simple sine-wave "tones" hidden inside it. Concrete example: feed it a recording of a piano chord and it answers "this is mostly 262 Hz (middle C) + 330 Hz (E) + 392 Hz (G)". How it works: it slides every candidate frequency past the signal, multiplies the two together point by point, and adds up the products (a dot product); when a test frequency really is present the bumps line up and the sum comes out large, and when it is absent the products cancel to near zero — so a big result means "yes, that pitch is in here." The catch is that it tells you which frequencies are present across the whole clip but not when each one happened, which is exactly why the STFT runs it on short overlapping slices instead. It is named after Joseph Fourier, who showed that any repeating signal can be rebuilt by adding up enough simple sine waves.

Foxglove​

An open-source visualization and debugging tool designed specifically for robotics systems. It allows developers to build custom interactive dashboards to visualize 3D scene environments, camera feeds, coordinate transforms, joint states, plots of system metrics, and log messages in real time or by playing back recorded files (like MCAP). Analogy: An advanced flight-telemetry dashboard for a drone or racecar, showing live camera feeds, battery levels, speed graphs, and a 3D model of the vehicle's position, letting engineers diagnose issues instantly rather than parsing raw columns of numbers. This tool is highly valued because it works out-of-the-box with standard formats like ROS 2 and MCAP, making it the primary window into a robot's internal state.

FP8​

8-bit floating point — half the bits of bfloat16. Comes in two flavors: E4M3 (4 exponent bits + 3 mantissa bits) keeps a bit more precision and is used for weights and the forward activations; E5M2 (5 exponent + 2 mantissa) trades precision for a wider range and is used for gradients, which can be very large or very small. Supported by Hopper and later NVIDIA GPUs, it is rapidly becoming the modern default serving precision.

Fragmentation​

Memory wasted in gaps too small to reuse, left behind when each request is given its own contiguous chunk — like a parking lot full of single empty spaces where no bus can fit even though there is plenty of total room. Paged schemes such as PagedAttention avoid it by handing out small fixed-size pages instead of one big block per request.

Frame interpolation​

Generating new frames between two existing ones to make motion smoother or a clip slower — turning, say, 24 frames per second into 60. It is sometimes called "video generation lite" because the model only has to invent the short motion between two anchors it can already see, not a whole scene from nothing. The classic analogy is hand-drawn animation: a lead artist draws the key poses and an assistant fills in the "in-between" frames — the industry literally calls this inbetweening. Modern neural versions such as FILM and Super SloMo estimate how each pixel moves between the two frames (closely related to optical flow) and warp the images toward the midpoint.

Frame rate (fps)​

How many still frames a video shows per second — "fps" stands for frames per second (e.g. 24, 30, 60). It sets how much real-world time sits between two neighboring frames, so the same motion looks bigger and choppier at low fps and smoother at high fps. Analogy: a flipbook drawn with 12 pages per second looks jerky; the same drawings at 60 pages per second look fluid. Example: sampling 16 frames evenly from a 2-second clip at 8 fps covers the whole clip, but grabbing 16 consecutive frames from a 60-fps clip covers only a quarter-second — so a model must be told which fps it is seeing.

Frame stacking​

Feeding a DQN several consecutive game frames at once (classically the last four) as a single multi-channel input, rather than one frame at a time. A single still image of Pong tells you where the ball is but not where it is going — speed and direction only become visible by comparing frames across time. Without that motion information the problem stops obeying the Markov property (the current observation no longer holds everything needed to act well), so stacking restores it cheaply. Analogy: judging a thrown ball's path from a single photo is impossible, but a short burst of photos makes the trajectory obvious.

Friction compensation​

Adding an extra, model-based torque to a motor that cancels the friction in its joint before friction has a chance to slow it down — a feedforward trick that simulators rarely need but real hardware almost always does. Real joints lose force to two effects a perfect model ignores: viscous friction, a drag that grows with speed, and stiction (static friction), an initial resistance that must be overcome before the joint will move at all. You measure how much torque each effect eats — typically by slowly driving the joint and recording the torque at which it breaks free and how torque rises with velocity — then build a small lookup or formula and add that torque to every command, so the motor's net output matches what the controller intended. Analogy: a cyclist who knows the chain is a bit rusty and pedals a little harder from the very first push, instead of waiting to feel the bike lag and then catching up. It is often the single change that turns a sloppy real arm into a precise one.

Friction cone​

A geometric representation of the limits of contact force that can be applied between two touching surfaces without slipping, based on the Coulomb friction model (F_t ≤ μF_n, where F_t is tangential friction force, F_n is normal force, and μ is the friction coefficient). The cone is formed with the normal vector at the contact point as its axis, and its half-angle α is determined by tan(α) = μ; any contact force vector falling inside the cone is resisted by static friction, whereas any force vector falling outside would cause the surfaces to slip.

  • Analogy: Imagine pushing a heavy wooden box on the floor. If you push straight down (normal force), the box doesn't move. If you push at a slight angle, it still doesn't slide because static friction balances your sideways push. But if you push at a very steep angle (outside the friction cone), the box immediately slides. The friction cone defines that boundary of angles.
  • How it is used: In grasp synthesis and contact mechanics, a grasp achieves force closure if the line of contact forces between fingers passes through the friction cones of the contact points, meaning the gripper can squeeze the object without slipping.

Frontier run​

A training run for one of the largest, most capable models at the leading edge of what is currently possible — the kind that ties up thousands of GPUs for weeks and costs millions of dollars. Because the stakes are so high, a loss spike that cannot be recovered cleanly can throw away days of that compute, which is why teams rehearse checkpoint recovery on small models first.

Free bits​

A fix for a specific failure of latent-variable models: the KL term in the loss keeps pushing the latent's distribution toward its prior, and the cheapest way to satisfy it is for the latent to stop carrying any information at all (posterior collapse). Free bits says: ignore the KL entirely while it is below some threshold (say 1 nat), and only start penalizing it above that. The model gets that much information "for free" and stops being punished for using it. Analogy: a budget with an allowance — you are not nagged about spending until you exceed it, so you actually spend what you need. Used by DreamerV3, among others.

Frozen​

A layer or whole sub-network is frozen when its weights are held fixed during training — the optimizer is told to skip them, so no gradients update them — while other parts of the model keep learning. Like renovating one room of a house while the rest stays sealed off and untouched. Freezing is how you reuse an expensive pretrained component (a CLIP image encoder, a big language model) as a fixed feature extractor and train only a small new piece — a projector, an adapter, or a LoRA — on top: it saves memory and compute and protects the pretrained knowledge from being overwritten by a small new dataset. The opposite is leaving a part trainable (or "unfrozen"), as fine-tuning does.

FrozenLake​

A small gridworld environment in Gymnasium (FrozenLake-v1) where the agent crosses a frozen lake from start to goal without falling into holes. In its default slippery mode the ice is stochastic: the chosen direction only happens part of the time, and the agent often slides to a perpendicular square instead — which makes the transition probabilities genuinely random and the problem a real MDP rather than a deterministic maze. Because its dynamics are small and fully known, it is the standard first testbed for both value iteration (planning with the model) and Q-learning (learning without it).

F.scaled_dot_product_attention​

PyTorch's built-in fused attention function (in torch.nn.functional) that computes softmax(QKᵀ/√d)·V in a single call, dispatching to an optimized backend such as FlashAttention.

FSDP​

Fully Sharded Data Parallel — shard params, grads, and optimizer state across ranks

FSQ​

Finite Scalar Quantization — a way to make discrete image tokens without a learned codebook. Instead of looking up the nearest entry in a trained table, it simply rounds each coordinate of the latent to the nearest value on a fixed grid, like snapping every measurement to the nearest tick on a ruler. Because there is nothing to train in the quantizer, it is simpler and sidesteps codebook collapse, yet stays competitive with VQ-VAE.

Function approximation​

Using a parameterized model — usually a neural network — to estimate a value function or policy instead of storing one number per state in a table. Tables only work when states are few and discrete; real problems have continuous or astronomically many states (an Atari screen has more possible images than there are atoms in the universe), so you must generalize from the states you have seen to ones you have not. The network learns features that let nearby states share what they have learned — the same reason a child who has seen a few dogs recognizes a new breed. The catch: combined with bootstrapping and off-policy data it forms the deadly triad, the trio that can make training diverge, which is exactly why DQN needs its stabilizing tricks.

Function calling​

The mechanism by which a model uses a tool: it emits a structured request (such as JSON naming a function and its arguments), an external program runs that request, and the result is handed back to the model. Also called tool use.

Fusion (early/middle/late)​

Where in the network the information from different modalities is combined. Late fusion encodes each modality fully on its own and only compares the two finished embeddings at the very end (CLIP matching an image vector to a text vector). Middle fusion encodes each separately but then lets one stream attend to the other partway through, usually with cross-attention (a VLM feeding image features into a language model). Early fusion turns every modality into one shared stream of tokens from the very start and runs a single model over the mix (native multimodal models like Chameleon). Think of three ways to combine a recipe's flavors: stir two finished sauces together at the table (late), blend them while each is still simmering (middle), or throw every raw ingredient into one pot from the beginning (early). The earlier the fusion, the more freely the modalities can shape each other — but the more compute and data it takes to train.

Future frame prediction​

The task of, given the first few frames of a video, predicting the frames that come next — an early benchmark for whether a model has learned how things move. It is the video cousin of next-word prediction in language: the model is trained to continue a sequence it has only partly seen. The classic toy benchmark is Moving MNIST and the classic baseline architecture is the ConvLSTM. Because the future is genuinely uncertain, a simple model trained with mean squared error tends to hedge by blurring — averaging all the plausible futures into one fuzzy guess rather than committing to a single sharp one.

FVD​

Fréchet Video Distance — the standard automatic quality metric for video generation, and the direct extension of image FID to clips. ("Fréchet," after mathematician Maurice Fréchet, names the Fréchet distance between two probability distributions.) Like FID, it runs both real and generated videos through a pretrained network — here a video-understanding network (I3D) that watches a clip and summarizes its appearance and motion into a feature vector — then measures how far apart the two clouds of feature vectors sit by comparing their means and spreads (covariances); a lower FVD means the generated clips look and move more like real ones. It is widely criticized because the score often disagrees with human judgment — a clip people clearly dislike can still post a good FVD — which is exactly why suites like VBench break quality into many separate dimensions instead of trusting one number.

GAE​

Generalized Advantage Estimation — the standard recipe for estimating the advantage with a tunable trade-off between bias and variance. The two extreme ways to estimate an advantage are a single TD step (low variance but biased, because it leans on the value estimate after just one step) and the full Monte Carlo return (unbiased but high variance, because it depends on the entire noisy trajectory). GAE blends all the in-between n-step estimates into one number using a decay parameter λ: λ = 0 recovers the one-step version, λ = 1 recovers the full-return version, and values like λ ≈ 0.95 sit in the sweet spot used by almost every modern paper. It is essentially TD(λ) applied to advantages, and it is a standard component of A2C and PPO. Analogy: rather than trusting only tomorrow's weather forecast (one step, steady but shortsighted) or waiting to see the whole month play out (the full record, accurate but wildly variable), you take a weighted blend of forecasts at every horizon, leaning on the nearer ones.

Gait​

A regular, repeating pattern of leg movements (such as walking, trotting, pacing, or galloping) used by animals or legged robots to move from one place to another.

  • Why it matters: Legged robots must coordinate multiple motors to support their weight, move forward, and maintain balance. Using a structured gait coordinates the legs so that the robot always has enough feet on the ground to support itself, or handles dynamic balance during periods where it is airborne.
  • How it works: A gait is defined by its duty factor (the fraction of time a foot is on the ground) and phase offsets (the timing difference between when different feet touch down). For example, a trot is a symmetric diagonal gait where the front-left and rear-right legs move together, alternating with the front-right and rear-left legs.
  • Analogy: Imagine a musical rhythm. A walk is a slow, steady four-beat drum roll where each leg takes turns stepping. A trot is a faster two-beat rhythm where pairs of legs strike the ground at the same time.

GameNGen​

A 2024 Google system (the name reads "Game-N-Gen", i.e. game engine) that showed a neural network can run a playable game entirely by generating its frames, with no traditional game code underneath. It is an action-conditioned diffusion model trained on recorded play of the classic shooter DOOM: given the recent frames plus the player's current button press, it predicts the next frame, fast enough to play in real time at about 20 frames per second. It is a landmark demonstration that a learned world model can stand in for a hand-written engine — the screen you see is being dreamed, frame by frame, in response to your controls.

GAN inversion​

Running a GAN backwards: given a real photo, find the input latent code that makes the generator reproduce it. A trained generator only goes code → image, so inversion recovers the missing code either by optimizing it to lower reconstruction error or by training an encoder to predict it in one shot. It is the step that lets you edit a real image — once you have its code, nudging the code changes the picture.

GANs (Generative Adversarial Networks)​

A class of generative models that trains two networks in a contest. A generator turns random noise into fake images, and a discriminator tries to tell those fakes from real ones; each one makes the other better, like a counterfeiter and a detective locked in an arms race. At the end you keep the generator, which by then makes images realistic enough to fool a well-trained critic. GANs produce sharp samples but are famously unstable to train — see mode collapse.

Gated​

An operation where one path of a neural network controls how much of another path gets through, by multiplying the two together value-by-value (element-wise multiplication). Picture a row of dimmer switches — or the valves on a bank of faucets. The main path carries the information; the second path produces a "gate" number for each value, and that number turns the corresponding value up or down. A gate near 0 shuts a value off (nothing passes), a gate near 1 lets it through untouched, and anything in between is a partial dribble. Because the multiply happens one number at a time, every feature gets its own private valve, so the network can wave some details through while damping others — all decided on the fly from the input. This is the trick behind LSTM "forget/input gates" (deciding what to keep vs. drop from memory) and modern MLP blocks like SwiGLU, where one half of the layer gates the other. It is closely related to scale-and-shift conditioning, except a pure gate only scales (multiplies) rather than also adding an offset.

GRU​

Gated Recurrent Unit — a small recurrent network that carries a running "memory" vector forward through a sequence, one step at a time. At each step it looks at the new input and its own previous memory, then uses gates to decide how much of the old memory to keep and how much of the new input to write in. That selective forgetting is what lets it hold onto something relevant from many steps ago instead of having its memory constantly overwritten by whatever just arrived. It is a simpler, cheaper cousin of the LSTM, with fewer gates and no separate cell state, and it is what carries the deterministic half of an RSSM's state in the Dreamer world models.

GCG​

Short for Greedy Coordinate Gradient — a gradient-based attack that finds an adversarial suffix (a short string of seemingly random tokens) which, when appended to a harmful question, causes an aligned LLM to comply anyway. It works by swapping one suffix token at a time for whatever the gradient says raises the probability of an unsafe answer most. Like picking a combination lock by feeling each dial until the click; once one model is unlocked the same suffix often opens other models, which is why GCG is the standard benchmark attack in jailbreak research.

GELU​

Gaussian Error Linear Unit — a smooth activation function widely used in transformer MLPs.

GEMM​

GEneral Matrix Multiply — the workhorse operation C = A × B on two matrices, and the single most common heavy computation inside a neural network. GPUs are built to do GEMMs fast; nearly every layer's forward pass is one. When one input is very "skinny" (a tiny batch, as in single-token decode) the GPU's Tensor Cores sit half-idle, so that case needs a different kernel from a big, square prefill GEMM.

Generalization​

How well a model performs on inputs it has never seen, as opposed to merely repeating its training examples. A model that generalizes has captured the underlying pattern; one that has only memorized has captured the examples — the difference between a student who learned how multiplication works and one who memorized a single times-table and is lost on any new numbers. For a style LoRA, generalization means the learned look transfers to prompts that never appeared in training; its opposite is overfitting. You measure it by checking performance on held-out inputs, not on the training set.

Generalized policy iteration​

The unifying idea that almost every RL algorithm is some interleaving of two processes: policy evaluation (make the value function more accurate for the current policy) and policy improvement (make the policy greedier with respect to the current values). Policy iteration runs evaluation to completion before each improvement; value iteration does a single evaluation step per improvement; but the two can be blended in any proportion and still converge to the optimal policy. Think of tuning a piano: policy iteration is like perfectly tuning one string before moving to the next, while value iteration is like giving every string a tiny twist and repeating the process until the whole chord sounds right. Either way, you eventually reach a perfectly tuned instrument. The processes both compete (improving the policy makes the old values stale) and cooperate (each hands the other something better to work with), and they come to rest only when neither can change anything — which is exactly the optimal solution.

Generator​

The half of a GAN that actually makes images: it takes a vector of random noise and maps it to a picture, learning to fool the discriminator into judging its output as real. It never sees the real images directly — it learns only from whether the discriminator was fooled, like a forger who improves purely from a detective's reactions. After training, the generator alone is what you keep and sample from.

GenEval​

A benchmark that measures how faithfully a text-to-image model obeys the structured content of a prompt — the right number of objects, the right colors, the right spatial arrangement ("a red cube to the left of a blue sphere"). Instead of asking a person, it runs an object detector on each generated image and checks automatically whether every requested object, count, color, and position is present; the score is the fraction of prompts whose requirements were all satisfied. Like an exam graded against a fixed answer key rather than on handwriting: "two cats? — yes; one is orange? — no, fail." It targets compositional skills (counting, positioning, attribute binding) that beauty metrics like FID completely ignore.

Genie​

A line of DeepMind "foundation world models" (Genie, Genie 2) that learn to generate playable, action-conditioned video from large amounts of ordinary internet video — without ever being told which action produced each frame. The trick is a latent action model: Genie infers a small set of consistent "actions" purely from how consecutive frames change, so afterwards a person can press one of those discovered actions and watch the model roll the world forward. This is what lets it build a world model you can actually control out of unlabeled footage alone, instead of needing recorded controller inputs.

GGUF​

GPT-Generated Unified Format (GGUF) is a single-file format for storing a quantized model — weights plus all the metadata needed to run it — popularized by llama.cpp. Like a self-contained zip that a laptop or phone can open and run without extra setup, it is the format of choice for edge and on-device inference.

Gimbal lock​

A failure of Euler angles (describing orientation as three separate spins — roll, pitch, yaw) where, at certain orientations, two of the three axes line up and you lose a degree of freedom: two different knobs now do the same thing, so some rotations become impossible to express and nearby motions make the angles jump wildly. The classic case is pitching straight up by 90°, after which roll and yaw both spin about the same vertical line. Analogy: it is like the nested rings of a gyroscope collapsing into one plane — you can still spin, but one direction of tilt has silently vanished. This is the headline reason robotics stores orientation as a quaternion or rotation matrix instead of Euler angles, neither of which can lock.

Glow​

A well-known normalizing flow model (from OpenAI, 2018) that improved on Real NVP by adding learnable 1×1 convolutions that shuffle and mix the channels between steps, letting it generate sharp, high-resolution faces. It showed that flows could produce convincing images and smoothly morph one face into another, though they were later overtaken by diffusion models on hard, real-world images.

GLU​

Gated Linear Unit — a layer that computes two things from the input and multiplies them together element by element: one is the actual content, the other is a "gate" (a non-linearity whose output sits near 0–1) that decides how much of that content to let through. Like a row of dimmer switches, one per wire, that the network learns to turn up or down — rather than a plain on/off. Being able to suppress parts of its own signal makes a GLU more expressive than a single linear layer; SwiGLU is the popular variant that uses Swish for the gate.

GPTQ​

Short for Generative Pre-trained Transformer Quantization — a post-training quantization (PTQ) method that compresses each layer's weights row by row, using second-order (Hessian) information to choose the int8 / int4 values that minimize the reconstruction error one layer at a time. Despite the name, GPTQ is not GPT-specific; it works on any transformer.

GPU​

Graphics Processing Unit — a specialized processor designed for high-throughput parallel computation. Instead of a few powerful cores, it consists of thousands of simpler cores running in parallel. It is optimized for repetitive, highly structured mathematical operations like matrix multiplication or pixel rendering. Analogy: A massive factory assembly line with thousands of basic workers. If you need to make one complicated, custom sculpture (a sequential task), the workers will struggle compared to the master chef. But if you need to package millions of identical boxes (a highly parallel task), the factory workers will finish it thousands of times faster than the chef.

GQA​

Grouped-Query Attention — sharing K/V heads across query heads; primary KV-cache saver at serving time

Go-Explore​

An exploration method that splits a problem most algorithms conflate: returning to a promising place, and exploring from there. It keeps an archive of interesting states it has reached, then repeatedly (1) picks one, (2) goes back to it directly — by restoring the simulator or by running a goal-reaching policy, not by hoping to rediscover it — and (3) explores randomly from that launch point, adding whatever new states it finds to the archive. The insight is that ordinary intrinsic-motivation agents suffer detachment: once the bonus near a distant frontier is consumed, the agent has no reliable way to get back there, and its own curiosity pulls it elsewhere. Go-Explore made headlines by beating human scores on Montezuma's Revenge and Pitfall, the two games that had defied every other method. Analogy: a cave explorer who leaves ropes at every chamber they reach, so each new expedition can start at the deepest point already known rather than crawling in from the entrance again.

Gradient accumulation​

Summing the gradients from several small batches before calling the optimizer, so the update matches a larger effective batch size without its memory cost.

Gradients​

The vector of partial derivatives of a function with respect to each of its parameters. Think of it as a list of slopes telling you exactly the rate of change of the output with respect to each parameter.

The Intuition This combination of direction and size is exactly what an optimizer follows downhill—much like feeling which way a hillside slopes and how steeply to find the lowest point.

A Concrete Example Consider a simple model y = w·x + b with weight w = 2, bias b = 1, and input x = 3.

  • Prediction: y = 2·3 + 1 = 7
  • Target: 10
  • Loss: L = (y - target)² = (7 - 10)² = 9

Calculating the Gradient Using the chain rule, we can determine the exact rate of change of the Loss (L) with respect to w and b. Since the derivative of L with respect to y is 2(y - target), and the partial derivative of y with respect to w is x (while with respect to b it is 1), the gradients are calculated as follows:

  • For Weight (w): ∂L/∂w = (∂L/∂y) · (∂y/∂w) = 2(y - target) · x = 2(-3) · 3 = -18
  • For Bias (b): ∂L/∂b = (∂L/∂y) · (∂y/∂b) = 2(y - target) · 1 = 2(-3) · 1 = -6

Interpreting the Results

  • Direction (The Sign): The negative signs indicate that you need to increase both w and b to reduce the error.
  • Magnitude (The Size): "Bigger" here refers to the absolute value (|-18| > |-6|). Even though -18 is a more negative number than -6, its magnitude is larger. This means the weight (w) has a much stronger influence on the loss than the bias (b).

Gradient checkpointing​

See activation checkpointing.

Gradient descent​

An optimization algorithm used to minimize a model's loss function by iteratively moving its parameters in the direction of the steepest decrease (the negative gradient). The size of the steps taken in this direction is determined by the learning rate.

Analogy: A hiker stuck on a foggy mountain who wants to find the valley at the bottom. Since they cannot see the path, they feel the slope of the ground under their feet and take a step in the direction that goes downhill most steeply. By repeating this step-by-step, they will eventually reach the bottom.

Example: During training, after the backward pass computes the gradients of the loss with respect to all weights, the optimizer updates the weights by subtracting a fraction of the gradient (e.g., weight = weight - learning_rate * gradient). This adjusts the model to make slightly more accurate predictions on the next batch.

Gradient penalty​

An extra loss term used in Wasserstein GANs (WGAN-GP) that keeps the critic 1-Lipschitz — meaning its output cannot change faster than its input. It works by measuring the size of the critic's gradient with respect to its input image and pushing that size toward 1. This replaces the original WGAN's blunt trick of clipping weights to a fixed range, which often hurt quality, and is the main reason WGAN-GP trains so stably.

Gradient clipping​

A hard cap on how big a single parameter update is allowed to be: if the gradient vector's length exceeds some threshold, it is rescaled back down to that length before the optimizer uses it (its direction is kept, only its size is cut). It exists because one freak batch — an outlier, a momentarily diverging bootstrapped target in RL — can otherwise produce an enormous gradient that throws the weights somewhere useless in a single step, undoing hours of training. Like a governor on an engine: normal driving is untouched, but the thing physically cannot redline. In DQN it is a standard companion to the Huber loss — one bounds the loss's slope, the other bounds the step actually taken.

GradScaler​

A helper used with float16 mixed-precision training that multiplies the loss before the backward pass, preventing small gradients from rounding to zero (underflow).

Graph break​

A point where torch.compile cannot trace the code (e.g. a print or a data-dependent branch), forcing it to split the model and fall back to eager mode — a common cause of lost speedup.

Grasp-quality network​

A trained deep network (e.g., Dex-Net) that evaluates a candidate grasp pose (such as a 6-DoF pose) on an object and predicts its probability of success. Instead of calculating physics equations for force closure directly, the network is trained on large datasets of simulated or real-world grasps to map visual inputs (like depth maps or point clouds) to a grasp quality score.

  • Analogy: Imagine an experienced warehouse worker looking at a cluttered bin. At a glance, they can tell you "grabbing it by the handle will work, but grabbing it by the slippery edge will fail." They aren't calculating friction coefficients in their head; they are matching visual patterns to past successes. A grasp-quality network does the same for a robot.
  • Example: A top-down grasp system evaluates 100 possible finger placements on an object mask and runs them through a grasp-quality network. The robot executes the pose that receives the highest score.

Grasp synthesis​

The process of finding a feasible robot gripper 6-DoF pose relative to an object that will result in a stable, secure grasp. Grasp synthesis approaches are split into classical analytic methods (which solve geometric contact equations for force closure) and data-driven methods (which use grasp-quality networks or models like AnyGrasp to predict grasp quality from images or point clouds).

  • Analogy: Before you pick up a mug, you look at it and plan where to put your fingers. If you pick it up by the handle, that is one grasp; if you cup the base, that is another. Grasp synthesis is the algorithm that determines these contact points for a robot.
  • Example: In 2D analytic grasping, the robot calculates the geometry of a polygonal block, draws friction cones from its edges, and selects finger contact points where the friction cones overlap to prevent slipping.

Greedy decoding​

The simplest sampling rule: at every step, pick the single most likely next token (the argmax of the logits) and never roll the dice. Like always ordering the most popular dish on the menu — boring but predictable. Useful when reproducibility matters, though on a GPU even greedy decoding is not bit-for-bit deterministic across batch sizes because floating-point sums reorder.

Greedy policy​

The policy that, in every state, picks the action with the highest current value estimate — no randomness, no looking past the numbers in front of it ("greedy" because it grabs the locally best-looking action). Acting greedily with respect to the optimal value function yields the optimal policy, which is why so many algorithms loop between improving value estimates and reading off the greedy policy. On its own a greedy policy never explores, so during learning it is usually softened into an ε-greedy policy that occasionally acts at random.

Grid​

In parallel GPU computing (like CUDA and Triton), a grid is the top-level collection of all thread blocks launched to execute a single program (or kernel).

  • Why it matters: Because a GPU has thousands of processing units and executes millions of threads at once, you cannot manage them individually. Instead, they are organized hierarchically. A grid represents the entire computation task (e.g., adding two large arrays). The grid is sharded into independent blocks, which are then sharded into individual threads. This allows the hardware scheduler to run different blocks of the grid in any order on any available processor unit (SM), providing massive hardware scalability.
  • How it works: When launching a kernel, you specify the grid's dimensions (number of blocks in 1D, 2D, or 3D) and the block's dimensions (number of threads per block). The GPU's hardware scheduler assigns each block in the grid to an available SM for execution. Since blocks in a grid execute independently, they cannot coordinate or synchronize with each other, ensuring the grid can execute on a GPU with 10 SMs just as easily as on a GPU with 100 SMs.
  • Analogy: Imagine a huge manufacturing company tasked with assembling a fleet of 1,000 cars. The entire assembly order is the grid. To divide the labor, the company splits the order into 1,000 separate work orders (the blocks), where each block builds one car. Each team of workers (the threads in a block) coordinates locally to build their car. The factory manager (the scheduler) assigns the work orders to any free assembly line (SM). Because each work order is self-contained and does not rely on other assembly lines, the factory can scale up the number of active assembly lines to build the fleet faster without changing the plans.
  • Example: When launching a Triton matrix multiplication kernel, the grid might be a 2D layout of blocks where the grid size (grid_x, grid_y) corresponds to the number of output tiles. Each block in the grid calculates one tile of the output matrix independently.

GridWorld​

A deliberately tiny reinforcement-learning environment: a 2D grid of cells in which an agent steps up/down/left/right, with a few walls, a goal cell, and maybe a hazard. Because the entire state is just "which cell am I in," it is trivial to simulate and to draw, which makes it the standard first sandbox for testing an idea — a world model, a policy, a planning algorithm — before scaling up to something as rich as Atari or a 3D game.

Gripper​

The grasping device at the end of a robot arm — the mechanical "hand" that physically picks up, holds, or manipulates objects. The simplest design is a parallel-jaw gripper: two flat fingers that open and close like a pair of pliers. More complex grippers have multiple fingers with their own joints, suction cups, or specialized tools for a particular task. In any robot task description, the gripper is usually the important part: "pick up the cup" means "move the arm so the gripper closes around the cup." A gripper is one type of end-effector — the broader term for whatever the arm's tip is equipped with (a gripper, suction cup, welding torch, or camera mount are all end-effectors).

Grokking​

A training curve where a model performs poorly for a long stretch and then improves suddenly — sometimes long after its training loss has flattened — as if it "finally got it". The name is borrowed from Robert Heinlein's novel Stranger in a Strange Land, where to grok means to understand something so completely it becomes part of you; researchers adopted it after observing tiny transformers on arithmetic tasks that memorize for thousands of steps and then abruptly switch to a general algorithm that also works on unseen examples (generalization). Practical consequence: evaluating a model mid-training can wildly misjudge what one more stretch of training would unlock, because capability can arrive as a cliff, not a slope.

Grounding​

Making a VLM point at where something is in an image, not just say that it is there — the model outputs spatial references like a bounding box (a rectangle (x1, y1, x2, y2) around an object) or a single point, instead of only words. The common trick is to add special tokens (e.g. <box>) plus tokens for quantized coordinates to the vocabulary, so a location becomes a few extra tokens the model emits with ordinary next-token prediction — no new architecture needed. In modern native multimodal models, this alignment is leveraged during the decode phase by attending heavily to visual features stored in the KV cache during prefill. Analogy: the difference between a tour guide who says "there's a fountain in this plaza" and one who actually points their finger at it. Example: asked "where is the dog?", a grounded model answers "<box> 0.10 0.20 0.45 0.80", which a viewer can draw as a rectangle on the photo; this is what benchmarks like RefCOCO measure.

GRPO​

Group Relative Policy Optimization — an algorithm for reinforcement learning from human feedback (RLHF) that simplifies Proximal Policy Optimization (PPO) by removing the need for a separate critic network (which estimates state values). Instead of comparing a model's response to an absolute value estimate from a critic, GRPO generates a group of candidate outputs for the same prompt, scores all of them using a reward model, and calculates the relative advantage of each response compared to the average of the group. This significantly reduces the memory and compute overhead of RL training. Analogy: Imagine grading a class of students. A traditional grading system (PPO) compares each student's test score to a global standard (a critic model). Under GRPO, you don't need a global grading scale. Instead, you sit a group of students down, have them answer the same question, and grade them relative to how the group did as a whole (e.g., marking them above or below the group average). Example: When training a chatbot to write Python code, GRPO prompts the model and generates 5 different code responses. A reward model checks each response. If response #3 runs successfully, is clean, and is much better than the average of the other 4 responses, it gets a positive advantage. If response #2 has syntax errors and is much worse than the group average, it gets a negative advantage. The model updates its parameters to favor the style of the above-average responses without ever needing a critic model to predict a baseline score.

GSM8K​

A benchmark of about 8,000 grade-school math word problems, widely used to test step-by-step reasoning because each problem has a single checkable numeric answer.

GTSAM​

Georgia Tech Smoothing and Mapping (GTSAM) is a popular open-source C++ library (with Python bindings) designed for solving sensor fusion and SLAM problems using factor graphs. Unlike filtering approaches that discard old states, GTSAM frames state estimation as a smoothing problem, optimizing the robot's entire historical trajectory (or a sliding window of it) at once using nonlinear least-squares solvers like Levenberg-Marquardt. It uses iSAM2 (Incremental Smoothing and Mapping), which updates only the affected parts of the factor graph when new measurements arrive, making full-history optimization fast enough to run in real time.

Analogy: A detective solving a mystery. Instead of guessing the culprit step-by-step and forgetting past details (filtering), the detective pins every clue, photo, and timeline connection to a large corkboard (the factor graph). When a new clue arrives, they don't rebuild the entire case; they only re-examine the specific suspect and photos connected to that clue (iSAM2's incremental update), maintaining a highly accurate, consistent picture of the whole case.

Gymnasium​

The standard Python library of reinforcement-learning environments — the maintained successor to OpenAI Gym. It defines the common API every environment follows: reset() to start an episode and step(action) to take an action and get back the next observation, reward, and whether the episode ended. Because classic testbeds like FrozenLake, Blackjack, Cliff Walking, CartPole, and the MuJoCo control tasks all share this interface, the same training loop can be pointed at any of them with almost no changes.

Gyroscope​

The part of an IMU that measures angular velocity — how fast it is rotating, in degrees or radians per second — about three axes. It reports the rate of turning, not the absolute heading; to get orientation you must sum (integrate) those rates over time, which lets a small constant bias slowly build up into a growing heading error (drift). Analogy: sensing how quickly you are spinning on an office chair with your eyes shut — you can feel the speed of each turn, but after many turns you have lost track of exactly which way you are now facing.

H.264​

The most common video codec on the internet, also called AVC (Advanced Video Coding) — the rules used to compress almost every .mp4 you have ever streamed. It compresses well and decodes fast on nearly all hardware, which is why it is the safe default, though newer codecs like AV1 shrink files further. Analogy: it is the JPEG of video — not the smallest or newest, but supported everywhere. Example: a 5-second 720p clip that is 333 MB as raw frames might be only a few megabytes as an H.264 .mp4.

H2O​

Short for Heavy-Hitter Oracle, a KV cache eviction method that keeps only the handful of past tokens that have been getting most of the attention — the "heavy hitters" — and throws the rest away. Like skimming a long book and keeping only the few sentences you keep flipping back to: you save shelf space while barely losing the plot, which lets a model serve much longer sequences in the same memory. It always keeps the very first tokens too (the attention sink), since those anchor the model no matter what they say.

Half-rotation​

An efficient way to apply RoPE: rather than rotating each adjacent pair of vector components on its own, you split the vector into two halves and combine them in one shot (the rotate_half trick, [x₁, x₂] → [−x₂, x₁]). It turns many tiny 2-D rotations into a couple of whole-vector operations, so it runs fast on a GPU while giving the same result.

HalfCheetah​

A MuJoCo continuous-control task: a two-dimensional, two-legged "cheetah" robot, fixed in a vertical plane so it cannot fall over, that must learn to run forward as fast as possible. It has 6 motors (continuous joint torques) and a state of joint angles and velocities. Because it cannot topple, it is one of the gentler MuJoCo benchmarks and a common first stop for testing a continuous-control algorithm before moving to balance-critical bodies like Walker2d, Ant, or Humanoid. Provided as HalfCheetah-v4 in Gymnasium. The name is literal: it is the bottom half of a cheetah — just the legs and spine, run in a flat 2-D world.

Hallo​

A specific talking-head model designed to generate high-quality portrait animations from a single image and audio. Like EMO, it focuses on making the facial movements, including lip sync and expressions, look naturally tied to the audio without the stiffness of older, landmark-based methods.

Hallucination​

When an LLM states something false with the same confident tone it uses for true things — invented citations, made-up people, fabricated facts. Like a student who didn't read the book but answers the essay question anyway in confident prose; the grammar is fine, the facts are not. Hallucination is built in to the next-token prediction objective, which rewards fluent continuation rather than truth, and is mitigated (not solved) by RAG, verifiers, and abstention training.

Hand-eye calibration​

The procedure that finds the fixed rigid transform between a camera and the robot it is mounted on — either camera-on-the-hand ("eye-in-hand") or camera-watching-the-arm ("eye-to-hand"). You move the arm to many known poses, record how a fixed marker like an AprilTag appears to shift in each image, and solve the matrix equation AX = XB, where A is how the hand moved, B is how the camera saw the tag move, and the unknown X is the camera-to-hand offset you want. Analogy: you know how your hand moved and how the view moved, and you are solving for how your eye is bolted to your hand so the two stories line up. Without it, the robot sees an object but points its gripper a few centimeters off — enough to miss every grasp.

Hard negatives​

In contrastive training, the wrong candidates the model finds hard to reject because they look almost right — as opposed to easy negatives so obviously wrong they teach it nothing. For a photo of a husky, the caption "a wolf in deep snow" is a hard negative (close, but wrong), while "a slice of pizza" is an easy one. Training learns fastest from hard negatives because they sit right on the boundary the model is still getting wrong, so each one delivers a large, informative gradient; mining them means actively searching the data for these near-misses (e.g. the highest-cosine-similarity mismatch) instead of hoping a random batch happens to contain some. Like a chess student who improves quickest by drilling against opponents just above their level, not by beating beginners over and over. Example: to mine hard negatives for a caption, retrieve the images it scores highly against but does not actually describe, and add those as negatives for the next training step.

HBM​

High-Bandwidth Memory (HBM) is a specialized type of ultra-fast 3D-stacked random-access memory (RAM) designed to sit directly alongside high-performance processors like modern GPUs (e.g., NVIDIA H100 or A100).

  • Why it matters: Deep learning requires feeding massive amounts of data (like model weights and activations) to the processor. Traditional memory is often too slow, creating a bottleneck where the fast processor sits idle waiting for data. HBM provides the massive bandwidth needed to keep modern AI chips running efficiently.
  • How it works: Instead of laying memory chips flat on a circuit board connected by long, thin wires, HBM stacks memory dies vertically on top of each other and places the stack directly next to the GPU on the same silicon package. They are connected by thousands of tiny microscopic vertical wires (called Through-Silicon Vias, or TSVs), creating an incredibly wide path for data to travel.
  • Analogy: Traditional memory is like a warehouse located miles away from a factory, connected by a two-lane highway; materials (data) can only arrive so fast. HBM is like building a multi-story warehouse directly next door to the factory floor, connected by a massive 100-lane highway. Even if the speed of each individual truck is the same, HBM can transport vastly more material per second because of the sheer number of lanes.
  • Example: Large language models have billions of parameters that must be read from memory for every single token generated. A GPU with HBM can read these parameters at rates exceeding 3 terabytes per second, allowing models to generate text at high speed.

Headroom​

The safety margin you have left before something breaks. In low-precision training it is the spare range of values a number format can still represent before it overflows or rounds down to zero and triggers numerical issues — like the gap between your head and the ceiling: the less you have, the easier it is to bump into trouble. FP8 packs numbers into far fewer bits than bfloat16, so it has much less headroom and is more likely to destabilize a run.

Heads (attention)​

The independent, parallel attention sub-computations in multi-head attention. Each head operates on its own learned projections of queries, keys, and values, so different heads can latch onto different relationships — one might track which word is the grammatical subject while another tracks what rhymes — and the model attends to several representation subspaces at once. They are called heads by analogy to the read/write "heads" of a tape or disk drive: several separate readers scanning the same strip of data in parallel, each pulling out something different. "Multi-head" attention simply runs many such readers side by side and then joins their findings.

Hessian​

The matrix of all second partial derivatives of a function — it captures the curvature of a loss landscape, not just its slope. Where the gradient tells you "which way is downhill," the Hessian tells you "and how sharply does it bend." Like the difference between knowing a road slopes down and knowing whether it banks into a tight curve or stretches out almost flat. For real LLMs the full Hessian is too big to store (rows × columns each equal to the parameter count), so methods like GPTQ use cheap approximations of it — typically built from a small batch of calibration activations — to decide which weights matter most when quantizing.

Heun's method​

A second-order ODE solver that improves on the Euler method with a predict-then-correct step: it takes a tentative Euler step, measures the slope at that new point too, then moves using the average of the start and end slopes. Averaging the two slopes cancels much of the error Euler makes, so Heun reaches the same accuracy in far fewer steps — which is why it is the default sampler in EDM. Like checking the road both where you are and where you're about to be, then steering down the middle. Named after the German mathematician Karl Heun.

Hierarchical generation​

Building a long or complex video in stages from coarse to fine rather than all at once: first decide the big structure — a shot list or a handful of keyframes spread across the timeline — then fill in the detailed frames between them. Working top-down keeps a long video coherent, because the overall plan is fixed before any single moment is rendered, the same way a director storyboards a film before shooting it. Contrast it with generating a clip straight through frame by frame, where the story has no plan to hold it together and tends to wander. It is one of the main strategies (alongside sliding-window generation and streaming) for getting past the few-second limit of a single model.

Hierarchical VAE​

A VAE with several layers of latent variables stacked at different scales instead of just one. Higher levels capture the big picture (overall layout and shape) while lower levels fill in fine detail (texture and edges), much like an artist who first sketches rough shapes and then adds the small touches. Splitting the work across levels lets the model represent complex images far better than a single flat latent space can. NVAE and Very Deep VAE are well-known examples.

Higher-order sampler​

In diffusion sampling, the solver takes a series of discrete steps along an ODE path from pure noise toward a finished image. A higher-order sampler estimates the shape of that path more accurately at each step by using extra slope measurements, instead of assuming the path is locally a straight line. A first-order method (Euler) just follows the slope where it currently stands; a second-order method like Heun's method or DPM-Solver++ also peeks ahead and averages the two slopes, cancelling most of the error. Because each step is more accurate, higher-order samplers reach good image quality in far fewer steps — often 15–25 instead of 50+. Picture driving toward a bend: a first-order driver steers only by where the road points right now and drifts wide, while a higher-order driver also notices how the road is curving ahead and corrects, staying on track with fewer adjustments.

HIP​

Heterogeneous-computing Interface for Portability — a C++ runtime API and programming language developed by AMD that allows developers to write code that can run on both NVIDIA and AMD GPUs from a single source. HIP provides a syntax that is highly compatible with CUDA, and includes translation tools (like hipify) that automatically convert existing CUDA code into HIP code, which can then compile on AMD's ROCm compiler or NVIDIA's CUDA compiler.

  • Analogy: Imagine a translation dictionary that translates Spanish (CUDA) into Esperanto (HIP). Because Esperanto was designed to be easily read by Spanish speakers, the translation is nearly word-for-word. The resulting Esperanto text (HIP code) can then be read and understood by both Spanish speakers and Portuguese speakers (AMD ROCm compiler), allowing the same message to reach both audiences.
  • Example: Running AMD's hipify-perl tool on a custom CUDA memory copy kernel to output a HIP source file, which is then compiled to run at native speed on an AMD Instinct MI300X accelerator.

Holonomic​

A vehicle whose instantaneous motion can be any direction (mecanum, omni)

Homogeneous transform​

A 4×4 matrix that bundles a 3D rotation and a 3D translation into a single object, so that composing two rigid motions is just one matrix multiply instead of separate rotation and translation steps. The layout is T = [[R, p], [0, 0, 0, 1]]: the top-left 3×3 block R is the rotation matrix and the top-right column p is the position vector; the bottom row [0 0 0 1] is fixed bookkeeping that makes the algebra work out. To move a 3D point x, you write it as a 4D vector [x; 1] and multiply: T · [x; 1] gives [Rx + p; 1] — rotation first, then shifted by p. Chaining two transforms is simply T_AC = T_AB · T_BC, which is what makes forward kinematics efficient: one matrix multiply per joint, accumulated from the base outward. The "homogeneous" in the name refers to the extra 1 appended to each point, a bookkeeping trick from projective geometry that turns affine (rotate-and-translate) operations into pure matrix products. Homogeneous transforms are the standard representation of poses in SE(3).

Hopper​

NVIDIA's 2022 GPU architecture (H100, H200) and the workhorse of LLM training and serving in 2023–2024. It was the first generation to ship dedicated FP8 Tensor Cores, which is what made FP8 inference a practical option. Named after Grace Hopper, the computer scientist who invented the compiler.

Huber loss​

A loss function that behaves like squared error for small mistakes and like absolute error for large ones, switching over at a threshold (usually 1). The point is the gradient: squared error's gradient grows without bound as the error grows, so one wildly wrong prediction can dominate a whole batch, while absolute error has a constant gradient that never settles near zero. Huber takes the good half of each. This matters in DQN, where the bootstrapped target is itself a moving and sometimes badly wrong prediction — one outlier TD error, squared, would yank the weights hard in a direction chosen by noise. Like a manager who takes small errors seriously but refuses to panic at a catastrophic one. In PyTorch it is F.smooth_l1_loss, and it pairs naturally with gradient clipping.

Hue​

The "color name" part of a color — red, orange, green, blue, and so on — separate from how light or dark it is and how vivid it is. It is one axis of the way computers describe color (the H in the HSV color model), and it wraps around in a circle, so the two ends meet and both 0° and 360° are red. Analogy: hue is the label on a paint tube ("blue"), brightness is how much white or black you stirred in, and saturation is how strong the color is. In an optical flow picture, hue is often used to show the direction each pixel moved — each compass direction gets its own color — while brightness shows how fast it moved.

Humanoid​

The hardest of the standard MuJoCo continuous-control tasks: a full two-legged humanoid robot — torso, arms, and legs, with 17 motorized joints — that must learn to stand and run forward without falling. Its state has dozens of numbers (joint angles, velocities, and contact forces) and its action sets a continuous torque for each of the 17 joints, a far larger and more fragile control problem than HalfCheetah or Ant because almost any clumsy move topples it. It is the usual stress test for whether an algorithm like SAC scales to high-dimensional bodies. Provided as Humanoid-v4 in Gymnasium.

Hybrid retrieval​

Retrieving with both dense embedding search (matches meaning) and sparse keyword search (BM25) (matches exact words) and merging the two result lists, so each method covers the other's weaknesses.

Hyperparameter​

A setting you choose before training starts, as opposed to a weight, which the model learns for itself during training. The learning rate, the number of layers, the batch size, and SAC's temperature α are all hyperparameters. They matter enormously and there is no gradient to guide them, so they are usually found by sweeping — training the same model several times with different values and keeping the winner — which is why an algorithm that needs fewer of them (or tunes its own, as automatic temperature tuning does) is worth a great deal in practice.

  • Analogy: Baking a cake. The oven temperature and the baking time are hyperparameters — you set them in advance, and getting them wrong ruins the cake. How the batter actually rises is not something you control; that is the learning.
  • Example: "SAC uses one hyperparameter setting across five different robot bodies" is a claim about generality: no human had to re-tune anything when moving from a 3-joint hopper to a 17-joint humanoid.

I2V​

Image-to-Video: the task of generating a short video clip starting from a single still image, where the model invents plausible motion while keeping the first frame's appearance fixed. It is easier than text-to-video (T2V) because the image already settles what the scene looks like, leaving the model to handle only how it moves — and its training data is essentially free, since any video clip can be split into "first frame = input, the rest = target" with no text caption needed. Stable Video Diffusion is the canonical open I2V model.

ICM​

ICM (Intrinsic Curiosity Module) is a curiosity-driven exploration method that pays the agent an intrinsic reward equal to how badly a forward model mispredicts the next state — surprising transitions are treated as worth revisiting. Crucially, ICM does not predict raw pixels; it first learns a compact feature space with an inverse model (a network that, given two consecutive states, guesses the action that connected them), which keeps only the parts of the world the agent can influence and throws away uncontrollable background detail. Predicting in that controllable space gives ICM partial immunity to the noisy-TV problem that traps pixel-level predictors. It is the close cousin of RND: both turn prediction error into a bonus, but ICM models the dynamics while RND distills a fixed random function. Like a toddler who pokes at things that don't behave as expected and ignores the ever-changing clouds it can do nothing about.

Identity function​

A function that returns its input unchanged: f(x) = x. In the context of straight-through estimators, gradients are passed through a non-differentiable operation as if it were the identity function.

Ideogram​

A text-to-image model (and product) built by a startup of the same name, especially praised for text rendering — drawing legible, correctly-spelled words, logos, and typography inside images, which makes it a favorite for posters and graphic design. Like a sign painter you can trust to spell the shop name right, not just paint pretty letters. It competes with DALL·E 3, Imagen 3, and Flux.

Image embedding​

The single dense vector a vision model boils a whole picture down to — a fixed-length list of numbers (say 512 of them) that captures what is in the image rather than its raw pixels. CLIP's image encoder, for instance, reads the pixels and outputs one such vector, placing pictures with similar content near each other in the shared embedding space. Analogy: distilling a whole meal down to a single flavor profile you can quickly compare against other dishes — you lose the individual ingredients but keep the essence needed to say "these two are alike." Because an image and a caption can then be compared just by the cosine similarity of their vectors, image embeddings are what make zero-shot classification and cross-modal retrieval work. Example: in CLIP the photo of a dog and the sentence "a photo of a dog" each become one vector, and the two land close together.

Imagen 3​

Google's text-to-image model, known for photorealistic detail and unusually good text rendering — it can spell words inside the picture correctly, long a weak spot for generators. It leans on a strong text encoder and carefully curated training data to follow prompts faithfully. Like a meticulous illustrator who not only paints the scene you describe but gets the lettering on the signs right. It is Google's competitor to DALL·E 3 and Stable Diffusion.

ImageNet​

A large benchmark dataset of about 1.2 million photos hand-labeled into 1,000 everyday categories (breeds of dog, kinds of mushroom, vehicles, and so on). For over a decade it has been the standard yardstick for "how well does this model see," so a new image encoder is almost always reported by its ImageNet accuracy. Think of it as the standardized entrance exam of computer vision — not perfect, but common enough that everyone's scores can be compared on one scale. A larger, even more finely labeled version is called ImageNet-21k (≈21,000 categories); see also its much smaller cousin CIFAR-10.

img2img​

Generating a new image that is guided by an existing input image instead of starting from pure noise. You partially noise the input — controlled by a denoising strength (0 = keep the original, 1 = ignore it) — then let the diffusion model denoise from there, so the result keeps the rough layout and colors of the input while following the new prompt. Like tracing over a rough sketch: the more you erase first, the more freedom the model has to redraw.

Impedance control​

A way to control a robot arm by commanding a relationship between motion and force rather than a position outright: you place a virtual spring-damper between the end-effector and a moving reference point, and the arm generates a push-back force proportional to how far it has been displaced from that reference (the spring) and how fast it is being pushed (the damper). The word "impedance" is borrowed from electronics, where it measures how much a circuit resists a flow; here it measures how much the arm resists being moved. The big knob is the virtual stiffness: high stiffness makes the arm hold its reference firmly (close to position control), low stiffness makes it soft and yielding (compliant), so it bends out of the way instead of fighting whatever it touches. This is the right tool whenever the robot must make contact — inserting a peg, polishing a surface, or sharing space with a person — because a purely position-commanded arm would push with unbounded force the instant the world got in its way. Analogy: instead of bolting the hand to a target, you connect it with a rubber band whose tightness you choose.

Importance sampling​

A statistics trick for estimating an average under one distribution while your samples actually came from a different one: you reweight each sample by the ratio of how likely it was under the target distribution versus the sampling distribution. In prioritized experience replay, transitions are deliberately sampled non-uniformly (surprising ones more often), which would skew the gradient; multiplying each update by an importance-sampling weight (1 / (N · P(i)))^β corrects for that over-sampling and keeps the learning target unbiased. The same idea lets off-policy methods reuse data collected by an older policy. Analogy: if you polled twice as many city-dwellers as their share of the population, you would weight each of their answers by half to recover the true nationwide average.

IMU​

Inertial Measurement Unit — a small sensor package that reports a body's own motion: a gyroscope measuring rotation rate and an accelerometer measuring linear acceleration, each on three axes (often plus a magnetometer for compass heading). It is fast and cheap and works anywhere, but it senses rates, not position — recovering where you are requires summing those rates over time (dead reckoning), which makes its estimate drift within seconds. That is why an IMU is almost always fused with a camera or GPS that can periodically pin the estimate back to reality. Analogy: the inner-ear balance organ — it instantly tells you that you are turning or speeding up, but on its own, eyes shut, you would soon misjudge exactly where you are.

Inception network​

A famous image-classification convolutional neural network (the "Inception" / GoogLeNet family) trained on millions of labeled photos. Along the way it learns to boil any image down to a compact feature vector — a list of numbers that captures what is in the picture (fur, wheels, sky) rather than the raw pixels. Because those features are such good summaries of image content, quality metrics like FID reuse a frozen, pretrained Inception network as a fixed yardstick instead of training anything new — like always using the same trusted scale to weigh two bags so the comparison is fair. (It was nicknamed "Inception" after the movie, for its "network inside a network" design.)

Indexing​

Mapping a multidimensional index [i, j, …] to a flat storage position via offset + Σ iₖ·strideₖ

Inertia​

The natural resistance of any physical object to a change in its state of motion (an object at rest wants to stay at rest, and an object in motion wants to keep moving at the same speed and direction, unless acted upon by an outside force). For rotating objects like robot joints, it is called rotational inertia (or moment of inertia), which measures how hard it is to start or stop a rotation.

  • How it works: In straight-line motion, mass measures inertia (a heavier box is harder to push). In rotation, inertia depends not just on the mass, but on where that mass is located relative to the pivot point. Mass located farther from the pivot point creates much more rotational resistance.
  • Analogy: Holding a broom. If you grip the broom right next to the heavy brush, it is very easy to swing back and forth because the mass is close to your hand. If you grip the very end of the handle, swinging the broom is much harder because the mass is now far from your hand. The mass of the broom is identical, but its rotational inertia is much higher.
  • Example: A figure skater spinning. When they tuck their arms in close, they minimize their rotational inertia, causing them to spin very fast. When they stretch their arms out, their rotational inertia increases, which slows their spin down. In a robot arm, the controller must calculate this changing inertia (represented by the mass matrix) as the arm folds or stretches, adjusting joint torque to maintain smooth movement.

Inference-time compute​

The work a model does while answering a question (not while training) — for reasoning models, mostly the tokens it spends "thinking" before it replies. Giving a fixed model more inference-time compute, like giving a student more time on an exam, can raise its accuracy without changing the model at all.

InfiniBand (IB)​

InfiniBand (IB) is a high-bandwidth, ultra-low-latency networking technology used in supercomputers and AI data centers to connect thousands of GPUs or TPUs together into a single, massive computing cluster.

  • Why it matters: When training large language models (LLMs), a single GPU does not have enough memory or processing power, so the workload is distributed across thousands of GPUs. During training, these GPUs must constantly share huge amounts of data (like gradients and weight updates) in lockstep. If they use standard computer networks, the GPUs spend most of their time waiting for data to travel between servers, causing a massive bottleneck. InfiniBand provides the speed and direct connections needed to keep all GPUs running at maximum efficiency.
  • How it works: InfiniBand achieves its high speed and efficiency through several key features:
    1. Massive Bandwidth: Modern InfiniBand links can transfer data at speeds up to 400 or 800 gigabits per second (Gbps) per connection, which is hundreds of times faster than standard office networks.
    2. Ultra-Low Latency: Latency is the delay before data transfer begins. InfiniBand's latency is measured in microseconds (millionths of a second), which is crucial for parallel algorithms like AllReduce where GPUs must sync up thousands of times per second.
    3. Remote Direct Memory Access (RDMA): RDMA allows one server's GPU to read or write directly to the memory (HBM or RAM) of another server's GPU. The data bypasses the operating systems and host CPUs of both servers, avoiding slow processing overhead.
  • Analogy: Imagine a group of researchers in different office rooms working together to solve a massive puzzle.
    • Standard Ethernet is like having the researchers communicate by writing letters, sealing them in envelopes, handing them to the office mailroom (the operating system and CPU), having the mailroom sort them, and sending them over the road. Every message takes a long time to deliver, and the researchers spend most of their time waiting.
    • InfiniBand with RDMA is like putting pneumatic tubes or direct conveyer belts through the walls between the rooms. If Researcher A needs a puzzle piece from Researcher B's desk, they can reach through the tube and grab it directly from Researcher B's desk drawer without B even needing to stop working or talk to the mailroom. The transfer is instantaneous and requires zero administrative overhead.

InfoNCE​

The contrastive loss that CLIP and most dual encoders train with: for each item it pulls the one correct match closer and pushes every other candidate away. How it is computed. Take a batch of N image–caption pairs, L2-normalize every vector, and build the N×N grid of cosine-similarity scores (one matmul). Each row is one image scored against all N captions, and the correct caption sits on the diagonal. Apply softmax across the row and ask that the diagonal entry get nearly all the probability — which is exactly cross-entropy with "the right answer is position i." Do this across rows and again across columns and average the two. The name is short for Noise-Contrastive Estimation of mutual Information: the off-diagonal pairs are the "noise" the true pair must be told apart from. Analogy: a police lineup where the model must point to the one caption that truly goes with this photo while N−1 decoys stand beside it, scored on how confidently it picks the right one.

In-hand manipulation​

The robotic task of reorienting or moving an object within a robot hand's grasp without dropping it, typically using a multi-fingered hand (like an Allegro or Shadow hand). This is one of the most complex tasks in robotics because it involves continuous contact state transitions, rolling and sliding mechanics, and severe visual self-occlusion. It is commonly solved using deep reinforcement learning combined with sim-to-real transfer.

  • Analogy: Pick up a pen and write with it, then use the fingers of the same hand to spin it around so the eraser is pointing down. You did that entirely within your hand without using your other hand or dropping the pen. That is in-hand manipulation.
  • Example: Training a policy in a physics simulator like MuJoCo to rotate a block to a target orientation using a five-fingered robotic hand, then transferring that policy to a physical hand.

Inpainting​

Filling in a masked-out region of an image so the patch blends seamlessly with the rest. You hand the model the surrounding pixels as fixed context and let it generate only the hole — like a restorer repainting a torn corner of a photo to match the surviving picture. With a diffusion model this is done by re-noising and denoising only inside the mask while pasting the known pixels back on every step.

Instruction tuning​

A second training stage that turns a model which merely continues text (or, for a VLM, describes an image) into one that follows requests — by fine-tuning it on many (instruction, response) examples instead of raw documents. For a VLM the examples are conversational (image, question, answer) triples, like the LLaVA-Instruct set whose dialogues a strong language model wrote from image annotations. Analogy: a fluent speaker who can ramble on any topic versus a helpful assistant who answers the exact question you asked — same vocabulary, very different behavior, and the gap is closed purely by showing thousands of question-and-answer demonstrations. Example: before tuning, shown a photo and "What is the dog doing?", the model might just caption "a dog on grass"; after tuning it answers "It is catching a frisbee." The key lesson is that this capability comes from data, not architecture — the network is unchanged; only what it trains on differs.

InstructPix2Pix​

An image-editing model that takes a photo and a plain-English instruction ("make it winter," "add sunglasses") and returns the edited photo in a single pass — no masks, no per-image optimization. Its real trick is the training data: since no one wants to hand-edit thousands of photos, the data is made synthetically — a large language model writes an instruction plus before/after captions, and a text-to-image model (Stable Diffusion) with Prompt-to-Prompt renders a matched image pair that differs only in the described change. The finished model is then fine-tuned on millions of these triples. Like teaching an editor by showing them countless "before, instruction, after" flashcards until they can follow any new instruction.

int4​

A 4-bit integer format used to compress neural network weights. In int4, each number is stored using only 4 bits of memory, which allows it to represent just 16 distinct integer values (usually from -8 to 7, or 0 to 15).

  • Why it matters: Storing weights in int4 uses only one-quarter of the memory of int8 and one-eighth of the memory of float16. This dramatic footprint reduction makes it possible to run large language models on consumer-grade hardware (like a local GPU or phone). However, because 16 values are too coarse to represent weights accurately on their own, int4 is always paired with a scaling factor (like in GPTQ or AWQ) that dynamically maps those 16 levels to the real weight range.
  • Analogy: Imagine trying to sketch a detailed portrait using a box of only 16 colored pencils. If you just draw directly, the face will look blocky and cartoonish (high quantization error). But if you are allowed to choose which 16 shades are in the box for each part of the drawing (using scaling and calibration), you can still capture a highly recognizable likeness.
  • Example: Quantizing a 7B parameter LLM from bfloat16 (~14 GB of memory) to int4 (~3.5 GB of memory) allows it to run smoothly on a single GPU with 8 GB of VRAM.

int8​

An 8-bit integer format used to compress neural network weights or activations for efficient inference. Storing numbers in int8 uses exactly 8 bits of memory, representing 256 distinct integer values (from -128 to 127).

  • Why it matters: Compared to standard float32, int8 uses only one-quarter of the memory and bandwidth. Furthermore, many modern GPUs and CPUs have dedicated hardware (like integer Tensor Cores) that can multiply int8 matrices much faster than floating-point matrices. Because it has 256 levels, int8 is often precise enough to quantize a model with almost zero loss in quality without needing complex retraining.
  • Analogy: Imagine a chef measuring ingredients. Instead of measuring flour down to the individual grain (high-precision floating-point), they round to the nearest gram (8-bit integer). The cake still bakes perfectly, and measuring is much faster and simpler.
  • Example: In SmoothQuant, activations and weights are both quantized to int8 before matrix multiplication to speed up LLM serving, while maintaining accuracy by smoothing out activation spikes first.

Inter-rater agreement​

A measure of how often two or more graders give the same scores to the same items — the check you run before trusting one grader to stand in for another. If a cheap LLM-as-judge and a human reviewer rate the same 100 answers and their scores line up, the automatic judge can replace expensive human review; if they disagree a lot, it cannot. It is computed with a statistic such as a correlation (how well two lists of numbers rise and fall together, on a −1-to-+1 scale) or Cohen's kappa (the fraction of agreement beyond what random guessing alone would produce, on a roughly 0-to-1 scale, named after the psychologist Jacob Cohen). Analogy: two teachers marking the same stack of essays — if their grades nearly match you can trust either one alone next time, but if they wildly differ then the rubric (or one of the teachers) is unreliable.

Imitation learning​

Learning to act by copying an expert's demonstrations rather than by trial-and-error against a reward. The plainest form is behavior cloning, which trains a policy to predict the demonstrator's action for each state as ordinary supervised learning. It is appealing when rewards are hard to design but good demonstrations are available — teaching a robot arm by guiding it, or a driving model from human logs. Its weakness is compounding error: one small mistake takes the agent into states the expert never visited, where it has no guidance and drifts further off. Analogy: learning a dance by mirroring a partner — fine while you stay in step, but one stumble and you are lost because you never learned to recover.

Implicit reward​

An approach where a model's goodness or "reward" is computed directly from its own word choices, rather than by using a separate reward-scoring neural network. In DPO, the math shows that the reward is already hidden (implicit) in the language model's own answer probabilities compared to those of a frozen reference model. So, instead of first training a separate network to grade answers, the model can look at the probability gap between its current self and its starting self to know how to improve. Analogy: Instead of having a teacher grade your essays with a score from 1 to 10 (explicit reward), you compare your essay's wording to a draft you wrote yesterday (reference model) to see how much more likely you are to use better words (implicit reward).

IoU (Intersection over Union)​

A metric used in computer vision, segmentation, and robotic manipulation to measure the overlap between two spatial regions (typically a predicted region and a ground-truth region). It is computed by dividing the area of intersection between the two regions by the area of their union (represented as |A ∩ B| / |A ∪ B|, where ∩ is the intersection and ∪ is the union); a score of 1 indicates perfect overlap, and 0 indicates no overlap.

  • Analogy: Imagine laying one towel over another on a table. The area where the two towels stack on top of each other is the intersection. The total table area covered by both towels combined is the union. Dividing the stacked area by the total covered area gives the IoU, showing how closely aligned the two towels are.
  • Example: In robotic cloth folding, the success of the fold is evaluated by computing the IoU between the final shape of the folded cloth and the target template shape.

IP-Adapter​

A lightweight add-on that lets a diffusion model take an image as a prompt alongside (or instead of) text — you hand it a reference picture and it copies that subject's appearance or style into what it generates. It works by encoding the reference image and feeding it through a small set of extra cross-attention layers added next to the existing text ones (the "IP" stands for image prompt), so the base model's own weights stay frozen and untouched. Because it needs no per-subject training and accepts any new reference on the fly, it is a popular way to hold a character or style steady across shots (see character consistency). Like handing an artist a photo and saying "draw new scenes, but keep this person looking exactly like this."

IPOPT​

Interior Point Optimizer (IPOPT) is a software package for large-scale, continuous non-linear optimization. It solves problems where you want to minimize a cost function subject to equality and inequality constraints. In robotics, IPOPT is the standard solver used alongside CasADi to solve trajectory optimization problems (like direct collocation) and Model Predictive Control (MPC) problems. Analogy: Imagine trying to find the lowest point in a hilly valley (minimizing cost) while staying inside a fenced area (inequality constraints) and walking along a specific walking trail (equality constraints). IPOPT starts with a guess and systematically walks downhill, adjusting its path to stay within the boundaries, until it finds the optimal point.

Intrinsic motivation​

A family of exploration techniques where the agent generates its own reward signal for seeking out novelty, instead of waiting for the environment's (extrinsic) reward. This self-generated bonus — an intrinsic reward — is what drives the agent into unexplored corners even when the real reward is rare or absent. The main flavors are count-based exploration (bonus for rarely visited states), prediction-error / curiosity methods like the ICM and RND (bonus for states a learned model finds surprising), and skill-discovery methods like DIAYN (bonus for behaving distinctly). Like a curious child who explores a new house for the sheer novelty of it, with no one promising a treat.

Intrinsic reward​

A bonus an agent adds to itself to encourage exploration, as opposed to the extrinsic reward that comes from the environment for actually solving the task. The total signal the agent optimizes is the extrinsic reward plus a weighted intrinsic term, so it is pulled both toward the goal and toward whatever its intrinsic motivation scheme calls "interesting" — typically novel or surprising states. Different methods compute the intrinsic part differently: a count-based 1/√N(s) bonus, or the prediction error of a learned model in the ICM and RND. Like giving yourself a small private "that was neat" reward for trying a new route home, on top of the real payoff of arriving.

Inverse dynamics​

Working out the joint torques required to produce a desired motion — the force-level counterpart to inverse kinematics (which works out joint angles for a desired pose). You hand it the joint positions, velocities, and the accelerations you want, and it returns the torques that achieve them by evaluating the manipulator equation M(q)q̈ + C(q,q̇)q̇ + g(q) = τ; the standard fast way to do this is the RNEA. It is the engine inside model-based controllers like computed-torque control and feedforward trajectory tracking, because to make a robot follow a path you must first know the forces that path demands. Its mirror image is forward dynamics — given the torques, find the resulting motion — which is what a simulator computes (via the ABA). Analogy: inverse dynamics is reading a planned dance routine and calculating exactly how hard each muscle must pull at each instant to perform it.

Inverse model​

A network that looks at two consecutive states and guesses which action connected them — the reverse of a forward model, which predicts the next state from the current one and an action. Its main use is not prediction but representation: it is the trick inside the ICM, where an encoder is trained only through the inverse model, so the features it learns must retain whatever the agent's actions affect and are free to discard everything else (swaying trees, flickering screens, other agents). That filter is what gives ICM partial immunity to the noisy-TV problem. Note this is unrelated to robotics' inverse dynamics, which computes joint torques. Analogy: watching two frames of a video of yourself and asking "what did I just do?" — you only need to notice the parts of the picture that you moved.

Inverse kinematics​

The reverse of forward kinematics: given a desired pose for the end-effector (where you want the hand and how you want it oriented), find joint angles that achieve it. It is much harder than the forward direction because it can have no solution (the target is out of reach), exactly one, several (elbow-up vs elbow-down), or infinitely many (a redundant arm). The common solvers are analytic (a closed-form formula for special arm geometries), iterative (repeatedly invert the Jacobian and step toward the goal — the approach damped least-squares makes robust), and optimization-based (pose it as a constrained minimization). Analogy: forward kinematics tells you where your fingertip is from your joint angles; inverse kinematics is working out how to bend your shoulder, elbow, and wrist to touch a specific spot on the wall.

IQL​

Implicit Q-Learning — an offline RL method that avoids the out-of-distribution value blow-up by never querying Q at any action outside the dataset, rather than penalizing such actions the way CQL does. It fits a value function V(s) with expectile regression — an asymmetric loss that leans toward the higher returns already in the data, so V approximates the value of the best in-dataset action without ever evaluating an unseen one — then extracts the policy by advantage-weighted regression, imitating dataset actions but up-weighting those with high advantage. Because every term is computed only from actions that truly occurred, there is nothing for the network to hallucinate, which makes IQL simpler, more robust, and harder to misconfigure than CQL — the modern default. Its guiding idea — constrain what you query, not what you output — also shows up in RLHF and reasoning-model training.

Isaac Lab​

An open-source robotics simulation framework built on NVIDIA Omniverse that is designed for robot learning, particularly reinforcement learning and imitation learning.

  • Why it matters: Training a robot dog to walk in the real world can take weeks of trial and error, during which the robot might break. Isaac Lab allows developers to simulate thousands of robots simultaneously on a single graphics card (GPU). This high-throughput parallel simulation allows reinforcement learning policies to learn complex behaviors in minutes instead of weeks.
  • How it works: It leverages NVIDIA's PhysX engine and GPU-accelerated APIs to run physics calculations and visual rendering directly in GPU memory, avoiding slow data transfers between the CPU and GPU.
  • Analogy: Imagine trying to learn how to play chess by playing one game at a time. It would take a long time to get good. Now imagine you could duplicate yourself into 4,000 clones who all play chess games at the exact same time, sharing what they learn instantly. You would master the game in a fraction of the time.

iso​

A prefix meaning "equal" or "the same" (from the Greek isos). In a phrase like iso-FLOP it marks a group of training runs that all spent the same compute budget, so they can be compared fairly — like rating cars by how far each travels on the same tank of fuel rather than on top speed. Plotting the loss of several iso-FLOP runs is how a Chinchilla-style scaling-law curve is drawn.

ITL / TPOT​

Inter-Token Latency (ITL), also known as Time Per Output Token (TPOT), is the average time elapsed between generating consecutive tokens during the steady-state decode phase of model inference.

  • Why it matters: While TTFT determines how quickly a user sees the first response, ITL/TPOT determines the reading speed or responsiveness of the rest of the generation. High ITL results in a sluggish, stuttering output.
  • Analogy: The speed at which a printer spits out subsequent pages after printing the first page. If the printer takes 2 minutes to warm up and print page one (high TTFT) but then prints pages two through ten every 2 seconds (low ITL), the steady-state printing speed is very fast.
  • Example: In a chat application, maintaining an ITL/TPOT of 30 ms per token ensures that the text streams faster than a typical human reading speed, providing a smooth conversational experience.

Iterative Closest Point (ICP)​

The standard algorithm for aligning two point clouds of the same scene captured from different viewpoints — the most common method of point cloud registration. It repeats a simple two-step loop: (1) pair every point with the closest point in the other cloud, then (2) compute the single rotation and translation that best pulls those pairs together, and apply it. Each pass slides the clouds a little closer; after a few iterations they snap into alignment. Its weakness is right there in the name — it always matches to the closest point, so if the two clouds start far apart or badly rotated it can settle into the wrong alignment. Analogy: laying one printed transparency over another and nudging it bit by bit until the lines coincide — easy if they already roughly overlap, hopeless if one is upside-down.

Jacobian​

The matrix of all first derivatives of a vector function — it linearly maps a small change in the inputs to the resulting small change in the outputs. In robotics the geometric Jacobian J(q) maps joint velocities q̇ to the end-effector's spatial velocity (its linear velocity v stacked with its angular velocity ω): (v, ω) = J(q) · q̇. It is the single most reused object in arm control — velocity control, force control (through its transpose), and inverse kinematics all run through it — and the configurations where it loses rank are exactly the kinematic singularities. Analogy: it is the "exchange rate" between turning the joints and moving the hand, and like an exchange rate it shifts depending on where you currently are (the configuration q).

Joint image-video training​

A training recipe that feeds a video model a mix of still images and video clips in the same run — treating each still image as a one-frame "video" — so the model keeps its sharp single-image skills while it learns motion. The problem it solves: training on video alone lets a model's still-image quality decay, because video datasets are smaller and more compressed than image datasets, so the rich appearance knowledge an inflated image model started with drifts away. Mixing in a large fraction of images (often the majority of each batch) keeps that knowledge fresh and makes the model far more data-efficient. It needs no architectural change because a still image is simply the T=1 special case of a video — the same layers process both.

Jailbreak​

A prompt — sometimes plain English, sometimes a gradient-found suffix like in GCG, sometimes a long role-play setup or a translation into a low-resource language — that gets a safety-trained model to do what its alignment training was supposed to refuse. Like picking a hotel-room door lock instead of asking for the key. Modern defenses assume any single safety layer can be jailbroken and use defense in depth — input filtering, output filtering, monitoring, refusal classifiers — instead of trusting the model alone.

JAX​

A high-performance numerical computing library developed by Google that combines a NumPy-like API with automatic differentiation (autograd) and a Just-in-Time (JIT) compiler called XLA. JAX is designed for high-performance machine learning research, allowing users to write pure Python functions and compile them to run efficiently on hardware accelerators like GPUs and TPUs.

  • Analogy: Imagine writing a draft recipe in Python. Standard Python executes it step-by-step, measuring out each ingredient slowly. JAX is like having a master chef read the entire draft recipe, rewrite it to be as fast as possible, and set up all the equipment (JIT compilation via XLA) so that cooking is optimized for the specific kitchen appliances (GPU/TPU) and completed in a fraction of the time.
  • Example: Defining a loss function for a neural network and using jax.grad to automatically compute gradients, combined with jax.jit to compile the entire training step into a single optimized kernel for execution on a TPU pod.

Jetson​

NVIDIA Jetson is a series of embedded computing boards designed by NVIDIA for edge AI, robotics, and low-power applications. These boards combine an ARM processor with an integrated NVIDIA GPU to run deep learning models locally without needing cloud connectivity.

  • Why it matters: Running AI models on physical systems (like drones, robots, or smart cameras) requires low latency and high power efficiency. A standard desktop GPU draws hundreds of watts, making it unusable for battery-powered systems. Jetson modules pack CUDA-capable compute into a tiny package that runs on 5 to 40 watts, allowing developers to deploy custom computer vision and robotic control models directly on physical devices.
  • How it works: Jetson boards share a unified memory architecture where the CPU and GPU access the same physical RAM. This eliminates the slow PCIe data transfer overhead found on desktop systems. Developers write standard PyTorch or CUDA code, quantize the models to INT8 or FP8 using tools like TensorRT or ExecuTorch, and run them directly on the board's hardware accelerators.
  • Analogy: Imagine trying to install a massive desktop refrigerator inside a small delivery van; it would take up all the room and drain the van's battery immediately. A Jetson is like a compact, battery-powered cooler designed specifically to sit in the passenger seat: it keeps things just as cold for your journey but uses a fraction of the size and power.
  • Example: Deploying a quantized Llama-3 8B model on an NVIDIA Jetson Orin Nano, achieving local conversational response generation at the edge for an autonomous robot using only 15 watts of power.

JIT (Just-in-Time)​

Just-in-Time (JIT) compilation is a method where computer code is translated into fast, optimized machine instructions during runtime (as the program runs), rather than ahead of time before the program starts.

  • Why it matters: Python is slow because it executes code line-by-line. In deep learning, calling Python code for every individual operation on the GPU causes huge overhead. A JIT compiler solves this by watching what operations the code performs, grouping them together, and dynamically compiling them into a single, high-performance kernel that executes directly on the hardware.
  • How it works: When a JIT compiler encounters a block of code, it first traces the operations to build a graph. It then optimizes this graph (for example, by fusing adjacent math operations to avoid slow memory reads) and compiles it into machine instructions specifically tuned for the CPU or GPU it is running on.
  • Analogy: Imagine a chef who receives individual orders for a table (e.g., "bring one glass of water," then "bring one plate of pasta," then "bring a fork"). A standard Python program is like a waiter walking back and forth to the kitchen for each individual request. JIT is like a smart waiter who waits until the table is finished ordering, writes a consolidated list (the optimized graph), and brings everything out in a single trip.
  • Example: In PyTorch, using torch.compile runs a JIT compiler under the hood (via TorchDynamo and Inductor) to analyze your model's forward pass and compile it into optimized Triton kernels, often speeding up training and inference.

Jitter​

The undesired variation in the time delay between periodic events, such as spikes or fluctuations in the execution rate of a control loop or message delivery. In robotics, a control loop commanded to run at 1 kHz expects exactly 1.0 milliseconds between steps; jitter is the deviation from this target (e.g., a step takes 1.2 ms, then 0.8 ms). High jitter can introduce instability in physical systems, causing jerky motions, tracking errors, or actuator damage. Analogy: Clapping your hands exactly once every second. If you clap at 1.0s, 2.0s, and 3.0s, you have zero jitter. If you clap at 1.1s, 1.8s, and 3.2s, your rhythm is erratic; that timing variability is jitter. On standard operating systems, background processes can interrupt the controller thread, causing high jitter; this is solved by running on a real-time kernel like PREEMPT_RT.

Kernel​

A specialized function designed to run in parallel across many execution threads on a GPU (or CPU) to carry out a single bulk operation, such as a matrix multiply or an element-wise addition. Analogy: A recipe given to thousands of cooks in a massive kitchen, where each cook follows the exact same instructions to chop their own individual vegetable. Instead of one chef chopping every vegetable one after another (a single thread), the kitchen executes the "chopping kernel" all at once across thousands of cooks (threads), transforming the entire heap of vegetables in a single step.

Kernel fusion​

Combining several sequential mathematical operations into a single kernel to reduce memory access overhead. In deep learning, many operations (like activation functions or normalizations) are memory-bound, meaning the time spent reading and writing data to slow global memory (HBM) dominates the compute time. Fusing these operations keeps intermediate values in fast registers on the chip, avoiding slow memory roundtrips. Analogy: A worker who needs to wash, dry, and fold a shirt. Instead of walking the shirt to the dryer in another building after washing (writing to global memory), and walking it back to fold (reading from global memory), the worker performs all three steps at a single workstation (in registers) before putting the finished shirt away. Example: Combining a LayerNorm normalization operation and a linear projection into a single Triton kernel avoids writing intermediate normalized values to HBM and reading them back, saving substantial memory bandwidth.

Keyframe​

A frame chosen to anchor a specific moment in a video — the picture you fix in place first and then build motion around. The term comes from hand-drawn animation, where a lead artist draws the important "key" poses and assistants fill in the frames between them. In long-form generation you create a few keyframes spread seconds apart to lock down how the scene looks at those points, then use an image-to-video or frame-interpolation model to invent the frames in between — a form of hierarchical generation. Choosing keyframes well matters: two that are too different cannot be smoothly bridged.

KF​

Kalman Filter (KF) is an algorithm that estimates the true state of a system (such as the position and speed of a moving object) by combining noisy sensor measurements with a physics-based prediction of how the system should behave. It uses a two-step cycle:

  1. Predict: Use physics equations to estimate where the system should be right now (e.g., "based on my previous speed, I should have moved 1 meter forward").
  2. Update: Look at the new sensor measurement (e.g., GPS says "you moved 1.2 meters forward"). Since the sensor is noisy, the filter balances the physics prediction and the sensor reading based on their relative uncertainties, finding a smart "middle ground" estimate.

Analogy: Tracking your position while walking inside a dark tunnel. You can estimate where you are by counting your steps (the physics/prediction step), but you will drift and lose accuracy over time. Occasionally, you catch a brief, blurry glimpse of a signpost through a crack in the wall (the noisy sensor measurement). You wouldn't rely solely on your step count, nor would you completely trust a quick, blurry glance. Instead, you combine both clues—balancing how confident you are in each—to make your best guess of where you are.

Example: A smartphone tracking its position on a map. GPS signals are noisy and jump around, while the phone's accelerometer measures acceleration to predict motion. A Kalman Filter combines both: if the GPS signal becomes noisy or temporarily drops out, the filter relies more on the accelerometer's predictions to keep the map pointer moving smoothly, rather than letting it jump wildly or freeze.

Kinematic bicycle model​

A simple model of a car or wheeled robot that pretends it has just two wheels — one front, one rear, like a bicycle — to make its motion easy to predict. The state is the vehicle's position and heading; the controls are its speed and the front wheel's steering angle, and a handful of trigonometry equations turn those into how the position and heading change each instant. It is "kinematic" because it tracks only geometry (where things are and how they point), ignoring mass, tire forces, and slip — a good enough approximation at the modest speeds of most mobile robots. Its close cousin the unicycle model drops the steering linkage and commands turning rate directly; the bicycle model is the better fit when an explicit steering angle matters (like a real car). Both share the nonholonomic constraint that the vehicle cannot move sideways. Analogy: modeling a shopping cart by its two fixed back wheels and one steerable front caster, enough to predict its path without simulating every wheel and bearing.

Kinematic redundancy​

Having more joints than the task strictly needs, so that many different joint configurations achieve the very same end-effector pose. A 7-DoF arm reaching a 6-DoF target (3D position + orientation) has one spare degree of freedom: it can swing its elbow through a whole arc while the hand stays perfectly still. That freedom lives in the null space of the Jacobian, and exploiting it lets the robot chase secondary goals — dodging obstacles, staying away from joint limits, keeping a comfortable posture — without disturbing the main task. Analogy: you can sign your name with your wrist locked or loose; either way the pen writes the same letters, but the loose wrist gives you room to also avoid knocking over your coffee mug.

Kinematic singularity​

A joint configuration where the robot momentarily loses the ability to move its end-effector in some direction, no matter how it drives the joints. Mathematically the Jacobian loses rank there (some of its output directions collapse onto each other), so the matrix can no longer be cleanly inverted. The famous example is the wrist singularity, when three wrist axes line up and one rotational direction simply disappears — following a path straight through it would demand infinite joint speed. Analogy: a fully straightened elbow is singular — to move your fingertip further along the line of your arm you would have to extend past straight, which you cannot do. Damped least-squares is the standard trick for passing near singularities without the joint speeds exploding.

Kinematics​

The study of a robot's motion (position, velocity, and acceleration) without considering the forces or torques that cause it. It describes the geometric relationship between the robot's joints and its physical body.

  • Why it matters: Before you can calculate how hard a motor needs to push (dynamics), you must first understand where the robot's limbs will end up when its joints rotate. Kinematics provides the fundamental mapping from joint movements to physical coordinates in the world.
  • How it works: Kinematics is split into two directions. Forward kinematics calculates where the robot's hand or foot is, given the angles of its joints. Inverse kinematics does the reverse: it calculates the joint angles needed to place the hand or foot at a specific target position.
  • Analogy: Imagine a puppet whose strings are pulled to move its limbs. Kinematics is the description of how the puppet's legs and arms bend and where they are in the air relative to the main body, without thinking about how heavy the puppet is or how hard you have to pull the strings.
  • Example: In a walking quadruped (four-legged robot), kinematics is used to determine how to bend the shoulder and knee joints to place a foot flat on the ground at a specific point, which is crucial for planning steps.

KL divergence​

Short for Kullback-Leibler divergence — a number that measures how far one probability distribution has drifted from another, growing larger the more the two disagree. In RLHF it acts as a leash on the policy being trained: the further its word probabilities wander from the frozen reference model, the bigger the penalty it pays. Like a tether that lets a climber explore but stops them straying somewhere dangerous, it lets the model chase reward without forgetting how to talk sensibly.

KV cache​

A scratchpad that stores the attention keys and values already computed for every earlier token in the sequence, so generating the next token only has to compute keys and values for that one new token instead of redoing all the previous ones. Like writing out a long multiplication table once and then looking up products instead of recalculating them — it turns each decode step from "redo the whole prompt" into "do one more token," which is what makes long-context serving fast enough to be usable.

L2 cache​

A large, chip-wide, high-speed cache memory on a GPU that is shared across all Streaming Multiprocessors (SMs). The L2 cache acts as an intermediate storage layer between fast on-chip memories (like shared memory and registers) and the much slower off-chip High-Bandwidth Memory (HBM). Accessing L2 cache takes roughly 200 cycles, which is about twice as fast as accessing HBM.

  • Analogy: A small, shared filing cabinet in the middle of an office floor. If an employee (SM) needs a document, they first check their desk (registers) or their office whiteboard (shared memory/L1). If it's not there, they check the filing cabinet in the hallway (L2 cache) before going all the way to the central archives in the basement (HBM), which takes much longer.
  • Example: In a matrix-multiplication or attention kernel, if multiple blocks need to read the same input rows or columns, those inputs are loaded from HBM into the L2 cache during the first block's request. Subsequent blocks reading the same data can fetch it directly from the L2 cache, saving valuable HBM bandwidth and increasing the overall roofline performance limit.

L2 normalization​

Rescaling a vector so its length becomes exactly 1 while keeping the direction it points unchanged — done by dividing every element by the vector's own length. It is called L2 because the length it uses is the L2 norm (also called the Euclidean norm — the ordinary straight-line distance you would measure with a ruler). The "2" comes from the p in the general Lp norm formula, which takes the p-th root of the sum of each element's p-th power; set p = 2 and that becomes the square root of the sum of squares — exactly the Pythagorean length √(x₁² + x₂² + …). Worked example: [3, 4] has length √(3² + 4²) = √25 = 5, so its L2-normalized form is [3/5, 4/5] = [0.6, 0.8] — same direction, but now length 1 and sitting on the unit sphere. Analogy: shrinking every arrow on a map to the same one-inch length so you can compare which way they point without the longer arrows drowning out the shorter ones. This is the step that turns a plain dot product into cosine similarity, which is why CLIP L2-normalizes every image and text embedding before scoring matches — so only direction (meaning), not magnitude, decides the score.

L2 regularization​

A regularization technique that adds a penalty proportional to the squared magnitude of model weights to the loss function, encouraging smaller weights and reducing overfitting. In standard adaptive optimizers such as Adam, this penalty is folded into the gradient and scaled by the adaptive learning rate, which is why AdamW uses decoupled weight decay instead.

L2-regularized least squares​

A way to find the best-fit solution to a system of equations when the problem is ill-conditioned — meaning the matrix to invert is near-singular and a plain solution would blow up. You add a small penalty λ² times the squared size of the answer to the cost, which keeps the solution bounded even when the equations nearly conflict. Mathematically, instead of solving Jx = e for x (which involves inverting JJᵀ), you solve (JJᵀ + λ²I)x = e, which adds a small number λ² along the diagonal before inverting. The λ² term acts like a spring pulling the answer toward zero: a tiny spring lets the solution vary freely but prevents infinity; a large spring forces a very small, safe answer at the cost of accuracy. Also called ridge regression in statistics. The name comes from L2 regularization: the penalty is the squared L2 norm of the solution. In robotics, damped least-squares IK applies this idea to the Jacobian to prevent joint velocities from exploding near singularities.

LAION​

A family of huge, openly released image-text datasets (LAION-400M, LAION-5B — the number counts the image-caption pairs) scraped from the public web by the non-profit LAION (short for Large-scale Artificial Intelligence Open Network). Each entry is just an image URL plus its alt-text caption, kept only if CLIP judged image and caption to roughly match. It is the public fuel that trained Stable Diffusion and many other open models. Like a giant secondhand library assembled by photographing every captioned picture on the open internet — enormous and free, but riddled with mislabeled, duplicated, and low-quality entries, which is why every serious user re-filters and deduplicates it before training. Example: "LAION-2B-en" is the roughly 2-billion-pair English-caption subset.

Langevin dynamics​

A way to draw samples from a distribution when you only know its score — the gradient of its log-density. You start from a random point and repeatedly take a small step in the score direction (uphill toward higher probability) while also adding a little random noise each step so you explore rather than collapse onto a single peak. The uphill pull plus the random shake settles the point into high-probability regions in the right proportions, like a ball jiggling around a bumpy bowl and spending most of its time in the deepest dips. It is the sampling method behind the original score-based generative models. Named after the physicist Paul Langevin.

Latency​

The time or delay elapsed between initiating a request and receiving the system's response. In large language model serving, latency is typically split into TTFT (Time To First Token) and ITL / TPOT.

  • Why it matters: For interactive applications like real-time chatbots, low latency is critical to ensure a responsive user experience so users do not feel they are waiting.
  • Analogy: The time it takes for a single commuter to drive from home to work. If the roads are clear, the commuter arrives quickly (low latency), regardless of how many other cars are using the highway.
  • Example: In an LLM interface, a target ITL / TPOT of under 30 ms per token provides a smooth, fluid reading experience that feels instantaneous to a human reader.

Latent action model​

A model that infers the action taken between two consecutive video frames when no action was ever recorded, by learning a small latent code that best explains how the first frame turned into the second. Train it on mountains of unlabeled video and it discovers, on its own, a compact and reusable vocabulary of "moves" — step left, jump, pan the camera — which is precisely what lets a world model like Genie become controllable without anyone hand-labeling a single action. Like watching thousands of silent chess games and deducing the set of legal moves purely from how the board changes between snapshots.

Lambda-return​

A weighted blend of all the ways you could estimate a return, written λ-return (λ is the Greek letter lambda). You can estimate what a state is worth from 1 real reward plus a value estimate of what follows, or from 2 real rewards plus a value estimate, or 3, and so on. Short estimates trust the value function and are stable but biased; long ones trust the observed rewards and are accurate but noisy. Rather than picking one, the λ-return averages them all with exponentially decaying weights, where λ (between 0 and 1) sets the decay: λ=0 uses only the 1-step estimate, λ=1 uses only the full observed return, and values in between interpolate. It is the same bias/variance dial as GAE, which is built from exactly this idea.

Latent dynamics​

Predicting how an environment evolves in a compressed latent code rather than in raw observations: an encoder squeezes each high-dimensional observation (say an image) into a small latent vector, and the model learns to step that code forward given an action — z, a → z′.

Why it is called "latent dynamics" instead of "latent model" or "latent space":

  • A latent space is just a static map or coordinate system of compressed points.
  • A latent model is any model that uses latent features (like a classifier that looks at compressed vectors).
  • Latent dynamics specifically describes the rules of motion and change within that latent space over time.

Here, "latent" means the states are hidden, compressed codes rather than raw pixels, and "dynamics" means the model predicts how those codes transition step-by-step (latent state + action → next latent state). Analogy: A "latent space" is like having a list of coordinate dots on a grid representing chess pieces. The "latent dynamics" are the rules of chess that tell you how those dots slide and jump across the grid when you make a move. It models the motion rather than just a static snapshot.

This is far cheaper than predicting every pixel and lets the model ignore visual detail that does not affect decisions, spending its capacity on what matters for control. Analogy: planning a road trip on a subway-style map of dots and lines instead of a full satellite photo — the stripped-down map keeps exactly the structure you need to navigate. It is the engine inside DreamerV3 and TD-MPC2, letting them "imagine" long rollouts quickly.

Latent space​

The compressed set of numbers a model uses to represent its data internally, after stripping away the raw detail. Each point in this space stands for one possible output, and nearby points usually mean similar outputs — so you can smoothly "walk" from one to another and watch the result morph. Think of it as the model's private map of its world: instead of a full 28×28-pixel image, an autoencoder might describe each digit with just 32 numbers, and that 32-number space is the latent space.

Latent video​

The compressed form of a video that a 3D VAE produces: instead of the raw (T, H, W, C) pixel tensor, you get a much smaller (T', H', W', C) grid where time, height, and width have all been shrunk (often ~100× fewer numbers overall). Modern video diffusion runs in this latent space rather than on pixels, because denoising a 100×-smaller tensor is what makes high-resolution video generation affordable at all.

LCM​

Latent Consistency Model — a consistency model distilled in the latent space of a VAE, giving 1–4-step Stable Diffusion-style sampling. It is the most practical few-step recipe for SD-style stacks, which is what makes near-interactive image generation possible.

LDM​

Latent Diffusion Model — a diffusion model that runs in the latent space of a VAE rather than on raw pixels. A VAE first compresses the image (or video) into a much smaller grid of numbers; the diffusion model learns to denoise that small grid, and the VAE decoder turns the finished latent back into pixels. Because the latent is often ~50–100× smaller than the image, every training and sampling step is dramatically cheaper, which is the whole reason high-resolution generation became affordable. Stable Diffusion is the canonical image LDM; modern video models apply the same idea on top of a 3D VAE.

LFQ​

Lookup-Free Quantization — a way to turn a continuous latent into a discrete token without a learned codebook. Instead of comparing each latent vector against a trained table of code entries and picking the nearest (the VQ-VAE way), LFQ squashes each latent dimension to a sign — roughly, "is this number positive or negative?" — so the pattern of signs across the dimensions is the integer code. With no table to look up, there is nothing that can go unused, which sidesteps codebook collapse and lets the effective vocabulary grow huge cheaply. It is the quantizer behind MagViT-v2 and a close cousin of FSQ, which snaps each dimension to a small grid of levels rather than just a sign.

Leaderboard​

A public ranking that lists models by their score on one or more benchmarks, best at the top — like a sports league table for AI models. It makes progress easy to see at a glance, but a single number hides many hidden choices (prompt wording, answer parsing, image resolution), so two groups can report different scores for the same model; a high rank is also suspect if the test questions leaked into training (see contamination). Example: the MMMU leaderboard ranks VLMs by their accuracy on the MMMU exam, and a new model's headline claim is usually "we moved up this board."

Learnable​

Refers to parts of an AI model (like weights or parameters) that are not set in stone by the programmer, but are instead adjusted automatically during training to improve performance. Like the knobs on a radio that tune themselves until the station comes in perfectly clear, rather than being glued in place.

Learning rate​

The step size an optimizer takes when nudging the weights along the gradient. Too large and training overshoots and diverges; too small and it crawls — like choosing how big a step to take walking downhill in fog. It is usually ramped up during warmup and then decayed over the run.

Learning rate annealing​

Decaying the learning rate over the course of training — in PPO's case, linearly all the way to exactly zero at the final step. It is detail #4 of the 37 PPO implementation details.

  • Why it matters in RL specifically: In supervised learning, decaying the learning rate is about settling into a minimum. In RL there is an additional and sharper reason: the guarantee that a policy-gradient step improves the policy is a local one, and the cost of overshooting is not merely a worse loss but a worse policy, which then collects worse data, which teaches a worse policy still. Early on the policy is bad and there is little to lose from a large step; late in training it is good and a single bad update can undo everything. Shrinking the step as the policy improves is insurance against that ratchet.

Legged locomotion​

The process of moving a robot or animal from one place to another using articulated legs that make discrete contact with the ground.

  • Why it matters: Wheeled robots are highly efficient on flat surfaces like roads, but they get stuck on uneven terrain, stairs, or obstacles. Articulated legs allow robots (such as bipeds, quadrupeds, or hexapods) to step over obstacles, climb stairs, and traverse rough outdoor terrains that are inaccessible to wheels.
  • How it works: It requires coordinating multiple joint motors to cycle each leg through a stance phase (where the foot is firmly planted on the ground to support and push the body) and a swing phase (where the foot is lifted and moved forward). This forms a hybrid dynamic system because the physical equations governing the robot change instantly whenever a foot touches down or lifts off.
  • Analogy: Imagine walking across a stream by stepping on slippery stones. Unlike a car that rolls smoothly over a bridge, you must plan exactly where to place each foot, balance your body while standing on one foot, and swing your other foot forward without tripping.

Length bias​

The tendency of RLHF-tuned models to drift toward longer and longer answers over training, whether or not the extra words actually help. It happens because human preference data — and the reward model trained on it — quietly correlate length with quality (a longer answer looks more thorough), so the policy discovers it can raise its score just by padding. It is a concrete, easy-to-measure case of reward hacking: the model maximizes a flawed proxy instead of true helpfulness. Length-aware losses such as SimPO (a DPO-family variant) explicitly normalize for or penalize length to counteract it.

LiDAR​

Light Detection and Ranging — a sensor that measures distances by firing rapid laser pulses at objects and timing how long they take to reflect back. Because the speed of light c is constant (roughly 300,000,000 meters per second, or c ≈ 3 × 10⁸ m/s), the sensor computes the distance d to an object from the time-of-flight Δt as: d = (c × Δt) / 2 (dividing by two because the light travels to the object and back). Doing this millions of times a second in all directions builds a detailed 3D point cloud of the environment. Unlike cameras, which can be fooled by shadows, glare, or darkness, LiDAR provides its own light and measures geometry directly, making it crucial for self-driving cars, drones, and mapping robots (SLAM). Analogy: standing in a pitch-black room with a stopwatch and a laser pointer, shooting a pulse at the wall, and timing its return. By doing this in every direction, you can locate all the walls and furniture to map the room without ever turning on the lights.

Lens distortion​

The way a real lens bends light so that straight lines in the world come out slightly curved in the image — most visibly barrel distortion, where a wide-angle lens makes a square bulge outward like a barrel, and its opposite, pincushion. It grows toward the edges of the frame, so a checkerboard photographed near the corners shows bowed rows. Camera calibration measures a few distortion coefficients alongside the camera intrinsics, and the image is then undistorted — warped back so lines are straight — before any geometry is computed, because otherwise every pixel-to-3D calculation inherits the curve as an error.

Lifecycle node​

A design pattern in robotics middleware (specifically ROS 2) where a software node manages its state explicitly through a standardized state machine. Instead of starting up and immediately running, a lifecycle node transitions through set states—such as Unconfigured, Inactive, Active, and Finalized—triggered by external lifecycle events. This enables a supervisor system to control the startup order, handle dynamic reconfiguration, and manage runtime errors (e.g., if a sensor disconnects, the node can transition to Inactive or Unconfigured to execute a recovery sequence). Analogy: A modern smart factory machine with a physical control panel showing states like "Ready," "Running," and "Paused," where you must manually press buttons to transition from startup to active operation, rather than a legacy machine that immediately starts spinning the moment it is plugged in.

Having chosen a direction to move the parameters in, deciding how far along it to actually step, by trying a step, checking whether it did what it promised, and shrinking it until it does. "Backtracking" line search starts at the largest step allowed and repeatedly multiplies it by a factor below 1.

  • Why it matters in RL: It is TRPO's last line of defence, and the reason TRPO can honestly claim to enforce its constraint. TRPO computes its step from a local approximation: a linear model of the objective and a quadratic model of the KL divergence. Those models are true only very near the current parameters, so the step they recommend may in reality both fail to improve the policy and violate the trust region. The line search checks the true objective and the true KL at the proposed point, and backs off until both promises actually hold — or, if none does, takes no step at all. This is the mechanism PPO throws away and replaces with a clamp.

Linear probe​

A small linear classifier trained on the frozen hidden activations of a layer of a neural network to test whether that layer has already encoded some property — for example, "is this sentence true?", "what is the capital of this country?", or "which language is this?" Like sticking a voltmeter into one wire of a circuit to see what signal is flowing past that point; you don't change the circuit, you just read what's already there. The standard first tool in mechanistic interpretability.

The physical mass (weight) of an individual rigid link (such as a forearm, upper arm, or torso) of a robot. In robotics, a robot's body is modeled as a chain of rigid links connected by joints; the mass of each individual link is a fundamental dynamic parameter that directly determines how much gravitational force acts on it and how much inertia it has when moving.

  • How it works: When a robot controller wants to move the robot along a path, it must calculate the necessary joint torques (using algorithms like computed-torque control or impedance control). To do this, it needs to know the mass, center of mass, and rotational inertia of each link. If the controller's model assumes a link is lighter than it actually is, the motors will not apply enough force, causing the robot to lag behind its target path or even drop under gravity.
  • Analogy: Imagine wearing heavy ankle weights while trying to run. If your brain (acting as the controller) does not account for the extra mass of your lower legs (the links), you will swing your legs too slowly. Just like your brain, a robot's controller needs to know the exact mass of each link to coordinate the correct muscle effort (motor torque) for movement.
  • Example: During the design and setup of a robotic manipulator, engineers obtain the mass of each link from CAD models or estimate them using system identification (for instance, by commanding joint motions using chirp signals and measuring the motor torques). These values are then written into the robot's URDF file so the control software can calculate the correct gravity compensation and inertia matrices.

Lipschitz constraint​

A limit on how fast a function's output can change as its input changes: a 1-Lipschitz function never changes its output by more than the distance you moved the input. Picture a road whose slope is capped so it can never get steeper than 45° — no cliffs allowed. (The name simply honors the 19th-century German mathematician Rudolf Lipschitz, who first wrote down this "bounded-steepness" condition; it is not a description of the rule itself, the way "Celsius" is just a person's name rather than a word about temperature.) Wasserstein GANs require their critic to obey this so the Earth Mover's Distance it estimates stays valid, which is what the gradient penalty enforces.

llama.cpp​

An open-source, lightweight software library written in C/C++ by Georgi Gerganov that performs high-performance local inference on various large language models (LLMs), such as LLaMA. It is designed to run with minimal dependencies across a wide range of hardware, including CPUs, consumer GPUs, and Apple Silicon, using quantized models stored in the GGUF format.

  • Analogy: Imagine a tiny, highly efficient portable generator (llama.cpp). Instead of needing to connect to a huge, complex power grid (the enterprise ML container stack with PyTorch, CUDA, etc.) just to power a light bulb, you plug your light bulb directly into this small, simple generator, which runs on almost any fuel (CPU, GPU, Mac) with very little overhead.
  • Example: Running a 7B parameter model at 4-bit quantization on a standard laptop CPU using llama.cpp's command line interface, achieving interactive token generation speeds without any GPU.

LLaVA​

Large Language and Vision Assistant — an open-source vision-language model that shows how far the simplest possible design can go: take a frozen CLIP image encoder, take a frozen LLM, and connect them with nothing but a lightweight projector (a single linear layer or small MLP) that translates each image patch's feature vector into the LLM's word-embedding space. The LLM then "reads" the image as if it were a sequence of extra words. Think of a United Nations translator who listens to a speech in one language and re-phrases each sentence for a listener who only speaks another — the translator (projector) does not change the content, just the format. Despite having no cross-attention or Q-Former, LLaVA matches or beats far more complex architectures on many visual-question-answering benchmarks, which is why its projector-only design became a widely-copied template. Compare with Flamingo, which uses gated cross-attention instead.

LLM​

Large Language Model — a transformer trained on large amounts of text to predict and generate language.

LLM-as-judge​

Using a strong LLM to grade or compare other models' answers in place of a human rater — fast, cheap, and surprisingly well-calibrated, though it tends to favor longer answers and ones written in its own style. To catch position bias you usually ask twice with the two answers swapped and trust only an agreeing verdict — like a blind wine tasting where the same two bottles are poured first as "Glass A, Glass B" and then again as "Glass B, Glass A"; you only believe the judge picked the better wine if they pick the same bottle both times, because that rules out them simply liking whichever glass sat on the left.

Load balancing​

Spreading incoming requests across several copies of a service so no single one is overwhelmed while others sit idle — like a supermarket opening more checkout lanes and a greeter waving each new customer to the shortest one. The simplest rule is round-robin (hand requests out in turn, 1-2-3-1-2-3…); smarter rules send each request to the least-busy replica or to the one whose cache is already warm. The component that does this is a load balancer.

Load shedding​

Deliberately dropping or rejecting some requests when a server is overloaded, so the ones it does accept still meet their targets. Returning a fast "try again later" to low-priority traffic is far kinder than letting every request crawl — like a busy restaurant turning new walk-ins away so the diners already seated still get served on time. It usually works hand in hand with admission control and request priority.

Log-derivative trick​

The piece of calculus that makes policy gradients computable. The problem: you want the gradient of an average taken over actions the policy samples, but the policy itself decides which actions get sampled, so the thing you average over keeps moving as you change the weights. The trick uses the identity ∇p = p · ∇log p (rearranged from ∇log p = ∇p / p, the ordinary derivative of a logarithm) to rewrite that awkward gradient as a plain average of ∇log π(a|s) weighted by how good the action was — and a plain average is something you can estimate just by sampling actions and watching the rewards. In short, it converts "the gradient of an expectation" into "an expectation of a gradient," which is why REINFORCE can learn from sampled rollouts without ever needing a model of the environment. It is also called the REINFORCE trick or score-function estimator.

Log-probability​

The logarithm of the probability a policy assigned to the action it actually took, written log π(a|s). Nearly every policy-learning algorithm works with the log rather than the raw probability, for three practical reasons: multiplying many small probabilities underflows to zero in floating point while adding their logs does not; the log-derivative trick that makes policy gradients computable is written in terms of ∇ log π; and ratios of probabilities (as in PPO's importance ratio) become cheap subtractions of logs. One trap catches everyone once: for continuous actions π(a|s) is a probability density, not a probability, so it may exceed 1 and its log may be positive — that is not a bug. Densities also change when you transform the action, which is why tanh squashing demands a correction term.

  • Analogy: Reporting earthquake strength on the Richter scale instead of raw ground motion. Nothing about the earthquake changed; the log scale just keeps the numbers in a range you can add up and reason about without them collapsing to zero.
  • Example: SAC uses the log-probability twice per update — once as the entropy bonus inside the critic's target, and once as the signal that automatic temperature tuning servos against.

Logits​

The raw, unnormalized scores a model produces at its output, one per vocabulary entry, before they are turned into probabilities by softmax. Like the points each contestant has scored at the end of a game — bigger means "more likely the next token" — but to read them as percentages you have to normalize. Sampling rules (temperature, top-k, top-p) all reshape the logits before the random draw, and argmax of the logits is what greedy decoding picks.

Long-horizon autonomy​

Long-horizon autonomy is the capability of an autonomous agent or robot to successfully plan, coordinate, and execute complex sequences consisting of dozens or hundreds of sequential actions over an extended time frame without human intervention.

  • Why it matters: In short-horizon tasks (like picking up a block), minor errors are easily corrected. In long-horizon tasks, however, errors compound exponentially over time. A single mistake at step 40 of a 50-step sequence can cause catastrophic failure for the entire mission, making long-horizon tasks the ultimate test of a robot's robustness and error-recovery systems.
  • How it works: Achieving long-horizon autonomy requires integrating high-level semantic planners (to decide what to do next) with robust low-level controllers (to carry out the actions). The evaluation of these systems is typically measured by tracking the compounding success rate. If a robot has a per-step success rate of p = 95% on a task requiring N = 50 steps, the overall probability of completing the task is p^N = 0.95^50 ≈ 7.7%, demonstrating how small individual errors lead to overall failure.
  • Analogy: Imagine a domino rally with 100 dominoes. To succeed, every single domino must fall in the correct direction. If the 45th domino falls slightly off-center and misses the next one, the entire sequence stops, and the run fails. The longer the chain of dominoes, the more perfect each transition must be.
  • Example: A warehouse mobile robot is tasked with patrolling a facility, opening doors, picking up parts, and delivering them to assembly stations. This task takes 1 hour and requires hundreds of coordinated navigation and manipulation actions. Measuring overall mission success rather than simple navigation steps is a long-horizon evaluation.

Loop closure​

In visual odometry and SLAM, the moment a robot recognizes it has returned to a place it visited before, and adds a constraint linking "here" to "there." Because position estimates accumulate drift, a robot that walks a big loop and comes back will believe it is meters away from its true starting point; recognizing the revisited spot lets the system pull the whole looped path back into agreement, erasing the built-up error in one correction. Analogy: tracing a circle with your eyes closed and, the instant a fingertip tells you you are back at the start, fixing the wobbly path you drew so its two ends meet.

LoRA​

Low-Rank Adaptation — a cheap way to fine-tune a huge model without rewriting it. Instead of changing the model's billions of frozen weights, you leave them all untouched and bolt on a tiny pair of extra low-rank matrices that nudge the output. Why a pair and not a single matrix? A lone update matrix would have to be the same full size as the weights it is correcting — which defeats the whole point of saving space. The trick is to split that update into two skinny matrices in a row: the first squeezes the big input down to just a handful of numbers, and the second expands those few numbers back out to full size. Picture an hourglass — wide, pinched to a narrow waist, then wide again: it is the narrow waist in the middle (the low rank) that keeps the total number of stored values tiny, and you need both halves of the hourglass to get from one side to the other. Like leaving a printed textbook exactly as it is and slipping in a few sticky notes that change how you read it: the notes are small to store, quick to write, and you can keep a different set of notes for each task and swap them in and out.

Loss function​

A mathematical function that measures the difference between a model's prediction and the actual target. The goal of training is to minimize this value using gradients.

Loss masking​

Telling the trainer to compute the loss only on the tokens you want the model to learn to produce — in SFT, the assistant's reply — and to ignore the rest, like grading only a student's answers and not the printed questions.

Loss scaling​

A technique used during mixed-precision training (specifically with float16) that multiplies the loss by a large factor before the backward pass to prevent gradients from underflowing (rounding down to zero). The gradients are scaled back down (divided by the same factor) before the optimizer updates the model's weights.

  • Analogy: Imagine trying to write down extremely small measurements, like 0.000003 meters, but your notepad only lets you write numbers as small as 0.0001 (anything smaller gets rounded to 0). To solve this, you multiply all your measurements by 10,000 before writing them down, turning 0.000003 into 0.03. When you are ready to do the final construction, you divide the final blueprints back by 10,000 to get the actual sizes.
  • Example: In PyTorch mixed-precision training, the GradScaler utility automatically scales the loss by a factor (such as 65536) before running the backward pass. If a training step produces gradients that are too large (overflow), the scaler automatically halves the scaling factor and skips that step, adjusting the scale dynamically to keep gradients within the representable range of float16.

Loss spike​

A sudden jump in the training loss, usually from an outlier batch or optimizer instability; small spikes are normal, but a diverging one can ruin a run.

Loss value​

The single scalar number produced by evaluating the loss function on a model's predictions. autograd's backward pass computes gradients of this one scalar with respect to every parameter, which is what makes reverse-mode differentiation efficient.

Lorax / S-LoRA​

Multi-LoRA serving engines; one base model + many adapters in HBM

Low-rank​

A way of approximating a big matrix as the product of two much skinnier ones, capturing most of its information with far fewer numbers. A full 1000×1000 weight matrix holds a million entries, but if its real content is "low rank" you can rebuild it well from, say, two 1000×8 matrices — a few thousand numbers instead of a million. Like mixing any shade of paint from just a few primary colors instead of stocking thousands of separate tubes—you capture the full variety using only a handful of basic components. This is the trick behind LoRA: freeze the giant base weights and learn only a small low-rank update on top.

Low-resource language​

A language for which little digital training data exists — few transcribed recordings, books, or web pages — compared with high-resource languages like English or Mandarin that have billions of words online. Models trained mostly on the abundant languages do worst here, simply because they have not seen enough examples to learn the language's sounds and spellings. Like a cook who has made thousands of Italian dishes but tasted Ethiopian food only once — they will be shaky at Ethiopian cooking until they practice it specifically. Example: Whisper transcribes English almost perfectly but makes far more errors on a language like Welsh or Amharic, which is exactly where a few hours of targeted fine-tuning data helps most.

LQR​

Linear-Quadratic Regulator — the optimal feedback controller for a linear system whose cost is defined as a quadratic (squared) function of state deviation and control effort, which is where the name comes from. You write the system in state-space form ẋ = Ax + Bu and choose two weight matrices: Q specifies the priority of keeping the state close to its target, and R specifies the priority of conserving control effort. LQR solves for a single constant gain matrix K so the control law is just u = -Kx, pushing back in proportion to how far off you are. The beauty of this linear-plus-quadratic setup is that the optimal controller has this simple form, and K can be computed once, offline, by solving a matrix equation (the Riccati equation). It works far beyond truly linear systems because you can linearize a nonlinear robot about a setpoint (an upright pole, a hovering drone) and apply LQR to the approximation.

Analogy: Q and R are two dials on a thermostat—one for "how strongly you prioritize maintaining the exact target temperature" and one for "how much you prioritize saving energy on the power bill"—and LQR works out the single best way to balance them.

LSTM​

Long Short-Term Memory — a type of recurrent neural network (RNN) cell designed to remember things over long sequences without the information fading away. A plain RNN is like whispering a message down a long line of people — by the end, the message is garbled. An LSTM fixes this with three gates: a forget gate that decides what old information to throw out, an input gate that decides what new information to store, and an output gate that decides what to actually hand to the next step. Together they maintain a "cell state" — a conveyor belt of memory that can carry important facts across hundreds of time steps with minimal loss. LSTMs were the go-to architecture for sequences (language, speech, time series) before transformers took over, and they remain the classic example of gated memory in neural networks.

LunarLander​

A classic Gymnasium control task where the agent flies a small spacecraft and must land it gently on a pad between two flags. At each step it reads an 8-number state (position, velocity, angle, and whether each leg has touched down) and fires one of four thrusters; reward comes from landing softly and on target, with penalties for crashing or wasting fuel. It is a step up from CartPole — the rewards are sparser and the dynamics less forgiving — which makes it a standard proving ground for policy-gradient and actor-critic agents like A2C and PPO. A continuous-control variant replaces the four discrete thrusters with smoothly adjustable engine power.

MagViT-v2​

The strongest open recipe for discrete video tokenization — turning a clip into a grid of integer tokens that an autoregressive or transformer model can generate the same way it generates language. It builds on the VQ-VAE idea of a discrete latent but replaces the learned codebook with LFQ (lookup-free quantization), which sidesteps codebook collapse and scales to a very large vocabulary cheaply. A single MagViT-v2 tokenizer handles both still images and video (it shares the causal trick of encoding the first frame on its own), and its reconstructions are sharp enough that token-based generators can finally rival diffusion models on quality — its headline claim is that a good enough tokenizer is what makes language-model-style video generation competitive.

Manifold​

The thin, curved surface inside a much larger space where real data actually lives. A 32×32 color image is a point in a space of 3,072 numbers, but almost every random point in that space looks like static — only a vanishingly small, smoothly connected sliver of it looks like a real photo, and that sliver is the manifold. A useful analogy: a sheet of paper is a 2D surface, but if you crumple it and drop it into a room it traces out a thin curved shape floating in 3D space; the paper is the manifold and the room is the full space. Learning to generate images is largely learning the shape of this surface so you only ever land on it.

Manipulability​

Scalar measure of how "easy" motion is from a given configuration (e.g. sqrt(det(JJᵀ)))

Manipulation​

The branch of robotics concerned with how a robot physically interacts with, moves, and alters objects in its environment. While mobile robots focus on moving through space, manipulation focuses on grasping, placing, inserting, assembling, cutting, or otherwise changing the state of objects, typically using a robot arm and a specialized end-effector or gripper.

  • Analogy: If a self-driving car or a delivery drone is a robot focusing on travel, a robotic arm assembling a car or a kitchen robot cutting vegetables is a robot focusing on manipulation. Travel is about where the robot is; manipulation is about where the objects are.
  • Concepts: Robotics manipulation is divided into rigid-body manipulation (such as pick-and-place) and deformable manipulation (such as cloth folding), and uses techniques ranging from analytic grasp synthesis to end-to-end visuomotor policies.

Mantissa​

The part of a floating-point number that holds the precision digits — the significant figures sitting in front of the scale factor. In 3.5 × 10¹², the 3.5 is the mantissa (also called the significand). More mantissa bits give finer resolution between nearby values; fewer mantissa bits leave larger gaps between representable numbers. FP8's E4M3 format means 4 exponent bits + 3 mantissa bits, so it can only distinguish about 8 distinct values between each consecutive power of two — coarse, but small enough to fit twice as many numbers in the same memory as bfloat16.

Manipulator equation​

The master equation of rigid-body dynamics for a robot arm — the one relationship every torque-level controller and simulator is built around. It reads M(q)q̈ + C(q,q̇)q̇ + g(q) = τ + Jᵀ·F_ext, and each piece has a plain meaning. M(q) is the mass matrix: how much the arm resists being accelerated, which changes with its configuration q (a stretched-out arm is harder to swing than a tucked one). C(q,q̇)q̇ collects the Coriolis and centrifugal terms — the velocity-dependent forces a moving, rotating chain feels, the same effect that makes a spinning skater's arms get flung outward. g(q) is the torque needed just to hold the arm up against gravity. τ is the joint torque you apply, and Jᵀ·F_ext is any external push (a contact or payload) mapped into joint space through the Jacobian transpose. Read left-to-right it says: the torques you command, plus outside forces, must supply exactly the inertia, velocity, and gravity terms the desired motion requires. Computing τ from a desired motion is inverse dynamics; computing the motion from given τ is forward dynamics.

Markov property​

The assumption that the current state already contains everything relevant about the past, so what happens next depends only on where you are now, not on how you got there. It is the "memoryless" condition that makes an MDP tractable: when it holds, a policy can ignore history and look only at the present state. Named after the mathematician Andrey Markov. Example: in chess the current board position is Markov (the move history doesn't change which moves are legal or good), but a single still photo of a moving ball is not — you can't tell which way it is heading without the previous frame. When the property fails, you have a POMDP.

Marlin​

A specialized GPU kernel for mixed-precision matmul — 4-bit weights multiplied by 16-bit activations — built to stay fast even on the skinny, small-batch shapes of decode. It unpacks the 4-bit weights on the fly while keeping the Tensor Cores busy, so a quantized model runs nearly as fast as the math allows. (Named after the fast-swimming marlin fish.)

MaskGIT​

A way to generate image tokens in parallel instead of one at a time. Starting from a grid where almost every token is hidden ("masked"), a transformer predicts them all at once, keeps only the predictions it is most confident about, and repeats over a handful of rounds until the grid is full. The analogy is filling in a crossword: lock in the answers you are sure of first, and the rest get easier. This makes it much faster than raster-order autoregressive generation, which must fill the grid one token at a time.

Mass matrix​

The part of the manipulator equation, written M(q), that captures how strongly a robot arm resists being accelerated — its rotational "heaviness" felt at the joints. It is the multi-joint generalization of the m in F = ma: instead of one number, it is a square, symmetric, positive-definite matrix (one row and column per joint) because pushing one joint also tends to swing the links attached to others, so the joints' inertias are coupled. It depends on the configuration q because the arm's effective inertia changes with its shape — a fully extended arm is far harder to accelerate than a tucked one, just as a figure skater spins faster with arms pulled in. Pinocchio's CRBA algorithm is the standard fast way to build it. It is what you must invert to go from torques to accelerations in forward dynamics, and the weighting that makes computed-torque control cancel the arm's inertia exactly.

matmul​

Matrix multiplication (matmul) is a mathematical operation that takes two grids of numbers (matrices) and multiplies them together to produce a new grid of numbers. In PyTorch, it is written as A @ B or torch.matmul(A, B).

  • Why it matters: Matrix multiplication is the engine of modern AI. Over 99% of the computational work inside deep learning models (like transformers or image generators) is matrix multiplication. When a model processes text or images, it uses matmul to compare input data against millions of learned weights at the same time.
  • How it works: To find the number that goes into a specific cell of the output grid, you take a row from the first matrix and a column from the second matrix, multiply their corresponding numbers, and add them up (this is called a dot product). You repeat this process for every row-column combination.
  • Analogy: Imagine you have a matrix listing three recipes (rows) and the amount of flour, sugar, and butter each recipe needs (columns). You also have a second matrix listing the prices of flour, sugar, and butter (rows) at two different grocery stores (columns). Doing a matmul on these two grids calculates the total cost of making each of the three recipes at both grocery stores in a single operation.
  • Example: In a neural network layer, the inputs are grouped into a matrix X and the layer's weights are grouped into a matrix W. Running X @ W applies all the weights to all the inputs simultaneously, allowing the network to extract features extremely quickly on parallel hardware like GPUs.

Matrix inverse​

For a square matrix A, its inverse A⁻¹ is the matrix that undoes it: A⁻¹A = I, where I is the identity matrix (the matrix equivalent of the number 1, leaving anything it multiplies unchanged). Multiplying by A⁻¹ is how you divide by a matrix, which is exactly what solving a system of linear equations Ax = b needs — the solution is x = A⁻¹b. In RL it gives the one-shot answer to policy evaluation: the Bellman equation for a fixed policy is the linear system (I − γPπ) V = rπ, so V = (I − γPπ)⁻¹ rπ. Analogy: if a matrix is a recipe that scrambles a list of numbers, its inverse is the recipe that unscrambles them. In practice you rarely form the inverse explicitly — solving the system directly (e.g. np.linalg.solve) is faster and more numerically stable — but the inverse is the cleanest way to think about the answer.

Maximum entropy RL​

A reinforcement-learning objective that rewards the agent not only for collecting reward but also for keeping its policy as random as it can while still doing well: it maximizes E[ Σ r_t + α · H(π(·|s_t)) ], where H is the policy's entropy (a measure of how spread-out its action choices are) and α, the temperature, sets how much that randomness is worth. The entropy term is a built-in, principled form of entropy regularization: the agent keeps exploring on its own and avoids committing to a single brittle action too early, which also makes the learned policy more robust to small changes in the environment. Analogy: a commuter who not only wants the fastest route but also deliberately keeps a few alternate routes in regular use, so a single road closure never strands them. SAC is the best-known algorithm built on this objective; standard RL is the special case α → 0, where only reward matters.

MBPO​

Model-Based Policy Optimization — a Dyna-style algorithm that trains a dynamics model (usually an ensemble), generates short imagined rollouts branched off real states, and feeds those synthetic transitions into the replay buffer of an off-policy learner like SAC. Its signature insight is that rollout length must stay tiny — often a single step — because a dynamics model's error compounds the further you roll, so short branches stay accurate while still multiplying the dataset. The result is a large sample-efficiency gain over the same learner run model-free. Analogy: a trainee who takes one real swing, then mentally rehearses just the next instant of follow-through — short enough that the imagined version still matches reality.

MCAP​

A high-performance, standardized serialization container file format designed specifically for recording and storing robotics data, such as images, point clouds, coordinate transforms, and sensor readings. Unlike older formats that are tied to specific middleware (such as ROS .db3 bags), MCAP is middleware-neutral, supports zero-copy serialization, and is structured for quick random access and index-based seeking without reading the entire file. Analogy: A digital flight data recorder (black box) for a robot; instead of dumping raw text logs or unstructured binary heaps that are slow to load and search, MCAP organizes high-frequency visual, physical, and temporal data into indexed channels so developers can instantly query any sensor at any timestamp. It has become the standard storage format for visualization suites like Foxglove.

MCTS​

Monte Carlo Tree Search — a planning algorithm that decides the next move by growing a search tree of possible futures, spending its effort on the most promising branches. Each round runs four steps: select a path down the tree by balancing known-good moves against under-explored ones, expand a new node, evaluate it (by a rollout or a learned value), and back up that result to update every node on the path. After many rounds, the move that was searched the most becomes the choice. Analogy: planning a chess move by mentally playing out the few most interesting lines deeply, rather than every legal line shallowly. It is the search engine inside AlphaZero and MuZero.

MDP​

Markov Decision Process — the standard mathematical description of a decision-making problem, written as the tuple (S, A, P, R, γ): the set of States the world can be in, the Actions the agent can take, the transition probabilities P saying where each action is likely to land you, the Reward function scoring what happens, and the discount factor γ weighing future reward against present. "Decision Process" because the agent makes a sequence of choices over time; "Markov" because the next state depends only on the current state and action, not on the full history of how you got there. Nearly every RL method assumes the problem is (or can be treated as) an MDP. When the agent cannot fully observe the state, it becomes a POMDP.

Mechanistic interpretability​

The line of research that tries to reverse-engineer what individual pieces of a neural network actually do — which neurons or attention heads detect what, where a fact is stored, why a particular output came out. Like opening up a watch to see which gears turn the hands, instead of only timing how fast the watch runs. Main tools: linear probes, sparse autoencoders, activation patching, and circuit analysis.

Media container​

The file format that wraps compressed video (plus audio, subtitles, and metadata) into one file — .mp4, .mov, .webm, and .mkv are containers. The container is the box; the video codec is how the picture inside was compressed, and the two are independent — the same H.264 video can sit in an .mp4 or a .mov. Analogy: a container is like a shipping box labeled on the outside, while the codec is the packing method used for the fragile thing inside. Example: a .webm file is a container that usually holds AV1- or VP9-compressed video, whereas .mp4 most often holds H.264.

Medical-image segmentation​

The task of labelling every pixel in a medical scan (MRI, CT, X-ray, microscopy) as belonging to a particular structure — outlining a tumour, an organ, or a cell boundary — rather than just classifying the whole image with one label. The output is a per-pixel mask, like a precise coloring-book page where each region is filled with its own color. It demands very fine spatial accuracy, since a few pixels can be the difference between the edge of a tumour and healthy tissue, which is exactly why the U-Net's skip connections — carrying fine detail straight across the network — were originally designed for it. Think of tracing the exact outline of each country on a map instead of just saying "this is a map of Europe."

Mel bands​

The output channels of a mel filterbank — the handful of frequency buckets (commonly 80) that a mel spectrogram keeps after squeezing the STFT's hundreds of fine frequency rows onto the perceptual mel scale. Low-pitch bands are narrow and closely spaced while high-pitch bands are wide, mirroring how human hearing tells low notes apart easily but lumps high ones together. Like sorting a piano's 88 keys into a few labeled bins where each bass key gets its own bin but many treble keys share one. Example: an 80-band mel spectrogram describes each moment of sound with 80 numbers instead of 500+ raw frequency values, small enough for a CNN or transformer to process like an image.

Mel spectrogram​

A picture of sound: a 2D map with time along one axis and pitch along the other, where brightness shows how much of each pitch is present at each moment. It is built by sliding a short window across the audio waveform and measuring its frequencies (a Short-Time Fourier Transform), then squashing the frequency axis onto the mel scale — a perceptual spacing that, like human hearing, gives lots of resolution to low pitches and lumps high ones together (the jump from 100 to 200 Hz sounds bigger than the jump from 5,000 to 5,100 Hz). The payoff is that audio becomes an image with, say, 80 frequency rows, so the same CNN or transformer machinery built for vision can process it. Like turning a song into sheet music — a flat diagram you can read at a glance instead of a wiggling waveform.

Meta-learning​

"Learning to learn" — training a model to adapt quickly to new tasks with few examples. Many meta-learning algorithms, such as MAML, rely on higher-order gradients to optimize across tasks.

Memorization​

When an LLM reproduces a chunk of its training data verbatim instead of generalizing from it — give it the right opening prompt and out comes the original passage word for word. Like a student who recites a textbook sentence rather than explaining the idea; useful for trivia, dangerous for copyright, PII, and security. Deduplication at training time and prompt filtering at serving time are the main mitigations.

Memory bandwidth​

The rate at which data can be read from or written to a processor's memory (such as HBM or system RAM) by the processor's cores, typically measured in gigabytes per second (GB/s) or terabytes per second (TB/s). It is the slope of the memory-bound region in the roofline model. Analogy: The width of a highway. Even if you have ultra-fast sports cars (fast compute cores), if the highway only has one lane (low memory bandwidth), the cars will get stuck in traffic (the processor will stall waiting for data to arrive from memory). Increasing memory bandwidth is like adding more lanes to the highway.

Memory coalescing​

A hardware optimization on modern GPUs where multiple global memory accesses by threads within a single warp are combined (coalesced) into a single, large memory transaction. When threads in a warp access contiguous (adjacent) memory locations, the GPU can fulfill the entire request in one go, maximizing the achieved memory bandwidth. If threads access scattered or strided memory locations, the hardware must perform multiple separate transactions, drastically reducing performance.

  • Analogy: Imagine a mail carrier delivering packages to an apartment building. If all 32 tenants on a floor order packages, and they are delivered together in one box to the lobby (coalesced), it takes a single trip. But if the packages are delivered one by one to different rooms scattered across different floors (non-coalesced), the mail carrier has to make 32 separate trips, taking much longer.
  • Example: In a simple vector-add kernel, having thread i load input[i] ensures that adjacent threads load adjacent floats. The GPU groups these into a single 128-byte transaction, achieving near-peak bandwidth. If the kernel instead loads input[i * 32] (strided access), each thread's load requires its own separate memory transaction, causing the effective memory bandwidth to collapse.

Memory leak​

An unintended increase in memory usage over time, often caused in PyTorch by holding onto references to the loss function or other parts of the dynamic computation graph across training iterations.

Memory mapping​

Accessing a file on disk as if it were an in-memory array, reading slices on demand without loading the whole file into RAM (e.g. numpy.memmap).

Memory snapshot​

A recording of how much GPU memory is allocated at one moment; comparing snapshots taken across training steps reveals a steadily growing memory leak.

Megatron​

NVIDIA's approach to tensor parallelism that splits attention and MLP layers column-wise and row-wise across GPUs with carefully placed AllReduce collectives, allowing efficient intra-layer parallelism.

MFU​

Model FLOPs Utilization — the fraction of a GPU's peak arithmetic speed a training run actually uses (e.g. 70% MFU). Like a delivery truck's fill rate, it shows how much of the hardware you are paying for is doing useful work instead of waiting on memory or the network.

Micrograd​

A tiny, educational autograd engine implemented in basic Python by Andrej Karpathy to illustrate how reverse-mode differentiation works.

Minimax​

The exact way to play a two-player game where one player's gain is the other's loss: assume your opponent will always make the best reply, and choose the move that leaves you best off under that assumption. Applied recursively to the end of the game, it yields perfect play. It is only feasible when the game is small enough to search exhaustively — Tic-Tac-Toe yes, chess no — which is why MCTS exists for anything larger. A useful side effect: on a small game, minimax gives you an unbeatable opponent to test a learned agent against, which is a far more honest yardstick than "beats a random player". Its sign-flipping formulation ("my gain is your loss, so negate the value at every level") is called negamax and is what makes tree searches like MuZero's work for two players.

MinHash​

A hashing technique for estimating how similar two documents are, used to find and remove near-duplicate text at corpus scale (see deduplication).

Minimum-snap trajectory​

A smooth trajectory designed for quadrotors or other dynamic systems that minimizes the integral of the square of snap (the fourth derivative of position with respect to time: acceleration of acceleration) over the duration of the flight.

  • Why it matters: Drones have limited motor torque and are highly sensitive to sudden changes in force. Minimizing snap ensures the trajectory is extremely smooth, which prevents motor saturations, reduces structural vibration, and allows the drone's attitude control loops to track the path with high precision.
  • How it works: It formulates trajectory generation as a constrained optimization problem. The path is represented as a piecewise polynomial (e.g., 7th-degree polynomials). By solving a quadratic program, it finds the polynomial coefficients that minimize snap while passing through a series of defined waypoints.
  • Analogy: Imagine drawing a path on paper with a pen. A low-snap trajectory is like drawing a single, continuous, sweeping stroke where your hand moves gracefully and slowly changes direction. A high-snap trajectory would be like drawing a jagged zigzag, requiring you to suddenly stop and jerk the pen in new directions, which would shake your hand.
  • Example: A warehouse mobile robot is tasked with patrolling a facility, opening doors, picking up parts, and delivering them to assembly stations. This task takes 1 hour and requires hundreds of coordinated navigation and manipulation actions. Measuring overall mission success rather than simple navigation steps is a long-horizon evaluation.

MLP​

Multi-Layer Perceptron — the simplest kind of neural network (also called a feedforward network): a stack of fully-connected layers with a non-linear activation in between. A fully-connected layer means every input number connects to every output number, each connection carrying its own weight — like a voting panel where every voter influences every result. A non-linear activation (such as ReLU or SwiGLU) is a simple bend applied after each layer; you need one because stacking plain linear layers just collapses back into a single straight line, so the bend is what lets the network learn curved, complicated patterns. In a transformer, the model is a tall stack of identical blocks, and each block has two sublayers in order: an attention sublayer (tokens look at each other) then an MLP sublayer (each token is processed on its own). So going up the stack it really does look like attention → MLP → attention → MLP → … — attention passes notes around the room, the MLP is each person quietly thinking about what they just read.

Attention comes before the MLP because a token should gather context from the others first and only then think for itself — you read the room, then form your own thought. The two are built differently: attention has each token build a weighted blend of every token's values (that blend is how they "look at each other"), while the MLP runs a plain feed-forward on each token's vector alone, with the same weights at every position (that is "on its own"). The bend inside that MLP has grown more capable over time: plain ReLU just clips negatives to 0; a GLU instead multiplies the content by a learned 0-to-1 "gate" so the network can dial parts of it down; and SwiGLU is a GLU whose gate uses the smooth Swish curve — the modern default.

MLX​

An open-source, array-based machine learning framework designed by Apple's silicon research team specifically for Apple Silicon. It is built to feel familiar to users of NumPy and PyTorch, but is optimized from the ground up to run efficiently on Apple's unified memory architecture. By allowing the CPU and GPU to operate on the same memory tensors without copying them, MLX makes local training and inference of large models fast and easy.

  • Analogy: Imagine a special language spoken natively in a kitchen. PyTorch is like a global language that has to be translated, whereas MLX is a local dialect that the local staff (Apple Silicon CPU/GPU/Neural Engine) understand perfectly without any translation or coordination overhead, making the kitchen run at absolute peak speed.
  • Example: Writing a simple Python script using MLX to load a LoRA adapter and run fine-tuning on a local dataset, where MLX automatically targets the GPU and CPU cooperatively without duplicate memory allocations.

MMDiT​

Multi-Modal Diffusion Transformer — the DiT variant used in SD3 and Flux where text tokens and image tokens flow through the same attention layers ("joint attention") instead of having the image attend to the text through a separate cross-attention step. Each modality (text vs image) keeps its own normalization and MLP weights, but they see and influence each other inside one shared attention operation, which helps the model get compositional prompts ("a red cube on a blue sphere") right. Like seating writers and illustrators at one table where everyone hears the whole conversation, instead of passing notes between two separate rooms.

MMBench​

A multiple-choice benchmark for VLMs that probes many separate abilities — object recognition, spatial relationships, attribute comparison, and more — with each question offering a few labeled answer choices. To stop a model from scoring well by luck or by always favoring one letter, it asks the same question several times with the choices shuffled and counts it correct only if the model picks the right answer every time (a trick its authors call CircularEval). Like re-asking a quiz question with the options reordered to be sure the student actually knows the answer rather than having memorized "it's always C." It is one of the standard general-capability scores reported for any new VLM.

MMLU​

Massive Multitask Language Understanding — a 57-subject multiple-choice benchmark (history, law, medicine, math, and more) that became the standard quick test of how much general knowledge a model has, like a giant trivia exam spanning many school subjects at once.

MNIST​

A classic dataset of 70,000 small 28×28 grayscale images of handwritten digits 0–9. It is the most common "hello world" for image models — tiny, clean, and quick to train on — so a brand-new idea is almost always tried on MNIST first, before anyone risks it on harder, fuller-color data like CIFAR-10.

MMMU​

Massive Multi-discipline Multimodal Understanding — a hard benchmark of college-exam questions across many fields (medicine, engineering, art, business), where each question mixes text with an image such as a diagram, chart, or chemical structure. It is built to require real subject reasoning rather than just reading the picture, which is why even strong VLMs still score far below human experts on it. Like a university final that hands you a figure and expects you to apply the course material to it, not merely describe what you see. It is the most-cited measure of frontier multimodal reasoning, and the harder MMMU-Pro variant adds more answer choices and trickier distractors to fight contamination.

MoCoGAN​

MoCoGAN (Motion and Content GAN) is an early video GAN whose key idea is to split a video's latent code into two parts: a single content vector that stays fixed for the whole clip (the identity of the person or object) and a sequence of motion vectors that change frame to frame (how it moves). Because content is held fixed while motion varies, the same face can be made to perform different expressions, or one motion can be replayed on different faces. This separation — disentangling what from how — keeps a subject from morphing as it moves; the generator reads a new motion vector each frame (produced by a small recurrent network) on top of the one shared content vector. The same content/motion split keeps reappearing inside later diffusion-based systems, which is why the 2017 model is still worth studying.

Modality​

One type or format of data — text, images, audio, video, a depth map, and so on. Each modality has its own structure (text is a sequence of tokens, an image is a grid of pixels, audio is a waveform), so a model usually needs a dedicated encoder for each one before their information can be combined. A model that handles more than one is called multimodal. Think of modalities as the different human senses — sight, hearing, touch — each carrying information about the same world but in a different form, which the brain then has to fuse into one understanding. Cross-attention is one common way to let two modalities exchange information.

Modality balancing​

In a single model trained on several modalities at once, the practice of adjusting how much each one contributes to the loss so that no single modality drowns out the others. The problem arises because modalities are rarely equal in size: if 99% of your tokens are text and only 1% are image, the text next-token-prediction loss dominates the gradient and the model barely learns to handle images. It is like a study schedule where, left alone, you would spend every hour on your strongest subject — you have to deliberately reweight so the weaker subjects get their share of attention. Concretely, you either oversample the under-represented modality's data or multiply its loss term by a larger coefficient, tuning until each modality's loss falls at a comparable rate.

Modality gap​

The repeated empirical finding that, even in a model like CLIP trained to put matching items in one shared space, the embeddings of one modality (all the images) sit in a different region from those of another (all the captions) — two separate clusters rather than one blended cloud. The pairs are still correctly aligned (a photo is nearer its own caption than to a wrong one), but a constant offset separates the two modalities, a side effect of how contrastive training and the random initial weights shape the geometry. Analogy: two choirs singing the same song in perfect harmony but standing on opposite sides of the stage — in tune with each other, yet never in the same spot. You can see it by encoding a batch of images and captions, reducing them with PCA, and watching the two colors land in separate blobs; it matters because it lowers the cosine-similarity scores of true pairs and can be partly fixed by shifting one modality's vectors toward the other.

Mode collapse​

A GAN failure where the generator discovers a few outputs that reliably fool the discriminator and just keeps making those, ignoring the variety in the real data — like a comedian who finds one joke that always lands and tells only that joke. Each sample may look fine on its own, but the model has stopped covering most of the data. It is the defining instability of GAN training; its discrete-latent cousin is codebook collapse.

Model-based RL​

A family of reinforcement-learning methods that learn an explicit model of the environment — how states change and what rewards follow — and then use that model to plan, to imagine extra training data, or both. Its big advantage is sample efficiency: once you can simulate the world, you can learn a lot without paying for real interaction, which matters when real steps are slow or expensive (a physical robot, a medical trial). Its cost is model error — the policy is only as good as the model's predictions, which degrade over long horizons. Named examples include PETS, MBPO, DreamerV3, MuZero, and TD-MPC2. Contrast with model-free RL, which skips the model and learns a policy or value directly from experience. Analogy: model-based RL is studying the map and rehearsing the route in your head before driving; model-free is learning the route only by actually driving it many times.

Model-free RL​

Reinforcement learning that never builds an explicit model of the environment's dynamics — it learns a policy or value function directly from sampled experience, treating the world as a black box. This is simpler and tends to reach higher final performance when samples are cheap, but it usually needs far more interaction than model-based RL because every lesson must come from a real step. Named examples: DQN, PPO, and SAC are all model-free. Analogy: learning to ride a bike purely by trial and error on the actual bike, never pausing to reason about the physics — slower to start, but you end up tuned to the real thing.

Model-parallel​

Model parallelism is a technique for training or running massive neural networks by splitting the model's parameters (weights) across multiple processing chips (like GPUs or TPUs) because the entire model is too large to fit into the memory of a single chip.

  • Why it matters: Modern large language models (LLMs) can have tens or hundreds of billions of parameters, requiring hundreds of gigabytes of memory just to load their weights. Since a single GPU or TPU typically only has 16 to 96 GB of memory, it is physically impossible to train or run these models on one chip. Model parallelism solves this by distributing the model across a network of chips.
  • How it works: Model parallelism is typically split into two main approaches:
    1. Tensor Parallelism: Splitting individual math operations (like large matrix multiplications within a single layer) across multiple chips. All chips calculate their part of the layer at the same time and combine their results using collective communication operations (like AllReduce).
    2. Pipeline Parallelism: Splitting the model's layers sequentially across a chain of chips. For example, in a 40-layer model, Chip 1 might handle layers 1–10, Chip 2 handles layers 11–20, and so on. Data is passed from one chip to the next like an assembly line.
  • Analogy: Imagine you want to build a massive Lego castle, but the instruction book is so huge and detailed that it won't fit on your small workspace.
    • Data parallelism is like having four people, each with their own identical copy of the instruction book and their own workspace, building four separate smaller castles at the same time using different Lego bricks.
    • Model parallelism (Pipeline) is like building one giant castle on a conveyor belt. The first person builds the foundation (layers 1–10) and slides it to the second person, who adds the walls (layers 11–20), who then passes it to the third person to add the roof and towers.
    • Model parallelism (Tensor) is like having four people crowd around a single castle piece, each holding a different section of the same page of instructions, putting bricks on the same piece simultaneously.
  • Contrast: Contrast with data parallelism, where the model is small enough to fit on a single chip, so every chip gets a complete copy of the model and is fed a different slice of the training data.

MoE​

Mixture-of-Experts — instead of one big MLP per layer, the model holds many parallel "expert" MLPs and a small router sends each token to only the top few. Like a big company where every question goes to just the two or three relevant specialists rather than the whole staff, the model can hold a huge number of total parameters while doing only a fixed, small amount of compute per token. The serving catch: which experts get used shifts with the workload, so keeping them evenly busy across GPUs (expert parallelism) is the hard part.

Momentum​

A technique that accumulates a moving average of past gradients to dampen oscillations and accelerate gradient descent in consistent directions

Monosemantic​

A feature inside a neural network that fires for exactly one concept — for example, a direction in activation space that lights up only for "Golden Gate Bridge," or only for "negation in a clause." The opposite is polysemantic: one neuron that activates for several unrelated concepts at once. Like a single word that means just one thing versus a homonym that means several. Recovering monosemantic features is the main goal of SAE-based interpretability.

Monte Carlo method​

A way to estimate a value function from complete sampled episodes alone, with no model of the environment (meaning the agent does not know the rules or transition probabilities of the world beforehand and must learn purely by trial and error). To estimate values, the agent plays an episode to the end, computes the actual return that followed each state, and averages those returns over many runs. Having "no model" is like learning to navigate a maze by actually walking through it, bumping into walls, and finding the exit, rather than studying a complete blueprint of the maze before entering.

"Monte Carlo" (after the casino) is the general name for estimating a quantity by random sampling instead of exact calculation. Two variants differ only in bookkeeping: first-visit MC averages the return from the first time a state appears in each episode (keeping the samples independent), while every-visit MC averages from every occurrence. Monte Carlo estimates are unbiased but high-variance — they wait for the true outcome but inherit the randomness of a whole trajectory — the opposite trade-off from temporal-difference learning.

Montezuma's Revenge​

A notoriously hard Atari game used as the benchmark for exploration research. The player must climb ladders, dodge enemies, and collect keys across several rooms before earning a single point, making it an extreme sparse-reward problem on which plain ε-greedy agents score zero essentially forever. It became famous as the game where curiosity-style intrinsic-motivation methods — notably RND — first reached human-level scores. Like a treasure hunt where you must find and fit together pieces scattered across many rooms before the scoreboard even ticks up once.

Multi-armed bandit​

The simplest possible exploration problem: several slot machines ("one-armed bandits"), each paying out at an unknown average rate, and a fixed number of pulls to spend. There are no states and no consequences — every pull is an independent fresh choice — which strips the problem down to the one question at the heart of RL: do I take the machine that has paid best so far, or the one I barely know? Because it is so bare, it is where exploration is actually solved: UCB and Thompson sampling come with proofs that their regret (the reward lost by not always playing the best arm) grows only logarithmically. The trouble in full RL is that actions now change the state, so a wrong choice can strand you somewhere you cannot easily leave — which is why those clean guarantees do not survive the trip to Montezuma's Revenge. Analogy: a row of slot machines and an evening's worth of coins.

Motion module​

The plug-in component at the heart of AnimateDiff: a stack of time-aware (temporal) layers — mostly attention along the time axis — inserted between the blocks of a frozen image U-Net. The frozen image model still produces each frame's appearance; the motion module's only job is to look across frames and nudge them so the sequence moves smoothly instead of flickering independently. Think of it as a "motion adapter" you clip onto a still-image model — trained once on video, then reused unchanged across many image checkpoints.

Motion score​

A single number handed to a video model that says how much motion a clip should contain — low for a near-still "animated photo", high for vigorous movement. During training it is measured from each real clip (commonly from the average optical-flow magnitude between frames — how far pixels travel), so the model learns to associate the number with an amount of movement; at inference you set it by hand to dial motion up or down. Stable Video Diffusion calls its version the motion bucket id, sorting clips into discrete buckets of increasing motion rather than using a continuous value. It is the simplest control surface for video: one knob that separates how much it moves from what is in it.

MoveIt​

ROS 2 manipulation-planning framework

Moving MNIST​

A simple synthetic video dataset built by taking handwritten digits from MNIST and bouncing two of them around inside a black 64×64 frame, where they drift in straight lines and ricochet off the edges. The motion is perfectly predictable (constant velocity plus bounces), but the digits overlap and pass in front of each other, which is just hard enough to test a future frame prediction model without the cost and decoding pain of real video. It became the standard first benchmark for video-prediction models such as the ConvLSTM.

MPC​

Model Predictive Control — a control strategy that, at every step, looks a short distance into the future, picks the best plan over that window, executes only the first action, then throws the rest away and re-plans from scratch next step with updated information. The "predictive" part uses a dynamics model to simulate where candidate action sequences would lead; "finite-horizon" means it plans only a few steps ahead, not to the end of the task; and re-planning every step (the "receding horizon") keeps it robust to model error and surprises. Common ways to find the best action sequence split into two families: sampling-based search that tries many random sequences and keeps the best ones — random shooting, the Cross-Entropy Method, and MPPI — and optimization-based solvers that write the look-ahead as a constrained math problem and hand it to a numerical optimizer (often built with a tool like CasADi), which is the dominant approach in classical robot control. Analogy: driving in fog — you plan only as far as your headlights reach, commit to the next moment, then re-plan as more road appears.

MPPI​

Model Predictive Path Integral control — a planning method closely related to the Cross-Entropy Method that, instead of keeping only a hard "elite" set, updates its action distribution as a reward-weighted average of all sampled action sequences: better-scoring sequences pull the next mean toward them in proportion to an exponential of their return. This soft weighting makes it smooth and well-suited to noisy continuous-control problems. Analogy: instead of picking the few warmest thermometers and ignoring the rest, you let every thermometer tug the search, with the warm ones tugging hardest. It is one of the standard samplers used inside model predictive control.

MPS​

Metal Performance Shaders — the GPU backend for Apple Silicon

MQA​

Multi-Query Attention — all query heads share a single key/value head; the most aggressive KV-cache saver, at some quality cost

MSE (mean squared error)​

The most basic way to score how wrong a prediction is: at each point take the difference between the predicted and true value, square it (so overshoots and undershoots both count as positive, and big misses are punished extra), then average over all points. For images it compares pixel by pixel, so a guess that is a little off everywhere still scores well — which is exactly why training on MSE alone tends to produce blurry results: when the model is unsure, the safest low-MSE answer is to predict the average of all the plausible pixels, and an average of sharp options looks like a smudge. This is the failure a perceptual loss is designed to avoid.

MT-Bench​

A benchmark that scores a chat model's answers to a set of multi-turn questions, often using a strong LLM as the judge; a quick proxy for how helpful an assistant feels.

MuJoCo​

MuJoCo (Multi-Joint dynamics with Contact) is a fast, highly accurate, and open-source physics engine designed for simulating articulated bodies in contact with their environment. It is widely used as a standard sandbox in robotics, biomechanics, and reinforcement learning to train agents in physical tasks like walking, running, and manipulation before moving them to physical robots. Analogy: Think of MuJoCo like a highly realistic video game engine (like Unreal Engine or Unity), but instead of focusing on beautiful graphics or explosions, it focuses entirely on the math of physical joints, friction, gravity, and collisions. It simulates exactly how a real leg would pivot, slide, or push against a floor. Example: When training a robotic hand to rotate a cube or a virtual humanoid to run, researchers build a 3D model of the joints and surfaces in MuJoCo. The reinforcement learning agent receives raw joint angles and velocity readings, controls virtual motor torques, and is simulated at hundreds of frames per second to learn how to balance and move.

Multi-head attention​

Running several attention operations (heads) in parallel, each with its own learned projections of queries, keys, and values, then concatenating their results. Like having several readers skim the same sentence for different things — one tracks the grammar, another tracks who-did-what — and then pooling what each one noticed.

Multi-LoRA​

Serving many LoRA adapters from one shared copy of the base model at the same time. Keeping LoRA's sticky-note picture: one cookbook (the base model) plus a drawer full of sticky-note sets (the adapters), one set per customer. The kitchen keeps a single cookbook and just grabs the right set of notes for each order, instead of buying a whole new cookbook for every customer — so one GPU can serve hundreds of fine-tunes at once.

Multi-tenant​

One shared system serving many independent users or customers ("tenants") at the same time, who must not see or slow down one another — like an apartment building where many families live under one roof but each behind their own locked door. A multi-tenant inference service mixes everyone's requests onto the same GPUs, which is why fair scheduling, per-user rate limits, and tricks like shared-prefix cache routing matter so much.

Multi-turn conversation​

A chat where the user and the AI take turns talking back and forth, building on what was said earlier, like a natural human conversation. For example, if you ask "What's a good movie?" and then ask "Who stars in it?", the AI remembers the movie from the first turn. Instead of starting from scratch every time, the system keeps the past conversation in its KV cache — like keeping an open notebook on your desk instead of erasing the whiteboard after every question.

Multimodal distribution​

A probability distribution that has multiple distinct peaks (modes), rather than a single peak. In robotics, action distributions are often multimodal—for instance, when a robot arm must bypass an obstacle, it can choose to go around it from the left or from the right, with both paths being equally valid but any average trajectory leading directly into the obstacle.

  • Why it matters: Standard behavior cloning policies using MLPs predict actions by outputting the mean of a Gaussian distribution (a single peak). When trained on multimodal human demonstrations, these models average the demonstrations, producing failed, "in-between" actions. Representing action distributions as multimodal is essential for complex manipulation, which is why diffusion policies (which represent action distributions as a denoising process) are so successful.
  • Analogy: Imagine coming to a fork in the road while walking to a park. You can go left or right. A multimodal plan naturally keeps both paths as distinct options. A single-peak Gaussian plan averages the two paths and commands you to walk straight ahead, crashing directly into the divider.
  • Example: In a grasping task, if a robot is presented with a cup that can be grabbed by the handle or the rim, a policy that models a multimodal distribution will commit to grasping one or the other, whereas a unimodal Gaussian policy might command the gripper to close on the empty space in between.

Mutual information​

A measure from information theory of how much knowing one variable tells you about another — zero when they are independent, large when one strongly predicts the other. Concretely it is the amount your uncertainty about variable X drops once you are told variable Y. In RL it shows up in skill discovery: DIAYN maximizes the mutual information between a skill code and the states that skill visits, which forces different skills to visit visibly different parts of the world. Like how knowing the season tells you a lot about the weather (high mutual information), whereas knowing a stranger's shoe size tells you almost nothing about their favorite color (near zero).

MuZero​

A model-based RL agent (DeepMind) that mastered chess, shogi, Go, and Atari from scratch by combining Monte Carlo Tree Search with a learned model that predicts only the quantities the search needs — reward, value, and policy — and never reconstructs the actual observation. Because it does not have to model irrelevant visual detail, only what affects decisions, the same algorithm works on board positions and on game pixels alike. It has three coupled networks: representation (encode the observation into an abstract state), dynamics (predict the next abstract state and reward from an action), and prediction (output a value and a policy for the search). It extends AlphaZero, which needed the true rules; MuZero learns its own internal rules instead. Analogy: a player who can plan ahead in their head without redrawing the whole board each move — they track only the few features that change the outcome.

NaN​

"Not a Number" — a floating-point value representing an undefined or unrepresentable result (e.g., 0/0 or inf - inf). In PyTorch, NaNs often appear when gradients explode or when taking the logarithm of zero/negative numbers.

Nat​

The unit entropy and log-probability are measured in — a "natural unit of information", so called because it comes from using the natural logarithm (base e) rather than base 2. If you use base 2 instead, the same quantity is measured in bits, and 1 nat ≈ 1.44 bits. Nothing deep separates them; it is a choice of ruler, like metres versus feet. What matters in practice is the scale: entropy values live in a small range (a few nats either side of zero), which is exactly why they cannot be compared directly against a reward that might be 0.5 or 500 per step — and why SAC needs automatic temperature tuning to convert between the two.

  • Analogy: Measuring distance in kilometres instead of miles. The road does not change; only the number on the sign does. Nats and bits are two rulers laid against the same quantity of "surprise".
  • Example: SAC's usual target is -1 nat per action dimension — "for each joint you control, keep roughly this much randomness in the policy."

Native multimodal​

A model trained from scratch on all modalities at once over a single shared vocabulary, instead of bolting a vision encoder onto a finished language model. Every modality is turned into tokens — text tokens, image tokens from a VQ-VAE, audio tokens from a neural codec — that all live in one alphabet, and one transformer reads and writes any mix of them with a single next-token objective. This is the early-fusion extreme, used by models like Chameleon and GPT-4o. Analogy: rather than hiring separate translators for each language and patching their notes together, you raise one person bilingual from birth, so switching between "languages" (modalities) is effortless and mid-thought. The payoff is true any-to-any flexibility; the cost is far more data and compute, since nothing is reused from a pretrained backbone.

Negative prompt​

A second text prompt describing what you do not want in the image (e.g. "blurry, extra fingers, watermark"). It works through classifier-free guidance: instead of pushing away from a blank unconditional prediction, the model pushes away from the negative prompt's prediction and toward your real prompt — so naming a flaw steers the result away from it. Like telling an artist "paint a beach, and whatever you do, no people."

Network in Network​

A design idea where a tiny neural network is tucked inside a single layer of a bigger one, so that layer can do more thinking than a plain filter could. A normal convolution layer slides a simple filter that just takes a weighted sum of each patch; a network-in-network slides a small multi-step mini-network over each patch instead, letting it recognize more complicated local patterns on the spot. Picture a factory line where, instead of one worker stamping each part, every station hides a little expert team that inspects and shapes the part before passing it on. The idea (from the 2013 Network In Network paper) inspired the Inception network's building blocks — which is why Inception was nicknamed after the movie about a dream inside a dream.

Natural gradient​

A gradient step that measures distance by how much the model's output distribution changes, rather than by how much its parameters change. Concretely it is the ordinary gradient premultiplied by the inverse Fisher information matrix, F⁻¹g.

  • Why it matters: Plain gradient descent implicitly assumes that a fixed-size step in parameter space is equally significant everywhere, and for a neural network that is simply false — the same nudge to two different weights can leave the policy untouched or destroy it. The natural gradient rescales each direction by how much the policy actually moves along it, giving a step that is invariant to how the model happens to be parameterized. It is the theoretical heart of TRPO.
  • Analogy: Walking one kilometre north near the equator and one kilometre north near the pole are the same distance on the ground but very different distances in longitude coordinates. The natural gradient works in kilometres; the ordinary gradient works in degrees of longitude and is baffled by the poles.
  • The catch: F⁻¹ is unaffordable to compute directly, which is why every practical implementation reaches for conjugate gradient instead — and why PPO, which approximates the whole idea with a clipped ratio, won on simplicity.

Neural codec​

A neural network that learns to compress a signal — audio, an image, or video — into a compact code and then rebuild it, a learned cousin of hand-designed formats like MP3 or JPEG. ("Codec" = coder + decoder.) The encoder squeezes the signal down to a small set of numbers or tokens and the decoder reconstructs it; because the whole thing is trained on real data instead of hand-tuned, it can often pack more quality into fewer bits. A VQ-VAE is one example used for images; for audio, EnCodec and SoundStream are the best-known examples, squeezing a waveform into a short stream of discrete tokens at a chosen bitrate (bits per second) — so a lower bitrate means fewer tokens and rougher sound, a higher one means more tokens and cleaner audio.

Null space​

The set of all inputs a matrix maps to zero — for a robot, the joint-velocity directions that produce no end-effector motion at all. Push the joints along a null-space direction and the hand stays frozen while the rest of the arm rearranges itself. This is the mathematical home of kinematic redundancy: a redundant arm's Jacobian has a non-empty null space, and controllers project secondary goals (posture, obstacle avoidance) into it so those goals never fight the primary hand-tracking task. Analogy: holding a steering wheel at "12 o'clock," you can still wiggle your elbows up and down without turning the wheel — those elbow motions live in the null space.

Numerical issues​

Problems arising from the finite precision of floating-point numbers, such as underflow, overflow, or loss of precision, which can lead to unstable training or NaN values.

ROS 2 navigation stack

NCCL​

NVIDIA Collective Communications Library — does AllReduce etc. on NVIDIA GPUs

nDCG​

Normalized Discounted Cumulative Gain — a ranking-quality score from 0 to 1 that rewards putting the most relevant results near the top of the list; the standard way to check whether a reranker actually improved the ordering.

Needle-in-a-haystack​

A long-context test that hides one fact (the "needle") inside a long stretch of irrelevant text (the "haystack") and checks whether the model can find it

NF4​

Short for NormalFloat 4-bit — an information-theoretically optimal quantile quantization data type introduced in QLoRA for compressing neural network weights to 4 bits.

  • Why it matters: Neural network weights typically follow a normal (Gaussian) distribution centered around zero. Standard 4-bit integer formats (int4) distribute their 16 representable levels evenly (uniformly) across the range, which means many levels are wasted on the empty outer edges, while the crowded center has too few levels to resolve differences. NF4 constructs a custom, non-uniform grid of 16 values such that each bin has an equal probability of receiving a weight. This maximizes the information content of the 4 bits and minimizes quantization error.
  • Analogy: Imagine trying to seat a crowd of people in a theater. If you space the chairs completely evenly across the entire building, including the empty lobbies and exits (uniform quantization), the main seating area will be extremely crowded with people sharing chairs. NF4 is like placing almost all the chairs in the main theater room where the people actually gather, and putting only a couple of chairs in the lobby, ensuring everyone gets a comfortable seat.
  • Example: In QLoRA, a pre-trained base model is loaded in NF4 precision to fit on a single GPU, while high-precision LoRA adapters are trained on top of it.

Network-on-chip (NoC)​

A communication system built directly onto a single silicon microchip that connects different processor cores or hardware modules using packet-switched routing, similar to how the internet routes data.

  • Why it matters: In traditional computer chips, all processor cores share a single set of wires (called a bus) to talk to memory or each other. If one core is using the bus, all other cores must wait, creating a massive bottleneck when scaling to hundreds of cores. A Network-on-chip (NoC) solves this by allowing multiple cores to send and receive data packets simultaneously over a grid-like network of tiny routers built on the silicon itself, dramatically increasing bandwidth and parallel performance.
  • How it works: Instead of hardwiring every core to every other core, an NoC groups data into packets (containing the payload and target address). These packets hop from core to core through a grid of micro-routers until they reach their destination. This structure allows chip designers to build grids of hundreds of small, simple cores (like in Tenstorrent chips) that stream data directly to their neighbors.
  • Analogy: Imagine a busy office building with 100 workers.
    • A traditional bus system is like having a single shared telephone line. Only one pair of workers can talk at a time; everyone else must wait for the line to clear.
    • A Network-on-chip is like giving every worker their own desk with a pneumatic tube system. Workers can write notes (data packets), pop them in a tube, and send them to any other desk. Multiple notes fly through the tubes simultaneously, allowing the entire office to collaborate without waiting.
  • Example: Accelerators like Tenstorrent's Wormhole chip use an NoC to link a grid of RISC-V cores. Instead of sending intermediate outputs back to slow system memory, a core computes its results and shoots them directly across the NoC to the next core in the model's pipeline.

Network topology​

The physical or logical layout of cabling, switches, and connections that link nodes and processors (like GPUs) together in a cluster. Standard topologies for AI networks include fat-tree (a hierarchical tree structure where bandwidth increases closer to the root to prevent congestion), dragonfly (a dense direct-connect scheme that minimizes switch hops), and rail-optimized (grouping matching GPU ranks on different nodes into dedicated physical networking rails).

Analogy: A city's road network. A simple grid layout lets you drive anywhere with a similar number of turns, but is prone to traffic jams. A highway hub-and-spoke layout (like a fat-tree) lets traffic zoom between distant parts of the city quickly, but can experience bottlenecks at major interchanges if too many cars travel through the same hub.

Example: When training a model across multiple nodes, knowing the network topology allows the NCCL library to route collective operations (like AllReduce) along the fastest hardware paths, avoiding crossing slow node-to-node links when faster local paths (like NVLink) are available.

Next-token prediction​

The training objective of an LLM: given the tokens so far, predict the next one, scored with cross-entropy loss.

N-gram​

A run of n tokens (or words) sitting next to each other. Here a gram just means one item — one word or token (the word comes from Greek gramma, "something written") — and the n says how many of them in a row, so "the cat sat" is a 3-gram (three words in a row) and "cat" on its own is a 1-gram. By matching the most recent few tokens against earlier text, you can often guess what comes next from what followed the same phrase before, which is exactly how prompt-lookup speculative decoding builds its drafts for free.

n-step returns​

A middle ground between the one-step TD target and the full Monte Carlo return: instead of bootstrapping after a single step (r + γ·maxₐ Q(s′, a)), you sum the actual rewards over the next n steps and only then bootstrap from the value at step n (r₀ + γr₁ + … + γⁿ⁻¹rₙ₋₁ + γⁿ·maxₐ Q(sₙ, a)). Larger n leans on real observed reward (lower bias) but mixes in more of the environment's randomness (higher variance); small n is the reverse. The practical payoff is faster credit assignment — when reward finally arrives, it propagates back n states in a single update instead of trickling one step per update. It is one of the ingredients combined in Rainbow.

nn.Module​

PyTorch's base class for all neural network components; acts as a registry that automatically tracks sub-modules, parameters, and buffers assigned in __init__

Node (distributed)​

One physical machine (server) in a distributed job, usually holding several GPUs; multi-node training spreads work across several of them over a network.

Noise schedule​

The recipe a diffusion model follows for how much noise to add at each step of its forward (noising) process — and therefore how much the denoiser must remove at each reverse step. A linear schedule raises the noise level by equal amounts every step; a cosine schedule ramps up gently at the start and end, keeping recognizable image structure alive for more of the process, which usually trains better. Think of it as a dimmer switch for how quickly a picture fades to static: turn it down too fast (linear) and most steps see only static, leaving little to learn from. The choice mainly affects training quality and how many sampling steps you need, not the model architecture.

non_blocking​

The non_blocking=True flag on .to() / .cuda() that lets a host→device copy run asynchronously from pinned memory

Nonholonomic constraint​

A restriction on the motion of a system (such as a robot or a car) that limits its velocity (which direction it can move at any given instant) but does not limit its eventual position (where it can go in the long run). Concretely, it means the system cannot move sideways instantly, but it can still reach any position in space by performing maneuvers.

  • Why it matters: In robotics, knowing if a system has nonholonomic constraints determines what path planning algorithms are needed. A holonomic robot (like a drone or an omnidirectional robot with special wheels) can drift in any direction instantly. A nonholonomic robot (like a car or a Segway) is harder to control because it cannot slide sideways; it must steer and drive along curved paths to maneuver.
  • Analogy: Imagine pushing a shopping cart vs driving a car. A shopping cart can slide sideways, turn in place, and move in any direction instantly (it is holonomic). A car cannot slide sideways (unless it drifts on ice). It is constrained by its wheels: at any instant, it can only roll forward or backward in the direction the wheels are pointing. This is a nonholonomic constraint.
  • Example: A car trying to parallel park in a tight space. Because the car cannot slide sideways, it cannot simply glide into the spot. Instead, it must drive forward, steer, reverse, and execute a series of maneuvers (like a three-point turn) to slide sideways into the spot.

Noisy nets​

An exploration method that replaces epsilon-greedy's external coin flip with noise inside the network: each weight gets a learned mean and a learned noise scale, and a fresh sample of that noise is drawn each time the agent acts. Two things follow. The exploration becomes state-dependent — the network can learn to be decisive where it is confident and erratic where it is not, instead of being uniformly random everywhere the way epsilon-greedy is. And the noise scale is learned by gradient descent, so the agent anneals its own exploration rather than following a schedule you guessed in advance. Like a musician who improvises wildly in passages they are unsure of and plays the rehearsed parts exactly. It is one of the six components of Rainbow.

Noisy-TV problem​

The classic failure mode of prediction-error exploration: if the world contains a source of pure randomness — the canonical example is a television screen showing fresh static every step — a curiosity agent that rewards itself for unpredictability will park in front of it forever. Because the static is genuinely unpredictable, the agent's model never stops being surprised, so the intrinsic reward stays high permanently and the agent mistakes noise for endless novelty. It draws the crucial line between novelty (something new the agent can actually learn) and stochasticity (randomness it can never learn), and it is why methods like the ICM measure surprise in a learned, controllable feature space instead of over raw pixels. Like a gambler hypnotized by a slot machine's flashing lights, mistaking pure randomness for something meaningful to discover.

Normalization​

Rescaling a layer's outputs so they keep a consistent size — typically zero mean and unit variance (LayerNorm) or unit root-mean-square (RMSNorm). Like adjusting every photo to the same brightness before comparing them, it stops numbers from ballooning or vanishing as they flow through a deep network, which is what keeps training stable.

Normalized score​

The convention D4RL uses to make returns readable across different tasks. A raw return of 1,800 tells you nothing on its own — is that good? — because HalfCheetah's rewards and a robot arm's rewards are on completely unrelated scales. So each task fixes two reference points, a purely random policy and a well-trained expert one, and rescales every result onto a ruler between them:

normalized score = 100 × (your return − random return) / (expert return − random return)

0 means "no better than flailing at the controls" and 100 means "as good as the expert that recorded the data." Like grading an exam on a curve where 0 is the score of someone who guessed at random and 100 is the class star's — the raw point total stops mattering and you can compare across subjects. A score above 100 is not a bug: an offline learner that succeeds at stitching can outperform the very behavior policy that produced its training data.

Normalizing flow​

A generative model that starts from simple random noise (usually a plain Gaussian "bell curve") and pushes it through a chain of reversible steps to reshape it into realistic data — like kneading a smooth ball of dough into a detailed shape, where you can always un-knead it back. Why can you always un-knead it? Because every step is deliberately built to be undoable: it only ever stretches, shifts, or folds the dough in a way that has an exact opposite, and it never merges two blobs into one or throws any dough away. For example, if a step's rule is "double this number and add 3," its reverse is simply "subtract 3, then halve" — feed the output back through and you recover the original number exactly, with nothing lost. (An ordinary neural network is not like this: it mashes information together — like flattening the dough — so there is no way to run it backwards.) Because every step can be run backwards exactly, a flow can also report the precise probability of any data point, which most generative models cannot do. The price for that exactness is that each step must stay reversible, which heavily constrains the architecture; examples include Real NVP and Glow.

np.linalg.solve​

A NumPy function that solves a system of linear equations Ax = b for the unknown vector x, given a square matrix A and a known vector b. It is the practical alternative to forming the matrix inverse and multiplying by it: rather than first computing A⁻¹ and then A⁻¹b (two costly steps), it finds x in a single pass, which is both faster and less prone to rounding error. Analogy: to undo "multiply by 3" you don't first write out "1 ÷ 3" and then multiply — you just divide by 3 directly; np.linalg.solve divides by a whole matrix at once. In RL it gives the closed-form answer to policy evaluation in one line — V = np.linalg.solve(I − γPπ, rπ) — the exact value function for a fixed policy without iterating the Bellman equation.

Nsight Compute​

A kernel-level profiling tool from NVIDIA (often run via the command-line command ncu) that provides detailed performance metrics for individual CUDA kernels. It measures register usage, shared memory utilization, cache hit rates, memory access patterns, and whether execution is bottlenecked by arithmetic throughput or memory bandwidth.

Analogy: A microscope for a watchmaker. Rather than looking at the clock's overall timekeeping, it zooms in on the individual gears and springs to see exactly which tooth is slipping or where friction is slowing down the movement.

Example: Profiling a custom matrix multiplication kernel with ncu reveals that the Tensor Cores are active only 20% of the time because non-coalesced memory loads are saturating the HBM bandwidth.

Nsight Systems​

A system-wide profiling tool from NVIDIA (often run via the command-line command nsys) that visualizes an application's CPU and GPU activity over a timeline. It helps identify coarse-grained bottlenecks, such as slow data transfers over PCIe, serialization issues, or gaps where the GPU is idle waiting for CPU dispatch.

Analogy: A Gantt chart for a construction site. It shows when the concrete trucks arrive, when the workers are building, and when work stops because they are waiting on materials, helping the foreman see the overall flow of the project.

Example: Profiling a PyTorch training step with nsys profile python train.py produces a timeline trace showing that the GPU is idle for 30% of the step because of slow CPU-side data preprocessing.

NumPy​

A foundational Python library for scientific computing, providing support for large, multi-dimensional arrays and matrices, along with a collection of high-level mathematical functions to operate on them. It executes operations in optimized C code, making it much faster than standard Python loops. Unlike PyTorch, NumPy arrays run only on the CPU and do not support automatic differentiation (autograd). Analogy: A highly efficient accountant who works solely with spreadsheets. The accountant is incredibly fast at doing math on whole columns and rows of numbers without having to type out every single equation, but works only at their office desk (the CPU) and doesn't record their steps to automatically undo or trace them backward (no autograd).

NVAE​

Short for Nouveau VAE — a hierarchical VAE from NVIDIA (2020) that stacks many layers of latent variables through a deep network built from depthwise separable convolutions and residual connections, reaching then state-of-the-art image generation quality. Like a skyscraper where each floor refines the blueprint handed down from above — the top floors sketch the overall shape and the lower floors fill in the fine details. The name "Nouveau" is French for "new," positioning it as a modern reimagining of the classic VAE.

nvidia-smi​

NVIDIA System Management Interface — a command-line utility that provides monitoring and management capabilities for NVIDIA GPUs. It reports real-time metrics such as GPU utilization, temperature, power draw, memory usage, compute capability, active processes, and NVLink topology.

Analogy: A car's dashboard. It displays real-time speed, engine temperature, fuel level, and warning lights, letting the driver know if the vehicle is running smoothly or hitting its limits without needing to look under the hood.

Example: Running nvidia-smi in the terminal tells a developer if their model's training loop is hitting a memory bottleneck (GPU memory usage is at 99%) or if the GPU is running hot and throttling its clock speed.

NVIDIA's GPU-GPU interconnect; much faster than PCIe

NVL72​

An NVIDIA rack-scale liquid-cooled hardware design that connects 72 Blackwell GPUs (and 36 Grace CPUs) in a single massive NVLink domain using copper cabling and NVSwitch chips. To the software, this cluster of 72 physical GPUs behaves as if it were one giant virtual GPU with a single shared pool of memory and up to 130 TB/s of aggregate NVLink bandwidth.

Analogy: Instead of having 72 separate computers in a room talking over standard networking (like 72 remote workers emailing each other), NVL72 is like putting all 72 workers in the exact same conference room at a single table, allowing them to exchange information instantly without any email or network delay.

Example: Large language models with hundreds of billions of parameters can be sharded across the entire NVL72 rack using tensor parallelism, allowing high-throughput inference without ever hitting the slower PCIe or InfiniBand network bottlenecks between racks.

NVSwitch​

NVLink switch chip; full-bandwidth all-to-all within a node

Observability​

The practice of making a running system's inner state visible from the outside — through metrics, logs, and traces — so you can ask new questions about why it is misbehaving without adding new code. Like the dashboard and warning lights in a car: you can tell what is wrong while still driving, instead of pulling the engine apart. For a serving stack it is the difference between knowing "p99 latency tripled at 9 a.m." and finding out only when users complain.

Object permanence​

The basic fact that objects keep existing even when you cannot see them — a ball that rolls behind a couch is still there and should reappear on the other side. For a video generator this is a surprisingly hard test: a model that only makes each frame look locally plausible may let an object silently vanish, change color, or duplicate while it is briefly hidden (occluded) and then revealed. Infants learn object permanence in their first year; generative video models still routinely fail it, which is why it is one of the world-behavior criteria Sora's report singles out. It is one facet of the broader physical plausibility problem and closely tied to world consistency.

Observation normalization​

Rescaling each component of an environment's observation by a running estimate of its mean and standard deviation, so the numbers reaching the network are all roughly zero-mean and unit-scale. It is detail #6 of the nine continuous-control PPO implementation details, and on MuJoCo tasks it is frequently the difference between an agent that learns and one that does not.

  • Why it matters: A robot's observation vector mixes quantities in wildly different units — a joint angle in radians (order 1), an angular velocity (order 10), a contact force (order 100). A single set of network weights and one learning rate must serve all of them, and the large-magnitude components dominate the early gradients simply by being numerically bigger, not by being more informative. Normalizing puts every input on equal footing.
  • The subtlety: The statistics are estimated from data as it arrives, which means the same observation maps to a different network input early and late in training — a moving target the network must track. It is also a common source of silent bugs at evaluation time, where the running statistics from training must be reused and frozen rather than re-estimated from a handful of fresh episodes.

Occupancy​

The ratio of active warps on a Streaming Multiprocessor (SM) to the maximum number of warps the SM is physically capable of supporting at once. High occupancy allows the GPU's warp scheduler to hide memory latency by switching to another active warp when the current one stalls on data retrieval.

Analogy: A busy call center. If you have only one operator (low occupancy) and they get put on hold (memory latency), the whole desk sits idle. If you have many operators (high occupancy), as soon as one is put on hold, another operator can immediately handle a different call, keeping the center productive.

Example: If a CUDA block uses too many registers or too much shared memory, the SM cannot fit many blocks at once, resulting in low occupancy. If a warp stalls waiting for HBM and there are no other active warps ready to run, the SM's compute units go unused, reducing performance.

OCR (Optical Character Recognition)​

Reading the text inside an image — turning pixels of letters into actual characters a computer can use — for example pulling the line items off a photographed receipt or the words out of a scanned page. It is the skill that separates a VLM that "sees a document" from one that can answer "what is the total?", and it is hard precisely because the answer often hides in small print that survives only if the image is fed in at high enough resolution (one reason AnyRes tiling helps). Analogy: the difference between glancing at a street sign and actually reading the words on it. Example: given a photo of a price tag, an OCR-capable model returns the string "$19.99" rather than just "a label"; benchmarks like DocVQA and OCRBench score exactly this ability.

ODE (Ordinary Differential Equation)​

A mathematical equation describing how a system's current state determines its rate of change (its slope). Rather than giving a fixed value as an answer, solving an ODE yields a full continuous function (a path). In diffusion models, the "Probability Flow ODE" acts as the exact navigation route transitioning pure random noise into a structured image. If the current state is a car's position, the ODE defines its exact velocity at that spot.

Odometry​

Odometry is the use of data from motion sensors to estimate change in position over time. In robotics, it is the most basic form of tracking: estimating how far a robot has traveled and in what direction by counting the rotations of its wheels (wheel odometry), tracking visual features in a video stream (visual odometry), or integrating inertial measurements from an IMU. Because odometry only measures relative motion from one step to the next, small errors accumulate relentlessly, leading to drift over time.

Analogy: Walking down a dark hallway while counting your steps. If each of your steps is roughly 2 feet, you can estimate that after 50 steps you have traveled 100 feet. However, if you slip slightly on a rug or your stride is slightly uneven, your count will be off. By the time you reach step 50, you might actually be at 95 feet or 105 feet, and you won't know it unless you open your eyes (a landmark measurement) to correct your position.

Off-policy​

An RL algorithm where the data used for learning comes from a different policy than the one being improved. The agent learns the value of the optimal policy while actually following a more exploratory policy (like ε-greedy) to gather data. Q-learning and DQN are off-policy methods. Unlike on-policy methods, off-policy methods can learn from old experience gathered by past versions of the agent or even by a completely different agent. Think of learning to play a game by watching someone else play: you figure out the best moves by observing their mistakes and successes, without having to make all those moves yourself.

Offline RL​

Reinforcement learning from a fixed dataset of past experience, with no ability to collect more — also called batch RL. This matches most real applications, where exploring live is too costly or unsafe: medical records, recommender-system logs, robot-fleet recordings, historical trading data. The core difficulty is distribution shift — the moment the learned policy wants an action the data's behavior policy never tried, the Q-function is queried out-of-distribution and can hallucinate huge values, so naive off-policy methods diverge. Fixes fall into two camps: keep the policy close to the data (policy constraint, e.g. BCQ, BEAR, AWAC) or be pessimistic about unseen actions (CQL, IQL); the Decision Transformer instead reframes the whole problem as sequence prediction. Analogy: learning to cook only from a stack of old recipe cards, never able to taste-test a new dish — you must squeeze the most out of what is written down.

Offset​

The starting index into the underlying storage where a tensor's data begins (.storage_offset())

OMPL​

Open Motion Planning Library — sampling-based planners

Online softmax​

An incremental method for computing softmax that maintains running maximum and sum statistics, enabling single-pass computation over tiled inputs without materializing the full exponent sum beforehand.

ONNX​

Open Neural Network Exchange — a framework-neutral file format that stores a model as a graph of operations, so it can run outside the framework that trained it.

ONNX Runtime​

A fast, cross-platform engine that runs models saved in the ONNX format, without needing the original framework like PyTorch.

On-policy​

An RL algorithm that learns exclusively from data gathered by the exact same policy it is currently trying to improve. If the agent explores by acting randomly 10% of the time, the policy it learns will account for that 10% randomness. PPO, REINFORCE, and SARSA are on-policy methods. Unlike off-policy methods, on-policy methods learn only from the policy currently being executed — they cannot reuse old experience. Think of learning to play a game entirely through your own trial and error: you can only learn from the moves you are making right now, not by watching old recordings of yourself or others.

Open-ended​

A task where many different answers can all be reasonable and there is no single right one to check against — writing a poem, summarizing an article, replying helpfully in a chat. The opposite of a closed-ended task like a multiple-choice question (one correct letter) or arithmetic (one correct number). Like grading a creative-writing assignment versus grading a true/false quiz: with the quiz you just count matches, but with the essay you need a human reader — or an LLM-as-judge — to weigh quality, which is why evaluating open-ended work is the hard part of LLM evals.

Open model​

A model whose weights you can download and run yourself — Meta's Llama, Mistral, Qwen, DeepSeek — as opposed to a closed model like GPT-4 or Claude where the weights stay on the provider's servers and you can only call them through an API. Like the difference between buying a recipe book (you have the actual instructions, can modify them, can bake offline) and ordering at a restaurant (you only see the finished dish). Open models are essential for any white-box research that needs the model's internals: methods like GCG optimize against the model's own gradients, and interpretability tools like SAEs read its hidden activations — neither is possible through a closed API.

Open-vocabulary perception​

Detecting or segmenting objects named by free-form language at runtime, instead of from a fixed list of categories the model was trained on. A traditional detector can only find the (say) 80 classes it was trained for; an open-vocabulary system can be asked for "the red cup" or "the thing you stir coffee with" and locate it, because it leans on a vision-language model like CLIP that has learned to match images with arbitrary text. This is what lets a robot act on spoken requests about objects it was never specifically taught. Analogy: the difference between a vending machine with a fixed set of labeled buttons and a clerk you can describe anything to in plain words.

OpenSora​

An open-source, fully documented re-implementation of OpenAI's Sora recipe, built by HPC-AI Tech (a closely related project, Open-Sora-Plan, comes from Peking University). It is a DiT that denoises 3D VAE latents and is trained with flow matching, with the code, model weights, and data pipeline all released publicly. Because nothing is hidden behind an API, it is the standard way to study a complete Sora-style pipeline hands-on — run it, swap a component such as the VAE, and retrain. Think of it as a working blueprint of a frontier video model that you are free to open up and rewire, where the closed originals only publish a sketch.

Open X-Embodiment​

Open X-Embodiment is a large-source collaborative dataset and suite of models designed to advance cross-embodiment robotic learning. It pools together robot trajectory data from over 20 institutions, containing more than 1 million trajectories across 22 different robot platforms (such as arms, quadrupeds, and mobile manipulators) performing a wide variety of tasks.

  • Why it matters: Before Open X-Embodiment, robotic policies were trained in silos on single robots, limiting their generalization. This dataset acts as the "ImageNet of robotics," providing the massive, diverse data needed to train large vision-language-action (VLA) models (like RT-1, RT-2, or RT-X) that generalize across different physical systems.
  • Analogy: Imagine trying to train an AI to translate languages, but you only give it books written by one single person. The AI will learn that person's specific writing style and vocabulary, but will fail in the real world. Open X-Embodiment is like giving the AI a library containing books from thousands of different authors across the globe.
  • Example: A researcher can download a pretrained RT-X model trained on Open X-Embodiment and deploy it directly on their custom robot arm for basic tasks like "pick up the sponge," even if their specific robot was not represented in the original training set.

Optical flow​

A per-pixel map of motion between two frames: for every pixel it gives an arrow saying which direction and how far that bit of the image moved. "Dense" optical flow computes an arrow for every pixel, versus "sparse" flow, which tracks only a few chosen points. It is the rawest form of the "motion signal" in video and shows up everywhere — data filtering, frame interpolation, and motion conditioning. Analogy: imagine laying a sheet of thin see-through paper (tracing paper, the kind you can see a drawing through to copy it) over two snapshots taken a moment apart, then drawing a tiny arrow from where each speck — a tiny spot of detail in the picture — sat in the first frame to where it ended up in the second. Example: between two frames of a car driving right, every pixel on the car gets a rightward arrow while the still background gets near-zero arrows. Common ways to compute it are the classical Farnebäck algorithm and the neural RAFT model.

Optimism in the face of uncertainty​

The guiding principle of principled exploration: when you are unsure how good an option is, assume it is good, and act on that assumption. The assumption is self-correcting, and that is the whole point — either the option really is good (you win) or it is not, and trying it teaches you so, which removes the uncertainty that made it attractive. Either outcome is progress; doing nothing is the only way to stay ignorant. It is the idea behind UCB, Thompson sampling, bootstrapped DQN, and the optimistic initialization of a Q-table. Its mirror image is pessimism, which is correct in offline RL for precisely the reason optimism is correct online: there, a hopeful guess can never be tested, so it can never be corrected. Analogy: a new restaurant with no reviews — the optimist tries it once, and either gains a favourite or loses one evening; the pessimist never learns either way.

Optimizer​

An algorithm that updates a neural network's parameters (like weights and biases) using computed gradients during training to minimize the loss function. In PyTorch, this is implemented as a subclass of torch.optim.Optimizer.

  • Why it matters: Calculating how much each parameter contributed to the model's error (its gradient) is only half the battle. The model must actually adjust those parameters to improve. An optimizer determines how to make those adjustments—balancing speed, stability, and memory usage so the model converges on the best possible settings without getting stuck.
  • How it works: At each training step, the optimizer reads the gradients calculated during the backward pass. It then applies a specific mathematical update rule to adjust each parameter. Basic optimizers just subtract the gradient multiplied by a learning rate; advanced optimizers maintain running statistics (like momentum) to adjust step sizes dynamically for every single parameter.
  • Analogy: Imagine a blindfolded hiker trying to find the absolute lowest point of a foggy valley (the minimum loss). At each step, they can only feel the slope of the ground beneath their feet (the gradient).
    • A simple optimizer is like taking steps of a constant size directly downhill. If the slope is too steep, they might overshoot the bottom and jump back and forth.
    • A smart optimizer (like Adam) is like a hiker who keeps track of their momentum (sliding down long slopes faster) and adjusts their step sizes: taking tiny, cautious steps on crumbly terrain and long strides on steady slopes.
  • Example: The standard optimizer in PyTorch is created using optimizer = torch.optim.Adam(model.parameters(), lr=1e-3). Inside the training loop, calling optimizer.step() updates all of the model's parameters using the Adam optimization algorithm.

Optimizer state​

The extra per-parameter values an optimizer stores between steps — for example, Adam keeps two (the first- and second-moment estimates) — which adds to training memory.

Orbit​

A camera move that circles around a subject while keeping it centered in frame — like walking in a ring around a statue, always looking inward at it. Because the viewpoint travels around the object, you see its different sides in turn, which makes the orbit a demanding test of whether a video model keeps an object's 3D shape consistent as the angle changes. It is one of the paths a model can follow under camera control.

Orthogonal initialization​

Setting a layer's initial weight matrix to a (scaled) orthogonal matrix — one whose rows are mutually perpendicular unit vectors — rather than to independent random numbers. An orthogonal matrix preserves the length of whatever it multiplies, so a signal passing through many such layers neither explodes nor decays away.

  • Why it matters in RL: It is detail #2 of the 37 PPO implementation details, and the gains used are as important as the orthogonality. The policy output layer is initialized with a deliberately tiny gain (0.01), which makes the initial action logits nearly identical and therefore the starting policy nearly uniform — maximum entropy, no accidental early commitment to an action that merely happened to win the initialization lottery. The value head gets a gain of 1.0 and the hidden layers √2 (the standard choice for ReLU-like activations).
  • Analogy: Starting a vote with every candidate on exactly zero, rather than handing one of them a random head start and hoping the electorate notices.

Optimal policy​

The policy (rule for choosing actions) that earns the highest expected return from every state — no other policy does better anywhere. Its value function is written V*, the one the Bellman optimality version of the Bellman equation describes. A key fact: which policy is optimal depends on the discount factor — make the agent more short-sighted or more patient and the best action in a state can change. Solving an MDP means finding this policy.

Ornstein-Uhlenbeck noise​

A way of generating time-correlated random noise, used by DDPG to make a deterministic policy explore. A deterministic actor always outputs the same action for a given state, so exploration has to be added as noise — but plain independent noise at each step tends to cancel out and barely moves the agent anywhere new. Ornstein-Uhlenbeck noise instead drifts smoothly: each step's value is pulled gently back toward zero but nudged by fresh randomness, so it wanders in slow, persistent gusts rather than jittering. On a robot with momentum, those sustained pushes actually carry it into unexplored territory. Analogy: instead of flicking a steering wheel randomly left and right (which keeps you going straight on average), you lean it one way for a while, then drift the other — so you actually wander off the road and see new places. Later work found plain Gaussian noise usually works just as well, so TD3 and SAC mostly dropped it; named after the physicists who first described this drifting random process for the motion of a particle.

Out-of-distribution​

Describes an input that lies outside the range of data a model was trained on, where its predictions are guesses with no evidence behind them — abbreviated OOD. In offline RL the dangerous OOD inputs are actions: the behavior policy only tried some actions in each state, so asking the Q-function to score an action it never saw returns an arbitrary, often wildly inflated number — and because Q-learning's target takes a max over actions, it actively seeks out whichever OOD action got the luckiest overestimate. Analogy: a restaurant critic asked to rate a dish nobody has ever cooked — with nothing to taste, the "review" is pure fiction, yet a naive planner will chase the highest fictional score. Taming OOD actions, by pessimism or by constraint, is the whole job of offline RL.

Outcome reward model​

A scorer that judges only a solution's final answer as right or wrong, ignoring the steps in between — simpler than a process reward model, which grades each step, but blind to where a wrong answer first went off track.

Outlines​

An open-source Python library for constrained generation: you hand it a regular expression, a JSON schema, or a Pydantic model and it patches the LLM's decoder to mask out any next-token choices that would break the structure. Like putting guardrails on a road so the car physically cannot drive off the edge no matter how the driver steers, it makes the model's output structurally valid by construction rather than by hope.

Outpainting​

Inpainting applied to the outside of an image: you place the original on a larger blank canvas, mark the new border area as the region to fill, and let the model extend the scene outward so it continues naturally past the original frame. Like a painter adding more landscape beyond the edges of an existing painting.

Overestimation bias​

The tendency of Q-learning-style methods to predict action-values that are too high, caused by the max in their target. Each Q(s′, a) estimate carries random error; taking the maximum over actions systematically singles out whichever action's error happened to be most positive, so the target is biased upward even when every individual estimate is unbiased on average. Over many updates this optimism compounds and can destabilize learning. Double DQN is the standard fix — it uses one network to pick the action and another to score it, so an inflated estimate is rarely confirmed twice. Analogy: if you always trust whichever of ten noisy thermometers reads highest, you will consistently believe the room is hotter than it really is.

Overfitting​

When a model learns its training examples too literally — memorizing their specific details and noise instead of the general pattern — so it does great on the training set but poorly on anything new. A personalization LoRA trained for too many steps overfits: asked for "the subject on the moon," it just spits back one of its training photos. The classic analogy is a student who memorizes the exact answers to the practice exam and then fails the real test because the questions are worded differently. You spot it when training accuracy keeps improving while held-out performance gets worse, and you fight it with more data, fewer training steps, or regularization. Its opposite — doing well on unseen inputs — is generalization.

Overflow​

A condition in computer arithmetic where the result of a calculation is a number too large for the floating-point or integer format to physically represent, causing the value to become infinity or a special "Not a Number" (NaN) value.

  • Analogy: An odometer on a car that can only show up to 999,999 miles. If you drive one more mile, the odometer cannot display the correct total and either rolls back to 0 or displays an error.
  • Example: In FP16 mixed-precision training, the maximum representable value is 65,504. If a gradient calculation yields a value of 70,000, it overflows, producing NaN values that propagate through the network and ruin training. This is typically managed using dynamic scaling or loss scaling.

Padding​

Filling shorter sequences with a placeholder value so that every sample in a batch has the same length.

Pan​

A camera move where the camera stays in one spot but rotates left or right — like standing still and turning only your head to sweep your gaze across a room. The viewpoint's position never changes, only the direction it faces, so near and far objects slide across the frame together. It is one of the basic moves a video model learns to follow under camera control.

Parameters​

The numbers a model learns during training — its adjustable internal settings. Picture thousands of tiny knobs on a giant mixing board: training nudges each knob a little at a time until the whole board produces good output, and the final knob positions are what the model "knows." They come in two kinds — weights and biases — are stored as tensors, and are adjusted by the optimizer during training. (When people say a "7B model," they mean 7 billion of these knobs.) In PyTorch they are nn.Parameter objects, registered automatically when assigned to an nn.Module.

Partial derivative​

How much a function changes when you nudge just one of its inputs and hold all the others still — the derivative taken one input at a time. If a recipe's tastiness depends on both salt and sugar, the partial derivative with respect to salt tells you the effect of adding a pinch more salt while keeping the sugar fixed. A gradient is simply the full list of these one-at-a-time slopes, one per parameter.

Particle filter​

Particle filter (also known as Monte Carlo Localization or MCL) is an algorithm that estimates the state of a system by representing its probability distribution with a set of discrete, weighted samples called particles. Each particle represents a single concrete hypothesis of what the system's state might be (e.g., a specific coordinate on a map). The filter updates the particles through a three-step process:

  1. Predict: Propagate each particle forward using the system's motion model and add random noise to simulate uncertainty.
  2. Update: Calculate a weight for each particle based on how closely its simulated sensors match the actual sensor readings.
  3. Resample: Draw a new set of particles from the current set, where particles with higher weights are more likely to be copied and particles with low weights are discarded.

This allows the particle filter to represent arbitrary, multi-modal probability distributions (e.g., "I might be in Room A OR Room B").

Analogy: A game of "hot or cold" played by thousands of players in a large, dark castle. Initially, players are scattered randomly in every room (particles). Every time you take a step, all players take the same step. When you report a clue ("I feel a warm breeze"), players near warm vents say "hot" and get higher scores (weights), while players in drafty hallways say "cold" and are eliminated (resampled). Eventually, the surviving players will all gather around the single warm room you are standing in.

PagedAttention​

A way of storing the KV cache for many concurrent requests by splitting each request's cache into small fixed-size "pages" that the engine can scatter freely around GPU memory and look up through a per-request page table — the same idea operating systems use for virtual memory. It removes the wasted space and fragmentation you get when each request needs its own contiguous chunk, which is why vLLM made it the default scheme.

Paged optimizers​

An optimization technique (most notably used during QLoRA training) that prevents out-of-memory errors by using the operating system's paging mechanism to dynamically move optimizer states between GPU memory (VRAM) and CPU memory (system RAM).

  • Analogy: Imagine working at a very small desk (VRAM) where you only have room for one textbook and a notebook. If you need to consult five different books at the same time (optimizer states for a large model), you will run out of space on your desk. A paged optimizer is like having an assistant who stands by the desk: when you need to read a different book, the assistant takes one book off your desk and puts it on a nearby shelf (CPU RAM), and brings you the book you need, keeping your desk from ever getting cluttered.
  • Example: When fine-tuning a 7B parameter model, the optimizer states (like Adam's running averages of gradients) can consume more than 28 GB of memory, which is too large for a standard 24 GB GPU. By using a paged optimizer (such as paged_adamw_32bit from bitsandbytes), PyTorch can offload optimizer states for layers not currently being processed to the CPU host memory, enabling training to complete successfully on a single consumer GPU (like an RTX 3090/4090) at the cost of a small transfer speed penalty.

Patch​

A small rectangular section of an image. Instead of looking at an entire image at once, models often break it down into a grid of these smaller blocks to process them one by one. Like cutting a jigsaw puzzle into individual pieces and examining each piece separately before seeing how they fit together.

Patchification​

Splitting a (latent) tensor into a sequence of small square patches and turning each one into a single token, so a transformer can treat an image like a sentence of words. For example, a 32×32 latent cut into 2×2 patches becomes a sequence of 256 tokens (a 16×16 grid), each a little block projected to the model's hidden width. The patch size is the key knob: smaller patches make more tokens (finer detail but more compute), bigger patches make fewer tokens (cheaper but coarser) — a suffix like "/2" in DiT-S/2 means patch size 2. Like slicing a photo into postage-stamp squares and reading them left-to-right, top-to-bottom. The same idea extends to video by cutting spatiotemporal patches — little 3D boxes that also span a few frames in time, so one sequence of tokens carries both motion and appearance.

PCA (principal component analysis)​

A technique that finds the few directions along which data varies the most and uses them to compress many numbers down to a handful, so high-dimensional data can be drawn on a 2D plot. Imagine photographing a 3D object from the angle that reveals its shape best — PCA picks that most-informative "camera angle" automatically. It is a quick, standard first step for seeing the structure in data, such as checking whether real images cluster together while random noise scatters apart.

PCIe​

The standard CPU-GPU connection (and slower GPU-GPU when no NVLink)

Peg-in-hole​

A classic benchmark task in robotic assembly where a robot inserts a peg into a closely-fitting hole. Because the tolerance is extremely small, small positioning errors can cause the peg to jam or damage the parts if the robot uses rigid position control. Solving it typically requires a combination of software compliance (like impedance control), force-search algorithms (like spiral search), or tactile feedback.

  • Analogy: Trying to plug a charger into a wall outlet in the dark. If you push straight forward and you're slightly off-center, you hit the wall and block. Instead, you touch the plug gently against the wall, slide it in a circular or spiral pattern until you feel it drop into the slot, and then push it in.
  • How it works: The robot moves to the target location and commands low virtual stiffness. When it detects contact force indicating it has missed the hole, it executes a spiral search pattern while maintaining a constant downward force. Once the contact forces drop (indicating alignment), the robot increases stiffness to complete the insertion.

Pendulum​

A small continuous-control task (Pendulum-v1 in Gymnasium): a single pole hangs from a pivot, and the agent applies a continuous torque to swing it upright and hold it balanced. The torque is too weak to lift the pole straight up in one go, so the agent must learn to rock it back and forth to build momentum first — which is what makes a seemingly trivial task a real test. The state is just three numbers (the pole's angle, given as its sine and cosine, plus its angular velocity) and the single action is the continuous torque. Because it is the smallest continuous-action environment, it is the standard sanity check for DDPG-family algorithms before scaling to MuJoCo robots.

Perceiver IO​

DeepMind's modality-agnostic architecture that handles inputs of any size or type — pixels, audio samples, point clouds — without the cost normally blowing up. Plain attention compares every input element with every other, so a million-pixel image would need a million-by-million grid; Perceiver instead keeps a small fixed set of learned latent vectors (say 256 of them) and lets only those latents cross-attend to the giant input, squeezing it into the small set once, then doing all the heavy processing among just the 256. The "IO" version adds a matching trick on the output side: a set of learned query vectors cross-attends to the processed latents to read out an answer of whatever shape you need. Like a small committee (the latents) that skims a huge pile of documents, takes compact notes, deliberates among themselves, and then answers any question put to them — the committee's workload depends on its own size, not on how tall the pile was. Because nothing in it assumes a grid or a sequence, the same architecture works across modalities with almost no changes, which is its headline selling point. It is closely related to the Q-Former, which uses the same small-set-of-learned-queries idea to distill an image for a language model.

Percentile​

A way to describe where a value ranks in a sorted list: the p99 latency is the time that 99% of requests beat, with only the slowest 1% taking longer. Unlike an average, which a single huge outlier can hide, percentiles expose the slow tail that users actually feel — like reporting "even the slowest of the top 99% of diners was served within 20 minutes" instead of a misleading table-wide average. Serving teams quote p50, p95, and p99 rather than the mean for exactly this reason.

Perceptual loss (LPIPS)​

A loss that compares two images by the features a pretrained network sees in them, rather than by their raw pixels. Two photos shifted by a single pixel are nearly identical to a human eye but very different under pixel-by-pixel error; a perceptual loss judges them the way an eye does, rewarding matching textures and shapes. Training with it (LPIPS — Learned Perceptual Image Patch Similarity — is the popular version) gives much sharper results than plain pixel MSE, which tends to blur. It is widely used inside VQ-GAN and VAE training.

Per-channel quantization​

A quantization scale granularity where a separate scaling factor (multiplier) is calculated and applied to each individual output channel (or row) of a weight matrix.

  • Why it matters: In a neural network, weights in different channels can have vastly different ranges of values. If you use a single scale for the entire tensor, a single channel with extremely large values (outliers) will dictate the scale, squeezing all other channels into just a few integer bins and causing high quantization error. Computing scales channel-by-channel preserves the relative differences in every channel, maintaining high model accuracy.
  • Analogy: Imagine tailoring clothes for a group of people. If you use a single "average" size for the entire group (per-tensor), the clothes will not fit anyone well. Per-channel quantization is like taking individual measurements for each person so everyone gets a well-fitting outfit.
  • Example: Quantizing an LLM's linear layer weights using per-channel scaling allows each row of the weight matrix to be scaled independently, preserving fine-grained details in the model's parameters.

Per-group quantization​

A quantization scale granularity that divides a matrix row or channel into smaller, fixed-size groups of elements (typically 32, 64, or 128) and calculates a separate scaling factor for each group.

  • Why it matters: It acts as a middle ground between per-tensor and per-channel quantization. By grouping elements, it isolates outliers to their local groups, preventing them from corrupting the scale of other elements in the same row. This is particularly popular in ultra-low-bit formats like int4, where the range of 16 values is too narrow to span an entire channel without grouping.
  • Analogy: Imagine organizing a large warehouse. Instead of using a single temperature control for the entire building (per-tensor) or placing an expensive thermostat on every single shelf (per-channel), you divide the warehouse into a few zones (per-group) and control the temperature of each zone independently.
  • Example: In AWQ or GPTQ, weights are often quantized with a group size of 128, meaning every 128 elements in a row share a scaling factor, keeping accuracy high while keeping scale overhead low.

Per-tensor quantization​

A quantization scale granularity where a single scaling factor (multiplier) is calculated and applied to an entire weight or activation tensor as a whole.

  • Why it matters: Because there is only one scale factor to store and apply per tensor, per-tensor quantization has the lowest memory overhead and is the simplest to implement on hardware. However, it is highly sensitive to outlier values: if even one number in the tensor is very large, the scale factor must accommodate it, which reduces the precision of all other numbers in the tensor.
  • Analogy: Imagine setting a single maximum speed limit for every road in a country—highways, suburban streets, and narrow alleys alike. While it is simple to enforce (only one number to remember), it is highly inefficient because cars on highways are forced to crawl, while suburban streets might have speed limits that are too high.
  • Example: In simple symmetric quantization, a tensor is scaled using its single maximum absolute value, which is cheap to compute but can lead to accuracy loss if the tensor has a wide range of values.

permute​

Reorders all of a tensor's dimensions by rewriting strides — never copies

Perplexity​

A score for how surprised a language model is by a piece of text — roughly, how many words it was effectively choosing between at each step. Lower is better: a perplexity of 1 means the model knew exactly what came next, while a high number means it was guessing wildly. Because it is cheap to compute and rises the moment a model gets worse, it is a common first tripwire in a quality gate after quantization.

Pessimism​

The deliberate strategy of under-estimating value where you are uncertain, so an agent never bets on something it has not really seen. It is the core principle behind value-based offline RL: rather than trusting a Q-function's optimistic guesses for out-of-distribution actions, methods like CQL push those estimates down so the policy sticks to actions the data supports. It is the exact opposite of the optimism in the face of uncertainty that drives online exploration: online, optimism is good because a hopeful guess that turns out wrong gets corrected the moment you try it; offline you can never try it, so hope is a trap and caution wins. Analogy: investing only in companies whose books you have actually audited, and treating the unaudited ones as worthless until proven otherwise.

PETS​

Probabilistic Ensembles with Trajectory Sampling — an early, influential model-based RL method that learns an ensemble of probabilistic dynamics models (each outputs a distribution over next states, not a single guess) and plans by random shooting: sample many action sequences, roll each through the ensemble, and execute the first action of the best-scoring one (model predictive control). The ensemble matters because it captures two kinds of uncertainty — the environment's own randomness and the model's ignorance where data is thin — so the planner is not fooled by a single overconfident network. PETS showed that careful uncertainty handling lets model-based RL match model-free methods like SAC using far fewer samples. Analogy: planning a hike from several different trail forecasts and trusting a route only when most of them agree it is safe.

Physical plausibility​

Whether the motion and interactions in a generated video obey the everyday rules of the physical world — gravity pulls things down, water flows downhill, solid objects do not pass through each other, dropped things fall instead of hovering. Also called physical realism or physical correctness. The catch is that a clip can score well on appearance metrics like FVD — every frame looks sharp and real — while still getting the physics badly wrong, because looking right and behaving right are different things. It stays largely unmeasured: there is no clean benchmark for it yet, so it is usually probed by hand with trick prompts (water flowing uphill, a glass that should shatter). Object permanence and world consistency are specific facets of it.

PID​

Proportional-Integral-Derivative (PID) is a classic control loop feedback mechanism used to keep a system at a desired target state (called a setpoint). It continuously calculates an error value as the difference between the target and the actual state, and applies a correction based on three terms:

  1. Proportional (P): Corrects based on the current error. If you are far from the target, make a big adjustment; if close, make a small one.
  2. Integral (I): Corrects based on the accumulation of past errors. If the error has been persisting for a long time, apply more force to push it over the finish line.
  3. Derivative (D): Corrects based on the predicted future error, by looking at how fast the error is changing. This acts as a brake to prevent overshooting the target.

Analogy: Imagine adjusting the temperature of a shower. The Proportional part is how much you turn the knob when the water is not right (freezing water gets a big turn, slightly cool water gets a tiny nudge). The Integral part is your growing frustration: if you have been standing in lukewarm water for five minutes, you turn the knob further to finally get it hot. The Derivative part is when you feel the water rapidly heating up and approaching your perfect temperature, so you start turning the knob back toward the middle before it gets too hot, preventing yourself from getting scorched.

Example: A drone trying to hover at a height of 5 meters.

  • Proportional: If the drone is at 1 meter (4 meters below target), the motors spin very fast to push it up. If it is at 4.9 meters, the motors spin only slightly faster than hover speed.
  • Integral: If a constant wind pushes the drone down, proportional control alone might leave it stuck hovering at 4.8 meters. The integral term notices this persistent error over time and gradually increases motor power to lift it to exactly 5 meters.
  • Derivative: As the drone shoots upward and quickly approaches 5 meters, the derivative term detects the rapid speed and slows down the motors so the drone gently settles at the target height instead of overshooting into the ceiling.

Pinned memory​

Page-locked CPU memory that enables faster, asynchronous transfers to the GPU; enabled with pin_memory=True on a DataLoader.

Pinocchio​

A fast open-source C++/Python library for rigid-body kinematics and dynamics — load a robot model and it efficiently computes forward kinematics, Jacobians, and the equations relating forces to accelerations, with derivatives for optimization. It is a standard reference implementation you check your own from-scratch code against. The named algorithms it ships — CRBA (build the mass matrix), RNEA (inverse dynamics: the torques needed for a desired motion), and ABA (forward dynamics: the motion produced by applied torques) — are the classical O(n) recursions of robot dynamics, meaning their cost grows only linearly with the number of joints n. (Named, with a wink, after the puppet whose strings pull his joints.)

Pinhole camera model​

The simple geometric rule for how a camera turns a 3D point into a 2D pixel: imagine light passing through a single tiny hole onto a flat sensor behind it, so every world point projects along one straight ray to one pixel. In math, the point is first moved into the camera's frame, then divided by its depth (things twice as far appear half as big — this divide is exactly what creates perspective), then scaled by the camera intrinsics. It ignores the lens's real thickness and its distortion, which is why calibration adds distortion terms on top of it. Analogy: a pinhole camera (camera obscura) made from a shoebox — the model is named after exactly that bare-bones device, because its math assumes an idealized hole with no lens at all.

Pitch​

The rotation of a vehicle or object tilting forward or backward (like a nose-down or nose-up tilt of an airplane).

  • Why it matters: In robotics and aerospace, knowing which way an agent is tilting up or down is critical for balance, trajectory tracking, and keeping cameras pointed correctly. For example, if a drone pitches forward too much, it will fly forward rapidly but lose altitude.
  • How it works: Pitch is one of the three Euler angles used to describe 3D orientation. It is rotation around the transverse (lateral or side-to-side) axis. In a quadrotor, pitch is controlled by making the front motors spin slower and the rear motors spin faster (to pitch forward), or vice-versa.
  • Analogy: Imagine nod-shaking your head "yes" (moving your chin up and down). Your head is pitching.
  • Example: When an autonomous car drives up a steep ramp, its pitch angle increases as the nose of the car points up.

PixelCNN​

An autoregressive image model — a CNN (Convolutional Neural Network) repurposed for generation — that draws a picture one pixel at a time, predicting each pixel from the pixels already drawn above it and to its left — like filling in a coloring grid square by square, always glancing back at what you have already colored to decide the next color. The image quality is strong and it can report an exact probability for any picture, but generating one is slow because the pixels must come out strictly in order, each waiting on the one before it.

Planning​

Using a model of the environment to think ahead before acting: simulate the consequences of candidate actions through the model and choose the one that looks best, rather than reacting from a memorized policy. At decision time this usually means model predictive control — search for a good short action sequence (by random shooting, the Cross-Entropy Method, or MCTS), execute only the first action, then re-plan with fresh information. Analogy: a GPS that re-routes at every intersection by simulating the roads ahead, instead of blindly following a route it memorized once. Planning trades extra computation at each step for better decisions, and is the defining tool of model-based RL.

Plücker coordinates​

A way to describe a single straight line (here, the ray of sight through one pixel) using six numbers instead of a point-plus-direction. The six split into the ray's direction and its moment (a cross product that pins down which parallel line it is), so a line floating anywhere in 3D space gets one compact, position-independent code. Video models use them for camera control: give every pixel of every frame its Plücker ray and the model knows exactly which way the camera is looking, which lets learned camera moves generalize to angles never seen in training — far better than feeding raw camera-position numbers. Named after the 19th-century mathematician Julius Plücker, who introduced this line geometry.

PnP (Perspective-n-Point)​

The algorithm that recovers a camera's 6-DoF pose relative to an object from n known 3D points on that object and where each appears as a 2D pixel in the image. Given those point-to-pixel pairs and the camera intrinsics, it back-solves the one camera position and orientation that would project every 3D point onto its observed pixel. It is the engine behind reading an AprilTag's pose from its four corners. Analogy: working out exactly where you stood and which way you held the camera, purely from how a building's known corners line up in your photo.

PoC​

Proof of Concept — a small, rough build whose only job is to show that an idea can work, before anyone invests in a polished version. Like frying one test pancake to check the batter before making the whole stack: you are not trying to serve it, just to learn whether the approach is sound.

Point cloud​

A set of points in space, each placed by its numbers. In robotics and 3D vision this is the most literal case: a depth sensor or LiDAR scanner returns thousands of (x, y, z) points sampled off the surfaces it sees — the raw 3D shape of a scene, before any meshing or labeling. The idea generalizes to data of any kind: turn every image in a batch into a feature vector — a single point — and the whole batch becomes a cloud of such points. Comparing two clouds (say, real images vs. generated ones) is how a metric like FID measures similarity: it is like comparing two swarms of bees and asking whether they hover in the same spot and spread out in the same shape.

Point cloud registration​

Finding the rigid transform — a rotation and translation — that brings one point cloud into alignment with another of the same scene captured from a different viewpoint. It is how a robot stitches many partial depth scans into one consistent 3D model, or figures out how far it moved between two scans. Iterative Closest Point (ICP) is the classic method. Analogy: fitting two overlapping photos of the same wall into a single panorama by sliding and rotating one until the shared bricks line up.

Policy​

In reinforcement learning, the model being trained to choose what to do next — for an LLM, the network that picks the next token. "Improving the policy" just means making those choices earn more reward.

Policy constraint​

One of the two main families of offline RL fixes: force the learned policy to stay close to the behavior policy that produced the dataset, so it can only choose actions the data actually contains and never wanders into out-of-distribution territory where values are unreliable. Methods differ in how they enforce closeness — BCQ generates only actions resembling the data, BEAR and BRAC add a distance penalty between the two policies, and AWAC tilts the policy toward high-advantage dataset actions. It is the counterpart to value pessimism (CQL, IQL): constrain what the policy outputs versus distrust what the value function predicts. Analogy: a new manager told to run things only the way the previous, trusted manager did, instead of being free to try untested ideas.

Policy evaluation​

The task of computing the value function of a given, fixed policy — answering "if I always act this way, how much reward should I expect from each state?" — without yet trying to improve the policy. Because the policy is fixed, the Bellman equation becomes a set of linear equations with one unknown per state, so it can be solved two ways: in one shot with a matrix inverse, or by repeatedly applying the Bellman operator until the numbers stop changing. It is the evaluation step that, alternated with a policy-improvement step, builds up to the optimal policy.

Policy gradient theorem​

The result that makes it possible to improve a policy by gradient ascent directly, without ever learning a value function to act greedily on. It states that the gradient of expected return with respect to the policy's weights is ∇J = E[ ∇log π(a|s) · Q(s,a) ] — read as: to make good outcomes more likely, push up the log-probability of each action in proportion to how good that action turned out to be. The factor ∇log π(a|s) is supplied by the log-derivative trick, and Q(s,a) (or an advantage, once you subtract a baseline) is the "how good" weight. Analogy: after each round you slightly increase your tendency toward the moves that paid off and decrease it toward the ones that did not — averaged over many rounds, this nudges the whole strategy uphill. It is the foundation of REINFORCE, A2C, PPO, and TRPO.

Policy iteration​

A dynamic-programming algorithm that solves a known MDP by strictly alternating two phases until neither changes: policy evaluation — compute the value function of the current policy exactly — and policy improvement — replace the policy with the greedy one with respect to those fresh values. Each improvement is guaranteed to give a policy at least as good, and because a finite MDP has only finitely many policies, it reaches the optimal policy in surprisingly few rounds — usually far fewer than value iteration, though each round costs more because the evaluation phase is solved to completion. Unlike value iteration, which takes a tiny step of improvement after every single evaluation sweep, policy iteration fully evaluates the policy before improving it. Think of finding the best route to work: policy iteration is like driving one specific route every day for a month until you perfectly know its average time, then choosing a new route to test; value iteration is like driving a route once and immediately updating your guess for the best path. It is the original instance of generalized policy iteration.

Polyak averaging​

A way of updating a target network smoothly instead of in jumps. Rather than copying the online network's weights wholesale every N steps (a "hard" update), every weight is nudged a tiny fraction τ of the way across on every step: θ_target ← (1 − τ)·θ_target + τ·θ_online, with τ typically around 0.005. Also called a soft update, or an exponential moving average of the weights. The trade is the same one the hard update makes, just spread out: the target still lags the online network (which is the point — a target that moves the instant the prediction does is what makes bootstrapping unstable), but it drifts continuously instead of lurching. Like repainting a wall with one thin coat a day rather than stripping and redoing it every month. DDPG, TD3, and SAC all default to it; DQN traditionally uses hard copies, and on easy tasks the choice matters less than having a target network at all.

POMDP​

Partially Observable Markov Decision Process — an MDP in which the agent does not see the true state, only a partial or noisy observation of it. Because that observation alone breaks the Markov property (it no longer holds everything needed to act well), a policy that reacts only to the current observation is provably suboptimal — to act well the agent generally has to remember past observations, e.g. with a recurrent network or an explicit belief state over which state it is probably in. Example: a robot with only a forward camera that cannot see what is behind it, or a poker player who cannot see opponents' cards. Many real problems that look like MDPs are secretly POMDPs because the "state" the designer chose leaves something important out.

Pong​

The simplest Atari game — a two-paddle table-tennis match — and the traditional first pixel-based benchmark for DQN. It is a favorite starting point because the reward is relatively dense (you score or concede a point every few seconds rather than waiting minutes) and the visuals are minimal, so an agent can reach expert play in far less training than hard-exploration Atari titles like Montezuma's Revenge.

Pose graph​

A network used in robotic mapping and SLAM where a robot's path and measurements are structured as nodes and edges. Each node represents a "pose" (the robot's estimated position and orientation at a specific moment), and each edge represents a relative measurement between two poses (for example, "Pose B is 1 meter directly in front of Pose A," as measured by wheel odometry or scan matching). When the robot detects a loop closure (recognizing it has returned to a previously visited location), it adds a new edge connecting the current pose node back to the older pose node, forming a closed loop. The system then runs a global optimization to adjust all pose nodes simultaneously to satisfy all the edges, which corrects accumulated tracking drift. Analogy: drawing a sketch of a walking path on paper step-by-step. Because your estimated steps are slightly off, your drawing slowly drifts. If you walk in a circle and return to your starting point, your drawing's end point won't line up with the start. By pinning the start and end points together (adding a loop closure edge) and gently shifting the drawn lines so everything aligns consistently, you are optimizing a pose graph.

Position bias​

A judge's tendency to pick an answer based on where it sits rather than what it says — for example, an LLM-as-judge that quietly prefers whichever response appears first (or last) when shown two side-by-side. Like a job interviewer who can't help favoring the candidate they meet right after lunch, regardless of qualifications. The standard fix is to ask the judge twice with the two answers swapped and accept the verdict only if both runs name the same winner.

Position interpolation​

Extending a model's context length by linearly rescaling RoPE position indices so longer sequences fall within the trained range

Position vector​

A vector that represents the exact location of a specific point in space. You make one by drawing a straight geometric arrow from the origin — the (0, 0, 0) center of the coordinate system — directly to your target point. If a point lives at coordinates (x, y, z), its position vector is simply the vector [x, y, z].

Why do we need this? In geometry, a point is just a fixed location, while a general vector is just a movement (a direction and a length, like "walk 5 steps North") that can float anywhere in space. A position vector bridges the two: by permanently anchoring the tail of the arrow to the origin, the vector perfectly describes that specific location. This is the mathematical trick that lets you plug a fixed point into vector operations like the cross product.

Posterior collapse​

A VAE failure where the decoder grows strong enough to reconstruct inputs on its own and simply ignores the latent space. The encoder then stops bothering to encode anything and just outputs the default prior, so the latent variables carry no information about the input — like a student who has memorized the answer key and no longer reads the question. When this happens the KL divergence term drops toward zero and the latent code becomes useless for generation.

Postmortem​

A written review done after an incident — an outage, a slowdown — that lays out what happened, how it was detected and fixed, and what will stop it recurring. A good one is blameless: it focuses on the system and the process, not on punishing a person, like an air-crash investigation whose goal is safer future flights rather than someone to fire.

Polysemantic​

A single neuron or activation dimension that fires for several unrelated concepts at once — the opposite of monosemantic. It arises from superposition: the network has far more concepts to represent than it has dimensions, so it packs many of them into overlapping directions and tolerates the interference. Like a single dictionary entry that lists five unrelated meanings, forcing you to guess which one is meant. Polysemanticity is why raw activations are hard to read, and why sparse autoencoders are trained to unpack them into separate monosemantic features.

PUCT​

The rule that decides which branch MCTS explores next, used by AlphaZero and MuZero. At each node it scores every move by Q + c · P · √N_parent / (1 + N_move) and takes the highest. Read the two halves as a negotiation: Q is what we have learned — the average outcome of the times we tried this move — and pulls the search toward moves that have already looked good. The rest is what we have not checked — it is large when the policy network's prior P likes the move but we have rarely tried it (N_move small), and it shrinks every time we do try it. So a move that looks promising but is untested gets pulled in; once it has been examined enough, its bonus fades and its actual results have to speak for themselves. That is exploration vs exploitation, settled per-node, with c setting the exchange rate.

PPO​

Proximal Policy Optimization — the workhorse on-policy actor-critic algorithm, the default starting point for most RL projects in 2026 and the engine of classic RLHF. "Proximal" means staying close: each update is kept near the current policy so a single noisy batch can never push it too far and wreck what already works. It achieves this cheaply by forming the importance ratio π_new/π_old (how much more likely the new policy makes each past action), multiplying it by the advantage, and then clipping that ratio to a narrow band around 1 — so once an update has moved far enough, pushing further earns no extra reward and the incentive to overshoot disappears. This is a one-line stand-in for the explicit trust region that TRPO enforces with heavy constrained optimization. PPO is famous less for its math than for its robustness — it tolerates wrong learning rates and missing normalizations that would sink fancier methods — which is exactly why it ships everywhere.

Precision and recall​

Two numbers that, used together, describe how a yes/no detector is doing — far more honest than a single accuracy figure. Precision asks "when the model says yes, how often is it right?" — of all the times it shouted "dog!", what fraction really had a dog. Recall asks "of all the real yes-cases, how many did it catch?" — of all the images that truly had a dog, how many it found. You compute each as a simple fraction: precision = true positives / (true positives + false positives); recall = true positives / (true positives + false negatives). They trade off against each other — a model that says "yes" to everything has perfect recall but terrible precision — which is exactly why a hallucination probe must report both, not just accuracy. Analogy: a fisherman's net — precision is how much of the catch is the fish you actually wanted (not boots and weeds), and recall is how many of the lake's fish you managed to net at all.

Prediction head (MuZero)​

The network inside MuZero that estimates the policy and the value from a given abstract latent state — written f(s) → p, v.

What it does: It looks at an abstract state (either a real one or one imagined by the dynamics head) and immediately outputs which moves are best to try (policy p) and how likely the agent is to win from here (value v), guiding the Monte Carlo Tree Search.

Analogy: A seasoned player looking at a board layout has a gut feeling: "I have about an 80% chance of winning from this position (value v), and my best next move is probably to move my rook or attack the bishop (policy p)." The prediction head is this quick intuition.

Prediction error​

The difference between what an agent's internal model expects to happen and what actually happens when it takes an action. In curiosity-driven exploration, the agent uses this difference as an intrinsic reward, treating a high prediction error (a big surprise) as a sign that the transition or state contains new information worth exploring. For example, in the Intrinsic Curiosity Module (ICM), prediction error is measured as the distance between the predicted next-state features and the actual next-state features. Analogy: Imagine trying to catch a ball. If you predict it will land in your hand but it bounces away unexpectedly, your "prediction error" is high. That surprise prompts you to pay closer attention to how the ball bounces so you can improve your prediction next time. Similarly, a curiosity agent seeks out situations where its mental model of the world fails, using those errors to guide its learning. However, if the error is caused by pure randomness (like a noisy-TV problem), prediction error fails as a guide because the agent can never learn to predict it.

PREEMPT_RT​

A popular real-time patch for the Linux kernel that turns Linux into a real-time operating system (RTOS). It works by making almost all parts of the kernel preemptible—meaning a high-priority user thread (like a robot control loop) can immediately interrupt a lower-priority task, even if that task is inside a system call or device driver. This guarantees that critical threads execute within a tightly bounded time limit, reducing scheduling latency and jitter. Analogy: An emergency lane on a freeway. In standard Linux, a garbage truck (a low-priority system task) can block the entire road, and an ambulance (your 1 kHz control loop) has to wait in line. With PREEMPT_RT, the ambulance has an absolute right-of-way and can immediately force all other traffic to pull over, ensuring it arrives exactly on schedule. Without this patch, standard Linux is prone to timing spikes that can cause a robot controller to miss steps, leading to unstable control or physical collisions.

Prefill​

The first stage of LLM inference: reading the entire prompt at once to fill the KV cache, before any new tokens are generated. Because all the prompt's tokens can be processed together in a single forward pass, prefill is compute-heavy and fast per token — like a reader skimming a whole page at a glance to grasp it before starting to write a reply. It is the opposite of decode, which then produces the answer one token at a time, and prefill time is what sets the time to first token.

Prefix cache​

Sharing KV cache across requests that begin with the same tokens (e.g., system prompts)

Pretraining​

Self-supervised training on a large unlabeled corpus to predict the next token

Primitives​

A set of basic, pre-programmed or learned sub-tasks or motion sequences that a robot can execute as a single command (such as picking up an object, placing it, or opening a drawer). By combining these simple building blocks, a high-level planning algorithm can solve complex, long-term tasks without having to calculate every individual joint torque or coordinate from scratch.

  • Why it matters: Planning a complex sequence (like "make a cup of coffee") at the level of raw motor voltages or millimeter-scale joint movements is extremely difficult and computationally expensive. By breaking the robot's capabilities down into a set of reliable, modular primitives, high-level schedulers can focus on abstract logic rather than fine motor control.
  • How it works: Primitives can be hand-coded controllers (like a specific trajectory-following algorithm) or learned policies trained via reinforcement learning. A high-level scheduler or planner (such as SayCan) decides which primitive to run based on the current situation, executing them one after the other.
  • Analogy: Think of cooking a meal using a recipe. Instead of telling your individual muscle fibers how to contract to hold a knife, slice a tomato, or turn a dial (which are basic, pre-learned "primitives"), the recipe instructions just say "chop the tomato" and "sauté."
  • Example: In a mobile kitchen task, a robot is equipped with primitives like pick(object), navigate(location), and place(object). To clear a cup, the high-level planner chains these primitives together: navigate(table) -> pick(cup) -> navigate(sink) -> place(cup).

Prior-preservation loss​

An extra training term used by DreamBooth to stop a model from forgetting a whole class while learning one specific member of it. When you fine-tune on five photos of your dog, the model risks deciding every "dog" now looks like yours — a form of catastrophic forgetting. Prior preservation counters this by mixing in the model's own generic "a photo of a dog" images during training and asking it to keep reproducing them, so the broad concept of "dog" is preserved while the narrow concept of your dog is added on top. Like teaching someone your cousin's face without making them forget what faces in general look like.

Prioritized experience replay (PER)​

An upgrade to experience replay that samples transitions in proportion to how surprising they are, instead of uniformly at random. "Surprise" is measured by the magnitude of a transition's last TD error — the gap between predicted and bootstrapped value — on the logic that the agent learns most from the experiences it currently predicts worst. Sampling by weight from a buffer of millions is made cheap with a sum-tree, and an importance-sampling weight is applied to each update to undo the bias the non-uniform sampling would otherwise introduce. It is one of the components folded into Rainbow. Analogy: a student who spends extra time on the flashcards they keep getting wrong, rather than reviewing every card equally.

PRM (Probabilistic Roadmap)​

A multi-query, sampling-based motion planning algorithm that builds a reusable graph (roadmap) of the robot's C-space. It has two phases: (1) Learning phase: it samples random configurations in C-space, keeps those that are collision-free, and connects neighboring points with straight-line paths if they don't collide, forming a roadmap. (2) Query phase: to plan a path from a start to a goal, it connects both to the roadmap and uses a graph search algorithm (like A* search) to find the shortest path along the roadmap. Analogy: Imagine building a transit map for a new city. First (learning phase), you survey the land and build a network of subway tracks and stations connecting major landmarks. Once the tracks are laid (query phase), any commuter can plan a trip between any two stations instantly by looking at the map. Example: A mobile warehouse robot operating in a static environment. A PRM roadmap is built once for the warehouse layout. The robot can then repeatedly plan collision-free paths between different inventory shelves instantly without rebuilding the map.

PRM800K​

A public dataset of about 800,000 human labels that mark each step of a math solution as right or wrong, released by OpenAI to train process reward models. Rather than only checking whether the final answer was correct, human graders read each worked solution line by line — like a math teacher putting a check or an X next to every step of a student's proof, not just the boxed answer at the bottom. Because the feedback is step-level, a model trained on it learns to spot exactly where the reasoning went off the rails instead of whether the ending happened to be lucky. It is the standard training set for the step-by-step scorers used in Best-of-N re-ranking.

Probability density​

A function that says how likely each possible value is — high where real data points pile up, low in the empty regions where they rarely fall. For a 2D dataset you can picture it as a heatmap: bright ridges over the crowded spots, dark valleys over the bare ones. It must stay non-negative everywhere, and all of it added up (the total volume under the surface) equals exactly 1, since some value always occurs. Most generative models can only draw new samples; a normalizing flow is special because it can also report the exact probability density of any point you hand it.

Probability flow ODE​

The deterministic twin of a diffusion model's reverse-time SDE: an ODE with no injected randomness that produces the same distribution of images at every noise level. Determinism buys two things the stochastic sampler can't: the same starting noise always maps to the same image (so you can interpolate between samples and invert a real image back to its noise), and the model's exact log-likelihood of any image — how probable it thinks that image is — becomes computable via the ODE's change-of-variables. It is the basis of fast deterministic samplers like DDIM.

Process reward model​

A scorer that grades each individual step of a model's reasoning rather than just the final answer — like a teacher marking every line of a proof, not only the last one — so a mistake can be caught at the exact step it happens. Contrast with an outcome reward model.

Profiler​

A tool (torch.profiler) that records how long each operation in a training step takes, used to locate performance bottlenecks.

Projection discriminator​

A way to feed a class label into a conditional GAN's discriminator by taking a dot product between the image's features and a learned vector for that class, then adding it to the score — rather than just gluing the label on as an extra input. This matches how the math of conditioning actually factorizes, so it conditions more strongly for almost no extra cost, and it became the standard trick for class-conditional GANs such as BigGAN.

Projector​

The small network — often a single linear layer or a two-layer MLP — that maps one modality's feature vectors into the space another model expects. It initially acts as a physical adapter cable that reshapes one plug into another (e.g., resizing a 1024-dimensional image vector into a 4096-dimensional word vector). Crucially, simply matching dimensions is not enough; the projector must undergo alignment training (like installing a software driver for the adapter) to learn the exact mathematical transformation that routes the visual semantics into the LLM's native coordinate space. This is the entire fusion mechanism in LLaVA: freeze the vision encoder, freeze the LLM, and train only this projector to perfectly align the two spaces. The catch is that all the image information must squeeze through this one thin bridge, so it can become a bottleneck on detail-heavy tasks.

Probing classifier​

A small classifier — usually logistic regression or a shallow MLP — trained on a frozen model's internal activations to test whether some property (is this statement true? what part of speech is this token?) is linearly readable from a given layer. The language model is never updated; only the probe is. If the probe succeeds, the model represents that property internally, and comparing probe accuracy across layers shows where. It is the workhorse first tool of mechanistic interpretability; see linear probe. The main caveat: a probe reveals correlation, not that the model actually uses that information downstream — for that you need causal methods like activation patching.

Prompt injection​

An attack in which adversarial text smuggled into something the model reads — a retrieved document, a tool's output, an email, even text inside an image — overrides the original system instructions. Like a customer slipping a fake "manager-approved" note into a server's order pile: the server can't easily tell the planted note from a real one. The hardest unsolved security problem in deployed LLMs, because the model has no built-in way to separate "instructions" from "data" in its input.

Prompt-to-Prompt​

A diffusion editing technique that changes what an image shows while keeping its layout intact, by reusing the cross-attention maps from the original generation. Those attention maps act like a set of stencils, recording exactly which word controls which region of the image (for example, the map for the word "cat" literally points to the pixels where the cat is drawn). If you change the prompt from "cat" to "dog" but force the new run to reuse the old attention maps—meaning you tell the model, "draw the dog exactly inside the stencil you previously used for the cat"—the dog lands in exactly the same pose and place as the cat. Picture keeping a painting's pencil under-drawing fixed and only changing the colors you fill in. It is one of the tools used to build paired before/after data for InstructPix2Pix.

Pseudo-count​

A soft visit count for worlds where exact counting is useless. Count-based exploration needs N(s), the number of times state s has been seen — but when states are camera images, no two are ever byte-for-byte identical, so every count stays stuck at 1 and the bonus never fades. A pseudo-count recovers the idea by fitting a density model over states — a model of "how typical does this picture look to me?" — and reading a count out of how much that model's opinion changes after training on the state once: a familiar-looking state barely moves the model (high pseudo-count, small bonus), a genuinely new one moves it a lot (low pseudo-count, big bonus). This is what turned counting into a method that works on Atari pixels, and it is the intellectual ancestor of RND, which reaches the same goal by replacing the density model with a much cheaper regression error. Analogy: a librarian who cannot remember every book but instantly senses whether a new arrival is "another one of those" or something they have never shelved before.

PTQ / QAT​

Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) are the two main strategies for quantizing a model — that is, shrinking its weights (and sometimes its activations) from high-precision numbers down to small low-precision ones like int8, so the model uses less memory and runs faster. The two differ in when the rounding happens relative to training.

  • PTQ — round after training is done: You take an already-trained model and convert its numbers to low precision in one shot, without any further training. To pick good rounding rules, PTQ usually runs a small set of example inputs through the model first (a step called calibration) to measure the typical range of values each layer produces. PTQ is fast and cheap — no GPUs grinding for days — but because the model never got a chance to adjust, the rounding errors can nudge accuracy down, especially at very low precision.
  • QAT — practice with the rounding during training: You simulate the low-precision rounding while the model is still being trained (or fine-tuned), so the model sees the rounding errors and learns to compensate for them. This recovers more accuracy than PTQ, particularly for aggressive low-bit formats, but it costs far more compute because you have to keep training.
  • Analogy: PTQ is like translating a finished book into a language with a smaller vocabulary after it is written — quick, but some nuances get lost in the rounding. QAT is like having the author write the book in that smaller vocabulary from the start, choosing words that survive the limitation — more work, but the result reads better.
  • Example: A team shipping a chatbot to phones might first try PTQ (e.g. GPTQ or AWQ) because it takes minutes; if the int8 or int4 model loses too much quality, they fall back to QAT and fine-tune the model with simulated rounding to win the accuracy back.

Pure pursuit​

A path-tracking algorithm that computes the steering angle or angular velocity command required to guide a robot from its current position to a designated "look-ahead" point on a reference path.

  • Why it matters: A robot cannot simply drive toward the final goal; it must stay on a planned path (to avoid walls or stay in a lane). Pure pursuit is a simple, computationally cheap, and robust controller for keeping a mobile robot on a path.
  • How it works: It continuously identifies a target point on the reference path that is a fixed "look-ahead distance" (L) ahead of the robot. It then fits a circular arc from the robot's current position to that target point and calculates the curvature γ of that arc, which is used to command the robot's steering or rotation speed.
  • Analogy: Imagine a dog chasing a frisbee. The dog doesn't run to where the frisbee is currently or where it will land; instead, the dog keeps its eyes locked on a point slightly ahead of its current path and continuously steers its body toward that point as it runs.
  • Example: Autonomous warehouse delivery robots use pure pursuit to trace virtual lines on the floor at a constant speed, dynamically adjusting their steering as they round corners.

Pydantic​

A popular Python library for declaring the shape of your data as a class — you write a class with typed fields (e.g. name: str, age: int) and Pydantic validates that any data you load actually matches, raising a clear error if a value is the wrong type or a required field is missing. Like a customs form for data: anything that does not match the listed fields gets stopped at the border. In LLM work it is the standard way to describe the JSON object you want the model to produce, which tools like Outlines or OpenAI's structured-output mode can then enforce during decoding.

PyTorch​

The most widely used open-source library for building and training neural networks, originally from Meta AI. Its core object is the tensor — a multi-dimensional array, like NumPy's, but able to run on a GPU and to record the operations performed on it so gradients can be computed automatically (autograd). That automatic differentiation is what lets you define a model as ordinary Python code and have the training gradients worked out for you. It is "define-by-run": the computation graph is built on the fly as your code executes, which makes models easy to write and debug. Analogy: NumPy with a GPU engine bolted on and a tape recorder that remembers every step so it can replay them backward to learn.

Q-Former​

The fusion module from BLIP-2 (a 2023 vision-language model) that shrinks a whole image down to a fixed small number of tokens — typically 32 — that a language model can read. It holds a set of learned query vectors that cross-attend to the frozen image encoder's many patch features, each query pulling out one summary of what it cares about; the 32 outputs are then projected and fed to the LLM as if they were 32 word tokens. Like 32 interviewers who each question a sprawling exhibit and walk away with one concise note, so the language model reads 32 notes instead of touring the whole gallery. The point of the fixed count is cost control: an image becomes a constant, small number of tokens no matter its resolution, instead of hundreds. It shares the small-set-of-learned-queries idea with the Perceiver IO; later VLMs like LLaVA showed a plain projector often matches it with less complexity.

Q-learning​

The foundational off-policy temporal-difference control algorithm. It keeps a table of action-values Q(s, a) and, after each (state, action, reward, next-state) transition, nudges Q(s, a) toward r + γ · maxₐ′ Q(s′, a′) — "reward now plus the discounted value of the best action available next." Using that max in the target — rather than the action the agent actually goes on to take — is what makes it off-policy: it learns the value of the optimal policy even while exploring with a softer rule like ε-greedy. Its on-policy cousin SARSA instead plugs in the action actually taken next, learning more cautious behavior; swapping the lookup table for a neural network is the leap to deep Q-networks in Phase 3.

QLoRA​

Short for Quantized Low-Rank Adaptation — an efficient fine-tuning method that reduces memory usage by freezing a pre-trained large language model in 4-bit precision (specifically NF4), and training small, high-precision LoRA adapters on top of it.

  • Why it matters: Fine-tuning a standard 7B parameter LLM in bfloat16 precision requires massive amounts of GPU memory (~14 GB just to hold the weights, and up to 40 GB once optimizer states and gradients are added). QLoRA compresses the base model weights to 4-bit NF4 format, which drops the base model's memory footprint to ~3.5 GB. During the backward pass, the gradients are computed through the 4-bit weights and backpropagated into the 16-bit LoRA adapter parameters, allowing full-quality fine-tuning on a single consumer GPU (like a 24 GB card) instead of requiring expensive data-center clusters.
  • Analogy: Imagine wanting to paint a beautiful mural on a massive brick wall. Instead of painting the entire wall by hand (which requires a huge amount of paint and effort), you hang a thin, transparent plastic sheet over the wall and paint your new details onto the sheet. The original wall remains unchanged and protected underneath (the frozen 4-bit base model), while your changes are recorded in high resolution on the light sheet (the 16-bit LoRA adapters).
  • Example: Fine-tuning a 7B model on a single 24 GB GPU using QLoRA, which achieves the exact same downstream accuracy and perplexity as standard 16-bit LoRA fine-tuning but uses less than half the peak GPU memory.

Quadratic Program (QP)​

A mathematical optimization problem where the goal is to find the best variables to minimize (or maximize) a quadratic cost (which involves squared terms like x^2 or products like x · y) while satisfying a set of linear constraints (boundaries written as flat lines, like x + y ≤ 5).

  • Why it matters: Many robot control tasks must balance multiple competing goals (like keeping balance and tracking a path) while respecting strict limits (like maximum motor torque or not slipping). QPs can be solved to their single, mathematically guaranteed best solution (global optimum) in a fraction of a millisecond. This makes them the standard tool for real-time robotic controllers like Whole-Body Control (WBC) and Model Predictive Control (MPC), which must recalculate commands hundreds of times per second.
  • How it works: A QP solver is given a cost function to minimize, written mathematically as 1/2 · xᵀ · H · x + gᵀ · x (where H defines the quadratic costs like squared errors, and g defines linear costs), alongside linear inequalities Ax ≤ b representing the limits. The solver uses numerical search algorithms (such as active-set or interior-point methods) to find the unique point x that has the lowest cost while remaining inside the boundary lines.
  • Analogy: Imagine releasing a marble inside a bowl-shaped valley (the quadratic cost, where the bottom of the bowl is the lowest cost). If there are no obstacles, the marble naturally rolls to the bottom. Now, imagine putting straight, stiff fences across the valley (the linear constraints). The marble will roll down and stop at the lowest point it can reach, which will be resting against one of the fences. The QP solver finds exactly where that marble will stop.
  • Example: In a walking robot, a QP solver calculates the forces each foot should apply to the ground. The quadratic cost penalizes deviations from the target body posture, while the linear constraints ensure the forces stay within the motors' torque limits and inside the friction cones so the feet do not slip.

Quadrotor​

An aerial robot (drone) propelled by four rotors, where flight control is achieved by varying the speed of each rotor individually.

  • Why it matters: Unlike helicopters, quadrotors use fixed-pitch blades and rely on changing rotor speeds to generate lift, pitch, roll, and yaw. This mechanical simplicity makes them highly reliable, maneuverable, and popular for research, photography, and delivery tasks.
  • How it works: To hover, all four rotors spin at the same speed to balance gravity. To roll or pitch, the speed of one pair of opposite rotors is increased relative to the other. To yaw (rotate in place), the speed of the two clockwise rotors is increased relative to the two counter-clockwise rotors.
  • Analogy: Imagine holding a square table by its four corners. If you pull up on the two left corners and push down on the right corners, the table tilts (rolls). If you pull up on all four corners equally, the table rises straight up.

Quality filter​

A classifier that scores each training document and keeps only the high-quality ones (e.g. educational web text), discarding low-value text before pretraining.

Quality gate​

An automatic check that a model must pass before it is allowed to serve real traffic — like a bouncer at the door who turns away anyone failing the dress code. It runs a fixed set of evaluations (such as perplexity and capability tests) and blocks the deploy if any score drops too far from the trusted baseline, which is how teams catch silent quantization regressions before users do.

Quantization​

Reducing weight / activation precision (FP16, BF16, FP8, INT8, INT4) to save memory and bandwidth

Quaternion​

A four-number way to store a 3D rotation (w, x, y, z) that avoids the traps of Euler angles: no gimbal lock, smooth interpolation between orientations (the "slerp" used for animation and IMU fusion), and cheap composition. The four numbers place the rotation as a point on a unit-length sphere in 4D — effectively an axis-angle rotation folded into half-angle form — which is why you must renormalize back to length 1 after math to keep it valid. The one quirk to handle: q and −q describe the same rotation (a "double cover"), so loss functions and solvers must treat them as equal. Analogy: where Euler angles give directions as "turn left, then up, then roll" (order-dependent and easy to garble), a quaternion names the single combined spin directly, so it never gets confused.

Qwen​

A family of open-weight LLMs released by Alibaba, spanning sizes from under a billion parameters (e.g. Qwen-0.5B) up to very large mixture-of-experts models. The small variants are popular for learning and experimentation: a base model tiny enough to fine-tune on a single GPU still produces sensible text, which makes it a convenient starting point for an SFT or RLHF project.

RadixAttention​

sglang's KV cache organized as a radix tree keyed on prompt prefixes for automatic sharing

RAFT​

RAFT (Recurrent All-Pairs Field Transforms) is a neural network for computing dense optical flow, and on release one of the most accurate. Its core idea is to compare all pairs of pixels between the two frames to build a similarity volume, then iteratively refine a flow estimate with a recurrent update — repeatedly nudging the guess until it stops improving (which is the "recurrent" in its name). Analogy: it is a careful editor who, instead of guessing the motion once, keeps revising the answer over many small passes. Example: feeding RAFT two adjacent video frames returns a (H, W, 2) flow field that is far cleaner on fast motion than the classical Farnebäck method.

RAG​

Retrieval-Augmented Generation — give the model an "open-book exam" instead of asking it to answer from memory alone. First a search step fetches the documents most relevant to the question (from a company wiki, a manual, the web), then those documents are pasted into the prompt, and only then does the model write its answer using them as notes. This lets it use fresh or private facts it was never trained on, and makes it easy to check where an answer came from.

RAM​

Random Access Memory (also called main memory or system memory) — a fast, volatile hardware component that holds the data and program instructions currently being used by the CPU or GPU. In contrast to permanent storage (like SSDs), RAM allows extremely quick read and write access, but loses its contents when powered off.

Analogy: A workspace desk. Anything you are actively working on (documents, pencils, calculator) is laid out on the desk for immediate access. If you need something else, you must open the filing cabinet (storage/SSD), which takes much longer. At the end of the day, the desk is wiped completely clean.

Example: When loading a PyTorch model, the weights are first read from the hard drive and loaded into the system RAM. If you are training on a GPU, the weights are then transferred from the system RAM to the GPU's dedicated high-bandwidth memory (HBM).

Rainbow​

A 2017 agent that combines six independent improvements to DQN into one, demonstrating that they are complementary — the combination clearly beats any single piece. The six are: Double DQN, Dueling DQN, prioritized experience replay, n-step returns, distributional RL (the C51 variant), and noisy nets (which add learnable noise to the network's weights so exploration is driven by the agent itself rather than a hand-set ε-greedy rate). The name reflects blending these separately-developed techniques into a single agent. Its lesson — that well-chosen tricks stack rather than cancel — is why "just add the known improvements" is a reliable recipe in deep RL.

Random shooting​

The simplest way to plan with a dynamics model: sample a large batch of action sequences uniformly at random, roll each one forward through the model, add up the predicted reward for each, and execute the first action of whichever sequence scored highest — then re-plan next step (model predictive control). It needs no gradients and no tuning, which makes it a great first baseline, but it scales badly: as the action sequence grows longer, good sequences become vanishingly rare among purely random guesses, so smarter searches like the Cross-Entropy Method usually replace it. Analogy: finding a radio station by mashing random presets and keeping the clearest — fine with a few buttons, hopeless across a whole dial.

rank​

The unique integer ID of a process in a distributed job. RANK is the global ID across all machines; LOCAL_RANK is the ID within one machine; WORLD_SIZE is the total number of processes.

Raster order​

Walking through a 2D grid of pixels (or image tokens) one row at a time, left to right and top to bottom — the exact path your eyes take reading a page. The name comes from how old CRT TVs and monitors painted the screen: an electron beam swept across in horizontal lines called raster lines (from the Latin rastrum, "rake," because the lines look raked across the glass). An autoregressive image model that generates in raster order produces the top-left pixel first and the bottom-right pixel last.

RDMA​

Remote Direct Memory Access — letting one machine read or write another machine's memory directly over the network, without either CPU stopping to copy the data. Like a pneumatic tube that drops a package straight onto a coworker's desk instead of handing it to a courier who walks it over. In disaggregated serving it is how a prefill node ships a multi-gigabyte KV cache to a decode node fast enough to be worth splitting them.

Real NVP​

Short for "Real-valued Non-Volume Preserving" — an early, influential normalizing flow design. Its trick at each step: split the numbers into two halves, leave one half completely untouched, and use that untouched half to decide how to stretch and shift the other half. Because the untouched half is still right there, the step is trivially reversible (you can recompute the stretch-and-shift and undo it) and its effect on probability density is cheap to calculate. This made flows practical to train and inspired later models like Glow.

Reality gap​

The difference between a physics simulator (the virtual environment) and the physical world (reality) that causes robot control policies trained in simulation to fail when run on a real robot. Simulators cannot perfectly model complex physical properties like friction, contact forces, sensor noise, or mechanical backlash, making the virtual environment cleaner and more predictable than the real world.

  • Analogy: Imagine learning to ride a bicycle in a perfectly smooth, windless virtual reality room. When you go outside onto a real street with loose gravel, wind gusts, and uneven pavement, you might wobble and fall. The difference between that smooth virtual room and the real street is the reality gap.
  • How to bridge it: Researchers use techniques like domain randomization (randomizing simulator properties) and domain adaptation to help policies generalize across this gap.

Reasoning model​

An LLM trained to think out loud at length — writing a long chain of thought before its final answer — to solve harder problems (math, code, logic). Like a student who fills a page of scratch work before writing the answer, it is far more capable on tough questions but also far more expensive to serve, because one hard problem can produce 10× the tokens of a normal chat reply. Managing that swing in output length is the main serving challenge it creates.

ReAct​

A simple agent pattern that interleaves Reasoning and Acting: the model writes a thought, takes an action with a tool, reads the observation, then repeats — the loop most basic agents are built on.

Reciprocal rank fusion​

A simple, robust way to merge several ranked lists into one: each item scores the sum of 1 / (rank + constant) across the lists, so items ranked highly by more than one retriever rise to the top. Common for combining dense and sparse search in hybrid retrieval.

Rectified flow​

A flow-matching parameterization whose training paths are straight lines from noise to data, popular in 2024+ models like SD3 and Flux. Straight trajectories are easy to follow in a few big steps, so sampling needs fewer steps than the curvy paths of older diffusion. You can also "re-flow": after training once, use the model to generate (noise, image) pairs and retrain on those straight pairs, which straightens the paths even further and lets you sample in as few as one or two steps. Like replacing a winding mountain road between two towns with a straight highway — same destination, far fewer turns to take.

Recurrent network​

A recurrent neural network (RNN) is a network built for sequences — text, audio, a stream of video frames — that reads one item at a time and carries a running summary, its hidden state, forward from step to step. That looping of the hidden state back into the next step is the "recurrent" part: the same small network is applied again and again, and its memory of everything seen so far is what lets it act on context rather than on the current item alone. Analogy: reading a sentence word by word while keeping a mental note of what came before, so "it" near the end still refers to the right thing. Plain RNNs struggle to hold information over long sequences (the running note fades), which is why gated variants like the LSTM were invented. In RL, a recurrent network is the standard way to give an agent memory in a POMDP, where it must summarize past observations to recover the missing state.

ReduceScatter​

A collective communication operation in distributed computing where every worker (rank) starts with its own array of numbers. The operation first sums (or otherwise reduces) all the inputs across workers (similar to AllReduce), but instead of sending the full combined array back to everyone, it splits (scatters) the final summed array into equal parts and hands each worker just one chunk.

Analogy: Imagine four neighbors who each write a list of donations collected in their respective streets. They pool all their lists to sum the donations, and then divide the final sum list into four equal sections, giving each neighbor just one section of the master list to file.

Example: In FSDP training, after calculating the gradients, the GPUs perform a ReduceScatter. This sums the gradients across all workers and distributes them so that each GPU ends up holding and storing only its own sharded chunk of the final summed gradients, saving GPU memory.

Reference model​

A frozen copy of the starting model that RLHF and DPO measure against (through a KL term) so the model being trained does not drift too far from sensible behavior — a "before" photo to compare every change against.

Registers​

The fastest, smallest, and most immediate storage locations inside a Streaming Multiprocessor (SM)'s processor cores. Registers are local to each individual thread; they hold the variables, intermediate calculations, and memory pointers currently being operated on, with zero-cycle access latency. However, the register file per SM is fixed in size (typically 256 KB on modern NVIDIA GPUs), meaning that if a thread uses too many registers, the GPU will run fewer active threads concurrently, reducing occupancy.

  • Analogy: The immediate desktop space in front of you. If you need to write a letter, keeping the pen in your hand and the paper directly in front of you (registers) allows you to work without delay. If your desk is too small or cluttered, you must store items in a drawer (shared memory) or on a bookshelf in another room (slow global memory/HBM), which takes much longer to access.
  • Example: In a custom CUDA kernel, variables declared inside the thread function (such as loop counters or intermediate products) are stored in registers. If a kernel uses 64 registers per thread, a block of 256 threads requires 16,384 registers. If the threads exceed the physical register limit per SM, the warp scheduler must run fewer blocks at the same time, limiting the GPU's ability to hide memory latency.

Regression testing​

A software testing practice that checks whether recent changes or updates to a codebase have accidentally broken existing functionality, introduced bugs, or degraded system performance (known as "regressions"). In software and model development, regression testing involves running a standardized suite of tests on the new version and comparing the outcomes directly against a known historical baseline.

  • Analogy: Imagine tuning the brakes on a bicycle to make them tighter. Before taking it out on a busy road, you perform a quick safety checklist: do the wheels still spin freely? Does the chain still shift gears smoothly? Do the handlebars turn without rubbing? This checklist ensures that your brake adjustment did not accidentally degrade or break other parts of the bicycle that were already working fine.
  • Example: In robotics, when engineers update a control policy to make a robotic arm grasp objects more gently, they run the new policy through an evaluation harness across a suite of simulated tasks (such as picking up boxes, opening doors, and reaching targets). Comparing the success rates to a baseline ensures that the grasp improvement did not cause a regression in door-opening or overall stability.

REINFORCE​

The foundational on-policy policy-gradient algorithm. Instead of learning a value function to guess how good actions are, REINFORCE directly updates the policy's weights using the returns from complete rollouts. If an episode went well, it nudges the weights to make all the actions taken during that episode more likely; if it went poorly, it makes them less likely. As a policy-gradient method, it contrasts with value-based methods like DQN which choose actions by estimating their expected future rewards rather than directly optimizing action probabilities. Analogy: training a pet with treats—you do not explain the exact mechanics of a trick (a value function), you just reward a successful attempt (the return) so the pet is more likely to repeat that exact sequence of movements in the future.

Reinforcement learning​

A machine learning paradigm where an agent learns to make decisions by performing actions in an environment to maximize a cumulative reward. Unlike supervised learning (which teaches by correct examples), the agent learns via trial and error from rewards or penalties it receives. Think of training a dog: you reward the dog with a treat (positive reward) when it successfully fetches a ball (action), making the dog more likely to repeat that behavior in the future. Examples of reinforcement learning algorithms include Q-learning, DQN, SARSA, and PPO.

Rejection sampling​

A way to draw samples from a target distribution by proposing easy guesses and keeping or throwing away each one with just the right probability, so the survivors are distributed exactly as if they came from the hard distribution directly. The "right probability" of keeping a guess is min(1, p ÷ q), where p is how likely the target model thinks that token is and q is how likely the draft model thought it was. The rule is intuitive: if the target wants the token at least as much as the draft did (p ≥ q), always keep it; if the target wants it only half as much (p is half of q), keep it half the time and otherwise draw a replacement. For example, the draft proposes "cat" with q = 0.6 but the target only gives it p = 0.3, so you keep "cat" with probability 0.3 ÷ 0.6 = 0.5 — a coin flip — which exactly cancels the draft's over-eagerness for that word. In speculative decoding this is the step that lets a draft model's guesses be reused for random sampling without changing the target model's true output distribution.

ReLU​

Rectified Linear Unit — the most common and simplest activation function: it keeps positive numbers unchanged and turns every negative number into 0 (max(0, x)). Like a one-way valve that lets water through in one direction and blocks it in the other. That single sharp bend is enough to give a network its non-linear power, and because it is so cheap to compute it was the default for years; newer models often swap it for smoother curves like Swish or GELU.

Replay ratio​

How many gradient updates an agent performs per environment step — equivalently, how many times the average transition in the replay buffer gets reused. It is the dial that trades compute against fresh data. Turning it up makes an agent more sample-efficient (each hard-won transition is squeezed harder), which is what you want when the environment is the expensive part — a real robot, say. Push it too far and the network starts overfitting to a stale buffer and chasing its own bootstrapped targets, and learning gets less stable rather than faster. Classic DQN sits at a ratio of 0.25 (one update every four environment steps); the modern sample-efficient variants push it to 1 or beyond, and have to add machinery — periodically re-initializing the network, stronger weight decay — to survive up there.

Reparameterization trick​

A method to keep the training signal flowing through a random sampling step, enabling models like VAEs to be trained with ordinary backpropagation.

  • The Problem: Drawing the latent variable z directly from the encoder's distribution introduces randomness that blocks the flow of gradients.
  • The Solution: The trick separates the randomness by drawing plain noise ε from a fixed standard normal distribution (a bell curve). You then compute z = μ + σ · ε.
  • Why it Works: The randomness is now isolated in ε (which has no learnable parts). As a result, the network's μ and σ remain on a clean, differentiable path.
  • Analogy: It is like rolling one shared die outside the machine and then scaling the result, rather than building the dice into the machine itself.

Representation function (MuZero)​

The network inside MuZero that encodes the raw, high-dimensional observations (like game pixels or board coordinates) into a compressed, abstract latent state — written h(o) → s.

What it does: Instead of trying to remember every pixel or details that do not matter for winning, it translates the messy inputs into a neat vector containing only what is relevant to decision-making.

Analogy: A chess grandmaster looking at a board does not focus on the wood grain of the knights or the color of the table. In their mind, they translate the physical board into abstract concepts like "safe king" or "blocked center." The representation function is this mental translator.

Reranker​

A second-stage model that re-scores the top candidates from a fast first-stage retriever and reorders them by true relevance — usually a cross-encoder. The "retrieve then rerank" two-stage pattern is standard in search and RAG.

reshape​

Returns a tensor with a new shape, copying only when a no-copy view isn't possible

Residual connection​

A shortcut that adds a block's input straight onto its output — written output = x + f(x), where x is what went in and f(x) is what the block computed. Instead of each block having to rebuild the whole signal from scratch, the original x flows past it on an express lane and the block only contributes a small f(x) correction on top. Think of editing a draft: rather than rewriting the entire essay at every pass, each editor keeps the existing text and just marks up the few changes that improve it.

What does "adds a block's input to its output" actually buy you? Two big things:

  • An easy "do nothing" default. If a block has nothing useful to add, it can simply output near-zero, and x + 0 = x passes the input through unchanged. So adding more layers can never make things worse than the layers already learned — a new block starts from "leave it alone" and only departs from that when it finds something helpful. (This is exactly why AdaLN-Zero zero-initializes its gate: each block begins as a clean pass-through.)
  • A gradient highway. On the backward pass, the + x term hands every layer a direct path back to the earlier layers, so gradients don't shrink toward zero as they travel through many layers (the vanishing-gradients problem). That direct path is what makes very deep networks trainable at all — before residual connections, stacking 50+ layers usually trained worse, not better. It is the same skip-and-add logic found in convolutional nets, and it is what carries the residual stream through a transformer.

Residual parameterization​

A modeling trick used in deep hierarchical VAEs where each layer of latent variables is expressed as a small correction to what the previous layer already predicted, rather than as a full absolute value. Like a GPS giving "turn left in 200 m" instead of stating exact coordinates — each step describes only the gap from where you already are, so no single step has to carry the whole story. Because each latent group only needs to represent a tiny residual change, gradients flow smoothly through many stacked layers and very deep hierarchies become trainable. The idea borrows from residual connections in standard networks, applying the same skip-and-add logic to the latent variable structure itself.

Residual stream​

In a transformer, the running activation vector that flows through every layer via residual connections — each attention block and MLP block reads from this stream and adds its update back to it, without erasing what came before. Like a shared bulletin board that every department reads and pins notes to as it passes through the office: by the end of the building, the board carries the combined contribution of every team. Because every layer reads and writes the same vector space, the residual stream is the most natural place to look for interpretable features, which is why sparse autoencoders (SAEs) are usually trained on residual-stream activations.

ResNet​

Residual Network — a deep CNN whose layers each learn a small change to add to their input rather than a brand-new output, thanks to residual (skip) connections that route the input straight past each block. The name is short for "residual," the leftover the layer adds on top. Before ResNet, stacking many layers made networks harder to train because the signal degraded on its way through; letting each block default to "pass the input through unchanged, plus a tweak" means adding depth can only help. Like a relay of editors who each suggest small edits to a draft instead of rewriting it from scratch — the original text is never lost. ResNet-50 (50 layers) is still a common, sturdy baseline image encoder.

Return​

The total reward an agent collects from a moment onward, discounted so that sooner rewards count for more: G_t = r_t + γ r_{t+1} + γ² r_{t+2} + …, with each step shrunk by the discount factor γ (0 ≤ γ < 1). The geometric shrinking keeps the sum finite even over an endless task and bakes in "a reward now is worth more than the same reward later." The value function is just the expected return, so the return is the raw quantity that every value estimate — and every Monte Carlo average — is ultimately trying to predict.

Return-to-go​

The total reward still remaining from a given moment to the end of an episode — the future part of the return, R̂_t = r_t + r_{t+1} + … + r_T. Where the return is usually something to predict, return-to-go is used as a command: the Decision Transformer is fed the return-to-go it should still achieve as an input token and learns to output actions consistent with hitting that target, so cranking the number up at test time is how you ask the model for higher-reward behavior. Analogy for return-to-go: Imagine setting a GPS target for a road trip: "We want to arrive in exactly 3 hours." Instead of just predicting when you will arrive based on your current speed, this target time behaves as a command—it actively dictates how fast you must drive to make sure you arrive on time. In the same way, the return-to-go is fed into the model as a target command, steering its actions so that it collects exactly that amount of reward.

reverse-mode​

The order autograd walks the computation graph when differentiating: the forward pass first, then a single backward pass that propagates gradients from the scalar output back to every input. It is the efficient choice when a model has many parameters but only one loss value.

Reprojection error​

The gap, in pixels, between where a calibrated camera model predicts a known 3D point should appear and where it actually shows up in the image. You take a point whose 3D location is known (a checkerboard corner), project it through the pinhole camera model using your current estimate of the camera intrinsics, and measure how far that prediction lands from the real detected corner. Camera calibration works by tuning the camera parameters to make this error as small as possible across many points, and the leftover average (ideally well under one pixel) is the standard report card for how trustworthy the calibration is. Analogy: checking a tailored suit by marking where each seam should fall and measuring the millimeters it misses by — small misses mean a good fit.

Reward clipping​

Replacing every raw reward with its sign — +1 for any gain, −1 for any loss, 0 for none — before training. Across the Atari suite, one game awards single points and another awards hundreds; without clipping, the high-scoring games would dominate the gradient and each game would need its own learning rate. Clipping puts every game on the same scale so a single set of hyperparameters works for all of them. The cost is that the agent can no longer tell a small reward from a huge one — it learns that something is good, not how good — which is a poor fit for games where the size of a reward carries real strategy.

Reward function​

The part of an MDP that scores what happens, R(s, a): the immediate number the agent receives for taking action a in state s (sometimes also depending on the next state). It defines what the agent is trying to achieve — the whole goal of RL is to collect as much total reward as possible over time, weighted by the discount factor. Designing it is deceptively hard: an agent optimizes the reward you wrote, not the behavior you meant, which is the source of reward hacking. Example: in a gridworld you might give +1 for reaching the goal, −1 for stepping in a hazard, and a small −0.01 each step to encourage short paths.

Reward hacking​

When a policy maximizes the reward signal without doing what the reward was meant to encourage — it exploits the gap between the measurable proxy and the real goal. In RLHF it shows up when a model finds quirks a reward model scores highly but a human would reject — degenerate text, or simply padding answers longer (see length bias) — and given enough optimization pressure, any learned reward model can be gamed this way. Analogy: a student who studies the grading rubric instead of the subject, racking up points while learning nothing. The standard defenses are a KL penalty to a frozen reference model, which limits how far the policy can stray to chase an exploit, and — where answers can be checked programmatically — replacing the learned reward model with a deterministic verifier that cannot be fooled (RLVR).

Reward model​

A model trained on human preference comparisons to score how good a response is; it stands in for a human rater so RLHF can score millions of answers automatically. It is usually built from the SFT model with its next-token head replaced by a head that outputs a single scalar score, then trained on (prompt, chosen, rejected) triples with the Bradley-Terry loss — which only pushes the chosen response's score above the rejected one's, so it learns relative preference rather than any absolute notion of correctness. Because it is just an imperfect proxy for human judgment, optimizing too hard against it invites reward hacking; RLVR sidesteps the reward model entirely by using a verifier when answers can be checked programmatically.

Rigid body​

An object whose shape does not change as it moves — every pair of points on it stays the same distance apart no matter how the object is rotated or translated. Real objects are never perfectly rigid, but the assumption holds well enough for most robot links (a metal arm segment does not measurably bend under normal loads). A rigid body in 3D has exactly six degrees of freedom: three for position (where its center is) and three for orientation (how it is turned). Rigid-body mechanics gives us forward kinematics — each robot link treated as a rigid body chained to neighbors by joints — and dynamics algorithms like those in Pinocchio that compute forces, torques, and accelerations by treating the whole arm as a chain of rigid bodies connected by joints.

Right-sizing​

Choosing the smallest, cheapest model that still clears your quality bar for a task, instead of defaulting to the biggest one available. A well-trained 8B model often passes the same eval as a 70B at a fraction of the cost per million tokens — like hiring a capable specialist instead of an expensive all-rounder for a job that doesn't need one. Most production teams over-serve, so right-sizing is one of the easiest cost wins.

Ring attention​

A way to run attention over a very long sequence that is split across several GPUs (context parallelism): each GPU passes its slice of the keys and values to its neighbor around a circle, round after round, until every GPU has seen every other slice. Like people seated around a dinner table passing dishes one seat at a time so everyone eventually tastes every dish. This lets the GPUs handle a sequence far longer than any one of them could hold alone.

RISC-V​

An open-source, royalty-free computer processor design standard (Instruction Set Architecture, or ISA) based on the Reduced Instruction Set Computer (RISC) design philosophy.

  • Why it matters: Historically, chip designers had to pay expensive licensing fees to use proprietary instruction sets like x86 (Intel/AMD) or ARM. RISC-V is open and free for anyone to use, modify, and build. This has led to a boom in custom silicon design, particularly in AI, where companies can build highly specialized chips by adding custom AI-math instructions directly onto standard RISC-V processor cores.
  • How it works: RISC-V defines the "vocabulary" of instructions (like add, subtract, and load) that a processor can understand. Because it is based on RISC, it keeps this vocabulary as small and simple as possible. Complex actions are broken down into sequences of these simple instructions, allowing the processor to execute each step in a single clock cycle with minimal power.
  • Analogy: Imagine instructing a kitchen helper to make coffee.
    • A Complex Instruction Set Computer (CISC) (like x86) is like having a single complex command: make_coffee(). The helper needs a massive manual (complex hardware circuitry) to understand this command, and it takes many steps to execute.
    • A Reduced Instruction Set Computer (RISC) (like RISC-V) is like using a small set of simple commands: boil_water(), grind_beans(), pour_water(). To make coffee, you write out the simple sequence. Because the helper only needs to know a few basic commands, they can be extremely fast, simple, and energy-efficient.
  • Example: AI accelerators from Tenstorrent use grids of hundreds of tiny RISC-V cores. By utilizing RISC-V's open standard, Tenstorrent added custom instructions specifically designed to handle tensor math directly on the cores, making them highly efficient at running transformer models.

RLAIF​

Reinforcement Learning from AI Feedback — the same recipe as RLHF but the preference labels (or grades) are produced by another, stronger LLM following a written rubric instead of by paid human raters. Like swapping a panel of human judges for a single expert judge who works for free, never sleeps, and applies the same rules every time. Cheaper and faster than human labeling, often nearly as good on well-defined tasks, and the basis of Constitutional AI.

RLHF​

Reinforcement Learning from Human Feedback — the post-training recipe that aligns a language model with human taste in four stages: pretrain a base model, SFT it on demonstrations, train a reward model on human preference comparisons, then use RL (classically PPO) to push the policy toward higher reward-model scores while a KL penalty to a frozen reference model keeps it from drifting into nonsense. The key idea: people find it far easier to compare two answers than to write the perfect one, so RLHF learns from cheap "A is better than B" judgments instead of gold answers. Analogy: a cooking student who improves not from a fixed recipe but from a mentor repeatedly tasting two dishes and saying which is better. DPO and GRPO are later algorithms that reach the same alignment with fewer moving parts, and RLVR swaps the human-trained reward model for an automatic verifier when answers can be checked.

RLVR​

RL with Verifiable Rewards — RLHF without the learned reward model: when a task's answers can be checked by a program — a math answer matching the known result, code passing its unit tests, a proof a checker accepts — that verifier hands back an exact right/wrong reward, so there is nothing for the model to game (a learned reward model can be reward-hacked; a correct-or-not checker cannot). You then run an ordinary policy-gradient update — GRPO, PPO, or a REINFORCE-style method — on that signal. Analogy: studying with an answer key that instantly marks each attempt right or wrong, instead of paying a tutor to guess how good it looks. RLVR is the engine of the reasoning-model wave (OpenAI o1, DeepSeek-R1), where training on verifiable math and code drives models to write longer chains of thought.

RMSNorm​

Root-Mean-Square LayerNorm without mean-centering; the modern default

RND​

RND (Random Network Distillation) is a curiosity-style exploration method that measures novelty with two networks: a fixed, randomly-initialized target network and a predictor network trained to match the target's output on the states the agent actually visits — that is, the predictor is trained by distillation to mimic the random target. On states seen many times the predictor matches the target well (low error); on a never-seen state it has had no practice (high error), and that error is handed to the agent as an intrinsic reward. It is simpler than the ICM — there is no learned forward model of the dynamics, just a random function to copy — and it was the method that first solved Montezuma's Revenge. Like quizzing yourself with random flashcards: questions you have drilled feel easy, while one you have never seen catches you out — and "I got caught out" is exactly the signal that this is new territory worth exploring.

RNEA​

The Recursive Newton-Euler Algorithm — the standard fast way to compute a robot arm's inverse dynamics: given the joint positions, velocities, and desired accelerations, it returns the exact joint torques needed (the right-hand side of the manipulator equation). It works in two sweeps along the chain of links. The outward sweep starts at the base and propagates each link's velocity and acceleration down to the fingertip — every joint adds its own motion on top of the link before it. The inward sweep then starts at the fingertip and adds up the forces and torques link by link back to the base, applying Newton's and Euler's laws (force = mass × acceleration, and its rotational twin) at each one. Its cost is O(n), meaning the work grows only linearly with the number of joints n — twice the joints, twice the work, not four times — which is what makes it cheap enough to run in a fast control loop. Analogy: to find how hard each person in a tug-of-war line is pulling, you first walk down the line noting how fast each is moving, then walk back up tallying the forces each must add to keep the chain consistent.

RoCE (RDMA over Converged Ethernet)​

RDMA over Converged Ethernet — a network protocol that allows RDMA communication over standard Ethernet networks instead of requiring specialized InfiniBand (IB) cables and hardware. It enables high-throughput, low-latency direct memory transfer between machines in a cluster, bypassing the operating system kernel and CPU on both ends.

Analogy: Delivering a package directly to the recipient's desk inside an office building (RDMA) using the existing standard hallways and elevators (Ethernet) instead of building a dedicated, expensive high-speed pneumatic tube system (InfiniBand) throughout the building.

Example: Many modern hyperscaler cloud data centers deploy RoCE networks rather than InfiniBand to connect thousands of training GPUs, achieving near-InfiniBand latency and bandwidth while utilizing standard Ethernet networking switches and cabling.

ROCm​

Radeon Open Compute — AMD's open-source software platform and collection of drivers, development tools, and libraries designed for GPU-accelerated computing on AMD hardware. ROCm is the primary software stack used to build, train, and run machine learning models on AMD's MI300X and other Instinct accelerators, serving as AMD's open competitor to NVIDIA's proprietary CUDA platform. It includes libraries like rocBLAS and miopen that parallel NVIDIA's cuBLAS and cuDNN offerings.

  • Analogy: Imagine NVIDIA's CUDA is a proprietary operating system (like iOS) that only works on their own hardware, while ROCm is an open-source alternative (like Android) designed to run on a competitor's hardware. While it has similar features and apps, it requires developers to adjust their code slightly to fit the different system layout.
  • Example: Using ROCm and AMD-optimized PyTorch versions to compile a training script for execution on an AMD Instinct MI300X GPU node, ensuring the hardware's matrix cores are fully utilized.

Roll​

The rotation of a vehicle or object tilting from side to side (like a plane dipping one wing lower than the other).

  • Why it matters: In robotics and locomotion, controlling roll is essential for maintaining lateral stability and turning. If a walking robot rolls too far to one side, it will tip over.
  • How it works: Roll is one of the three Euler angles used to describe 3D orientation. It is rotation around the longitudinal (front-to-back) axis. In a quadrotor, roll is controlled by spinning the left-side motors faster and the right-side motors slower (to roll to the right), or vice-versa.
  • Analogy: Imagine tilting your head sideways to rest your ear on your shoulder (first to the left, then to the right). Your head is rolling.
  • Example: When a robot dog steps sideways or is pushed from the side, it must adjust its joint positions to correct its roll and keep its torso level with the ground.

RSSM​

Recurrent State-Space Model — the world model at the heart of the Dreamer family. It splits the latent state into two pieces that do different jobs. A deterministic part (h, carried by a recurrent network such as a GRU) reliably remembers the past: "the ball has been falling on the left for three frames." A stochastic part (z, a sample from a learned distribution) captures what could not have been predicted from the past and therefore has to be guessed: "the ball started in column 3." Neither half works alone — a purely deterministic model cannot represent genuine uncertainty and ends up predicting a blurred average of several possible futures, while a purely stochastic one forgets. Together: h remembers, z guesses. During training a posterior (which is allowed to see the current frame) and a prior (which is not) are both computed, and the KL between them teaches the prior to predict the future — which matters because during imagination the prior is all the model has.

Rollout​

In reinforcement learning, a rollout is a complete run or sequence of actions generated by the agent as it interacts with the environment from a starting point to an end point (often a full episode or a fixed number of steps). In the context of LLM reinforcement learning (like PPO or GRPO), a rollout is a single complete response generated by the model to a prompt. Analogy: Think of a rollout like a practice match or a rehearsal. Before a dancer performs, they practice the whole routine from start to finish. Each full run-through is a "rollout" where they can see what moves worked, where they stumbled, and how to improve. Example: If you are training an RL agent to play Super Mario, one rollout is the agent playing a level from the start until Mario either dies or reaches the flagpole. The sequence of all states visited, actions taken, and rewards received during this run is recorded and used to update the policy.

Rollout distribution​

The spread of responses a model is currently generating when it produces rollouts during RL training — what it tends to say and how varied those answers are. This distribution shifts as training proceeds, which is the whole point; but if it drifts toward weird, repetitive, or gamed outputs, that is a warning sign of reward hacking. Watching how it moves is like checking what a student actually writes on practice tests, not just their final score.

Roofline​

The Roofline model is a simple performance model that determines the maximum possible computational speed (throughput) of an algorithm on a specific hardware device (like a GPU). It defines a hard limit or "roof" on performance based on the hardware's architecture and the properties of the algorithm.

According to the model, an algorithm's performance is capped by the minimum of two hardware bottlenecks:

  1. Compute Limit: The hardware's maximum computational speed (peak FLOPS), representing how fast the processor cores can execute calculations when they are fully occupied.
  2. Memory Limit: The hardware's memory bandwidth multiplied by the algorithm's arithmetic intensity, representing how fast data can be loaded from main memory into the processor scaled by how many calculations are performed on each byte of data.

The performance ceiling is represented mathematically as: performance limit = min(peak FLOPS, memory bandwidth × arithmetic intensity)

Analogy: Imagine a busy kitchen preparing fruit salads.

  • Peak FLOPS (Compute Limit): The speed of the chefs. If you have extremely fast chefs, they can chop up to 10 fruits per second.
  • Memory Bandwidth: The speed of the helper bringing fruit from the pantry. If the helper can only carry 2 fruits per second, the kitchen's supply is limited.
  • Arithmetic Intensity: The complexity of the recipe.
    • Memory-bound scenario: If a recipe requires only 1 chop per fruit, the helper brings 2 fruits, and the chefs perform 2 chops per second. The chefs sit idle most of the time because they are waiting for fruits. The kitchen's throughput is limited by the helper (memory bandwidth).
    • Compute-bound scenario: If a recipe requires 10 complex decorative chops per fruit, the helper brings 2 fruits, and the chefs must perform 20 chops. However, the chefs chop as fast as they can and reach their maximum capacity of 10 chops per second (leaving fruits piled up). The kitchen's throughput is limited by the chefs (peak FLOPS).

When plotted on a graph, this performance ceiling resembles a slanted roof (the memory-limited phase) that bends and flattens out into a horizontal ceiling (the compute-limited phase) as arithmetic intensity increases.

RoPE​

Rotary Position Embedding — a way to tell a transformer where each token sits by physically rotating its query and key vectors by an angle proportional to the position, so the attention dot product between two tokens depends only on how far apart they are. Because the encoding lives in the rotation rather than an added vector, it extrapolates to longer sequences than the model trained on. 2D RoPE extends the trick to images: a patch token is rotated by its row and its column, encoding 2D spatial position. 3D RoPE adds a third axis — time — so a video token is rotated by its row, column, and frame index; this is the standard position encoding in DiT-based video models, and because the rotation extrapolates, it is what lets a model trained on short clips generate longer ones at variable resolution. Like giving every seat in a theater a precise angle on a dial, so the model can always work out the spacing between any two seats — and, for 3D RoPE, every seat across every showtime.

ROS / ROS 2​

Robot Operating System — robotics middleware (ROS 2 is the modern version)

Router model​

A small, cheap model that sits at the front of a serving stack and decides which model should answer each request — for example, sending an easy question to a fast 1B model and only escalating hard ones to a slow, expensive 70B model. Like a hospital triage nurse who handles simple cases on the spot and forwards the serious ones to a specialist, it saves money because most queries never need the biggest model.

RRT (Rapidly-exploring Random Tree)​

A single-query, sampling-based motion planning algorithm that builds a tree of paths starting from the robot's initial configuration. At each iteration, it samples a random point in C-space, finds the closest node in the existing tree, and extends the tree a small step toward the sample. If the step is collision-free, it adds a new node. Because it biases growth toward unexplored areas, it quickly spans the free space. Analogy: Imagine a tree root growing in dry soil. The root grows by sending out tiny shoots in random directions. When a shoot finds empty, fertile soil (unexplored free space), it grows further. If it hits a rock (obstacle), it stops. Over time, the root network branches out to cover the entire garden bed. Example: A robotic arm planning a path to reach inside a box. RRT samples random joint configurations, grows the tree around the box's edges, and eventually finds a collision-free path of joint angles that guides the gripper into the box.

RRT-Connect​

An efficient variant of the RRT algorithm designed for single-query motion planning. Instead of growing one tree from the start configuration, RRT-Connect grows two trees simultaneously: one starting from the start configuration and one from the goal configuration. In each step, one tree grows toward a random sample, and then the other tree attempts to grow directly toward the new node of the first tree to connect them. Analogy: Imagine two tunnel-boring machines trying to dig a tunnel through a mountain. Instead of one machine digging all the way from the west side to the east, one starts on the west side and one starts on the east side, digging toward each other until they meet in the middle. Example: A robotic manipulator planning a path to move an object from a shelf to a table. By growing trees from both the shelf (start) and the table (goal) configurations and trying to connect them, the planner finds a path much faster than growing a single tree from the shelf.

SAC​

Soft Actor-Critic (SAC) is an off-policy actor-critic reinforcement learning algorithm designed for continuous control (like setting the joint torques of a robotic arm). Its distinguishing feature is being built on maximum-entropy RL: instead of only maximizing the expected sum of rewards, the agent is also rewarded for acting as randomly as it can afford to. This prevents the policy from prematurely converging to a single sub-optimal strategy and encourages active exploration. It borrows two stabilizers from TD3 — twin critics and a tanh-squashed Gaussian policy trained with the reparameterization trick — and tunes the entropy weight with automatic temperature tuning so one configuration works across many tasks. Analogy: Imagine learning to paint. A traditional artist (standard RL) might find one style that sells reasonably well and paint only that forever to maximize profit. An artist trained with SAC (maximum entropy) wants to make money but also wants to explore as many different styles, colors, and techniques as possible. This exploration keeps their art diverse and creative, which might lead them to discover even more lucrative or stable artistic directions they would have otherwise missed. Example: When training a bipedal robot to walk, a standard RL agent might quickly settle on a stiff, unstable shuffle because it gets a small forward reward. A SAC-trained robot will try many different gaits, arm swings, and steps, eventually finding a smooth, highly robust walk that handles uneven terrain much better.

SAE​

Sparse Autoencoder — interpretability tool decomposing activations into monosemantic features

SAM (Segment Anything Model)​

A general-purpose segmentation model from Meta that, given a prompt as simple as a point or a box, outlines the exact pixels of whatever object sits there — without having been trained on that specific object category. It was trained on a billion masks, so it generalizes to almost any object, which makes it the standard "carve out this thing" tool in modern robot perception: a VLM decides which region a phrase like "the red cup" refers to, and SAM produces the precise pixel-level mask of it. Analogy: a smart lasso tool in photo software that selects a whole object from a single click, instead of you tracing its outline by hand.

Sample​

A single example in a dataset or batch — one sentence, one image, one prompt. If a batch is a carton of eggs, a sample is one egg. The word can confuse beginners because sampling in text generation means something else entirely (randomly drawing the next token); here it simply means "one item."

Sample efficiency​

How much an algorithm learns per unit of environment experience — how good a policy it reaches for each transition collected. It is the axis that most sharply separates the two camps of deep RL. Off-policy methods like SAC, TD3, and DQN store transitions in a replay buffer and reuse each one many times, so they are sample-efficient; on-policy methods like PPO must discard data after each update and so need far more samples, but make up for it with cheap, massively parallel simulation. Sample efficiency matters most when each real sample is expensive — a physical robot, a medical trial — and matters less when the simulator is fast and you care about wall-clock time instead. Analogy: two students studying the same exam — one (off-policy) re-reads each practice problem until it sticks; the other (on-policy) only ever glances at fresh problems once but can blaze through thousands an hour.

Sampler​

The component that decides the order in which a DataLoader visits dataset examples (e.g. random, sequential, or class-weighted).

Sandbox​

An isolated, throwaway environment — like a fenced-off playground — where an agent or program can run commands, create files, and make mistakes without affecting your real computer. If the agent breaks something inside the sandbox, you just throw the sandbox away; nothing outside it is touched. Containers (like Docker) and virtual machines are common ways to build one.

Sampling​

Drawing the next token from the model's predicted probability distribution instead of always taking the most likely one; temperature, top-k, and top-p control how random the choice is.

SARSA​

An on-policy temporal-difference control algorithm, named for the five pieces of experience it uses in one update — State, Action, Reward, next State, next Action (s, a, r, s′, a′). It nudges Q(s, a) toward r + γ Q(s′, a′), where a′ is the action the agent actually takes next under its current (usually ε-greedy) policy. That one choice — using the real next action rather than the greedy one — is what separates it from Q-learning and makes it on-policy: because it accounts for its own exploration, it learns safer, more conservative behavior in risky environments like Cliff Walking.

SayCan​

SayCan is a foundation-model-driven task planning framework that combines the high-level semantic reasoning of large language models (LLMs) with the physical capability of low-level robot policies.

  • Why it matters: LLMs are excellent at breaking down abstract instructions (like "clean up this spill") into logical steps, but they do not know what the robot is physically capable of doing in its current environment. SayCan solves this by letting the LLM propose actions while using feasibility scores to ensure the robot only attempts steps it can actually succeed at.
  • How it works: The LLM acts as the "Say" module, outputting the probability of different task-step options based on the instruction. The "Can" module consists of value functions trained via reinforcement learning for individual robotic primitives (like picking, placing, or opening a drawer); these functions output the probability of success for each primitive given the robot's current sensors. SayCan multiplies the "Say" and "Can" probabilities together, selecting the step that maximizes this combined score, executing it, and repeating the cycle.
  • Analogy: Imagine planning a vacation with a smart assistant (the LLM/"Say") and a travel agent (the value function/"Can"). The smart assistant suggests, "We should fly to Hawaii, or drive to the local park, or walk to the moon." The assistant thinks walking to the moon is a great adventure. However, the travel agent checks your budget and physical constraints, noting that walking to the moon has a 0% chance of success, while driving to the local park has a 100% chance. By multiplying the desirability of the trip by its feasibility, you decide to drive to the park.
  • Example: In a simulator kitchen, a user says, "I spilled my drink, can you help?" The LLM ("Say") proposes primitives like "find a sponge," "find a soda," or "vacuum the floor." The robot's camera shows there is a sponge on the counter but no vacuum cleaner. The value function ("Can") scores "find a sponge" highly and "find a vacuum" very low. SayCan combines these scores, leading the robot to fetch the sponge and wipe the spill.

Scale-and-shift​

A two-step tweak applied to a layer's activations: multiply every value by a learned scale and then add a learned shift — the operation y = scale × x + shift. It is exactly like the brightness and contrast sliders on a photo editor: scale stretches or squashes the range (contrast), and shift nudges everything up or down (brightness). The two numbers are usually the weights and biases a normalization layer learns; when they are instead predicted from a condition such as a class label, you get conditioning schemes like AdaGN, AdaIN, and AdaLN.

Scaling factor​

A multiplier used to map values from one numerical range or representation to another. In the context of quantization, a scaling factor is used to scale high-precision floating-point numbers (like float32 or float16) into the representable range of low-bit integers (like int8 or int4).

  • Analogy: Imagine trying to draw a map of a city on a single sheet of paper. You cannot draw it to actual size. Instead, you use a scaling factor (like "1 inch = 1 mile") to shrink the real-world distances down so they fit on the page, and write down the scale factor in the map's legend so anyone reading it can multiply the map measurements to find the real distances.
  • Example: If we quantize weights from float32 (which can represent values up to 3.4e38) to signed int8 (which can only represent integers from −128 to 127), and the maximum absolute weight is 4.0, we use a scaling factor of 127 / 4.0 = 31.75. We multiply all weights by 31.75 and round to the nearest integer. To dequantize back to floating-point during computation, we divide the int8 values by 31.75.

Scaling laws​

The empirical finding that a model's loss drops in a smooth, predictable curve as you add parameters, training data, and compute — like a growth chart that lets you forecast a bigger model's quality from smaller ones before you ever build it.

Scene detection​

Automatically finding the "cuts" in a video — the hard jumps where the footage switches from one shot to another — so a long video can be split into clean single-shot clips. It works by watching for a sudden, large change between two adjacent frames, measured by something like the difference in their color histograms (a tally of how many pixels fall into each color bucket) or in deep features. Analogy: flipping through a photo album and starting a new pile every time the picture suddenly looks completely different. Example: a 90-minute movie might be split into roughly 1,500 single-shot clips, each safe to use as a training example because the motion inside it is continuous rather than spanning an editing splice.

Scheduler​

The part of an inference server that decides, at every step, which requests to start, which to keep generating, and which to pause when memory runs low — like an air-traffic controller choosing which planes take off, keep flying, or circle, so the runway (the GPU) is always busy but never overloaded. A good scheduler is often worth more real-world throughput than any single clever kernel.

Score​

The gradient of the log-probability of the data with respect to the input, written ∇_x log p(x). It points in the direction that makes an image more likely under the data distribution — in plain terms, "which way should I nudge these pixels to make this look more like a real image?" Diffusion models implicitly learn this at every noise level, so generation becomes a matter of repeatedly stepping in the score's direction, from noise toward a realistic sample.

Score matching​

A way to train a generative model by teaching it the score — the gradient of log-density, "which way makes this more likely" — instead of the density itself, which avoids ever computing an intractable normalizing constant. The practical version, denoising score matching, sidesteps needing the true score: add a known amount of Gaussian noise to each training example and have the network predict the direction back to the clean point, which provably equals the score of the noised data. (A relative, sliced score matching, estimates it instead by checking random one-dimensional projections.) Once the score is learned, you generate by following it with Langevin dynamics. This is the lens that reveals diffusion models as score estimators trained at many noise levels.

Scratchpad​

A temporary, fast-access workspace where intermediate results are stashed so they don't have to be recomputed later. Like a math student's scratch paper next to an exam: jot the partial sums, look them up later, move on much faster than redoing each calculation. In serving, the KV cache is the model's scratchpad — every key and value it has already computed sits there ready to be reused on the next decode step.

SD3​

Stable Diffusion 3 — the 2024 release of Stable Diffusion from Stability AI that switched the architecture to an MMDiT transformer and trained it with rectified flow instead of the older U-Net-plus-DDPM recipe of earlier versions. Letting text and image tokens share the same attention layers, and feeding prompts through both CLIP and a large T5 text encoder, gave it noticeably better prompt-following and spelling than SD1.x/SDXL. Think of it as the bridge release that moved Stable Diffusion from the U-Net era into the modern transformer-and-flow era that Flux then built on.

SDE (stochastic differential equation)​

An equation describing how something evolves over time under both a predictable push (the "drift") and continuous random jitter (the "diffusion") — like the path of a pollen grain carried by a current while being constantly buffeted by water molecules. A diffusion model can be written as an SDE that gradually turns an image into noise; reversing that SDE turns noise back into an image. The reverse SDE has a deterministic twin with identical statistics, the probability flow ODE, and the two standard noising conventions are the VP and VE SDE families.

SDF (Signed Distance Field)​

A representation of 3D geometry where space is divided into a grid (or represented as a continuous function), and each point stores the signed distance to the boundary of the nearest obstacle. A positive value indicates the point is outside any obstacle, a negative value indicates it is inside, and zero represents the boundary surface. In motion planning (like CHOMP), the gradient of the SDF points directly away from the obstacle, providing a natural direction to push trajectories out of collision. Analogy: Imagine a heat map around a block of ice. Instead of temperature, the map shows how many steps you are away from the ice. Standing outside, it shows positive steps (e.g., +5 steps). If you could step inside the ice, it would show negative steps (e.g., -3 steps). By following the direction where the numbers increase fastest, you can walk away from the ice as quickly as possible. Example: A robot arm navigating around a table. A camera scans the table to build a 3D grid SDF. When planning a path, the collision checking algorithm queries the SDF at the robot's joint positions; if the value is negative or below a safety margin (e.g. +2 cm), the optimizer uses the SDF gradient to adjust the path away from the table.

SDXL Turbo​

A speed-tuned version of Stable Diffusion XL that produces a usable image in a single step (or just a few), instead of the usual 20–50. It was created with Adversarial Diffusion Distillation (ADD), which trains a fast "student" model under the eye of a GAN-style discriminator that rejects any quick output which doesn't look real — keeping the picture sharp despite the shortcut. Like a chef who learns to plate a dish in seconds because a tough critic tastes every rushed attempt. The trade-off: near-instant generation, with slightly less fine detail and variety than the slow original.

SE(3) / SO(3)​

The two mathematical groups that describe rigid motion in 3D. SO(3) (the special orthogonal group) is the set of all 3D rotations, represented as a 3×3 rotation matrix R: nine numbers whose columns are unit-length and mutually perpendicular, encoding three rotational degrees of freedom. SE(3) (the special Euclidean group) adds position, giving rigid-body poses — rotation and translation — usually packed into a 4×4 homogeneous transform T = [[R, p], [0, 1]], so that composing two motions is just matrix multiplication: T_AB · T_BC = T_AC. Analogy: SO(3) says how an object is turned; SE(3) says how it is turned and placed. The 4×4 form's bottom row of [0 0 0 1] is a bookkeeping trick that lets one matrix multiply handle both the rotate and the move-over steps at once. For the rotation part alone, the quaternion, axis-angle, and Euler angles representations are common alternatives.

Seed​

A fixed starting number for a random-number generator; setting the same seed makes random operations (shuffling, initialization, dropout) produce the identical sequence every run.

Segmentation map​

A picture that has been divided up so that every pixel is painted a flat color standing for what kind of thing it belongs to — all the "sky" pixels one color, all the "road" pixels another, all the "person" pixels a third. It is like a color-by-numbers outline of a scene: it throws away the photographic detail and keeps only a labeled map of which region is which. (Splitting an image into these labeled regions is called segmentation; the result is the segmentation map, sometimes a segmentation mask.) ControlNet can take one as a conditioning signal so a generated image places each kind of object exactly where its colored region sits — the prompt decides what a "building" looks like, but the map decides where the building goes.

Sentence embedding​

A single dense vector that captures the meaning of an entire sentence (or short passage), so two sentences about the same topic end up close together in vector space even if they use completely different words. Think of it as a GPS coordinate for meaning — two sentences that "mean the same thing" land near the same point on the map. Sentence embeddings are the backbone of semantic search in RAG: you embed the user's question and every stored passage, then find the passages whose coordinates are closest.

Self-consistency​

Sampling many independent chain-of-thought solutions to the same problem and taking a majority vote on the final answer — like asking several people to solve a puzzle on their own and trusting the answer most of them land on.

Self-distillation​

A twist on distillation where the "teacher" and the "student" are the same model instead of a big teacher and a smaller student — the network learns by trying to match its own output on a slightly different view of the same input. Like checking your work by solving a problem a second way and forcing the two answers to agree: there is no answer key and no smarter tutor, so the network teaches itself just by staying consistent. Concrete example: in DINOv2, a "teacher" copy (which is just a slowly-updated running average of the "student") looks at one crop of a photo while the student looks at a different crop, and the student is trained to reproduce the description the teacher gave — so the model learns features that stay the same when an object is moved or cropped, all with no human labels. Because the teacher is only a smoothed copy of the student, this is a form of self-supervised learning, and the slow averaging is what stops the network from cheating by collapsing to one constant answer for every image.

Self-Forcing​

A training method for autoregressive video models that closes the gap between how they are trained and how they actually run. Normally such a model is trained to predict the next frame given real past frames, but at generation time it must feed on its own earlier outputs, whose small errors compound into drift — a mismatch called exposure bias. Self-Forcing fixes this by making the model generate from its own predictions during training too ("forcing" it to rely on itself), so it learns to cope with its own mistakes. Combined with distillation, it is a current recipe (alongside CausVid) for real-time, long, streaming video.

Self-supervised​

Learning from raw, unlabeled data by inventing the labels from the data itself — for example hiding part of an input and asking the model to predict the missing piece, or asking whether two altered views came from the same original. No human annotation is needed, so the model can train on billions of images or sentences nobody had to tag. Like learning a language by covering up words in books you already own and guessing them, instead of paying a tutor to quiz you. This is how DINOv2 learns vision features and how the masked- and next-token objectives behind most LLMs work; contrast it with supervised training, which needs an answer key.

Sequence modeling​

A machine learning paradigm where the goal is to predict, generate, or classify ordered sequences of data — such as words in a sentence, frames in a video, or states and actions in a reinforcement learning trajectory. Instead of treating each data point as independent, a sequence model learns the rules of what comes next based on what happened before. In modern AI, sequence modeling is usually implemented using transformers trained as autoregressive models to predict the next token in the sequence. Analogy for sequence modeling: Imagine playing a game of telephone or writing a story sentence-by-sentence, where the next word must make sense given the whole history of the story so far. In offline RL, Decision Transformers and Trajectory Transformers reframe reinforcement learning as sequence modeling: instead of using value functions, they treat states, actions, and rewards as a sequence of tokens and learn to predict the next action just like a language model predicts the next word.

Setpoint​

The target value a controller is trying to make the system reach and hold — the "desired" that the measured "actual" is compared against. The difference between the setpoint and the current measurement is the error, and every feedback controller (PID, LQR, and the rest) is just a recipe for turning that error into a correcting command. A setpoint can be a single fixed value (hold this joint at 30°, keep the cabin at 21 °C) or a moving target that traces out a whole trajectory over time. Analogy: the temperature you dial on a thermostat — the furnace runs harder or eases off based purely on how far the room is from that chosen number.

SFT​

Supervised Fine-Tuning — the first post-training stage: take a base model that only continues text and fine-tune it on a dataset of (instruction, response) demonstrations so it learns to follow requests — the same idea as instruction tuning. It is plain supervised learning: each demonstrated response is the answer key, and a cross-entropy loss nudges the model to reproduce its tokens. Analogy: showing a fluent-but-rambling speaker thousands of worked question-and-answer examples until they answer the exact question asked. SFT alone yields a usable assistant, and its checkpoint becomes the frozen reference model that later RLHF, DPO, and GRPO stages measure drift against.

SGD​

Stochastic Gradient Descent — updates parameters by subtracting a scaled gradient computed on a mini-batch; the simplest optimizer and the basis for more advanced methods

Superposition​

The trick a neural network uses to store more features than it has dimensions: it packs many concepts into overlapping directions in activation space, accepting a little interference between them because most features are rare and seldom active at the same time. The consequence is polysemantic neurons — a single dimension that lights up for several unrelated things. Like a small office where each desk is shared by several people who rarely come in on the same day. Superposition is why interpretability is hard, and undoing it — recovering the packed features as separate directions — is exactly what a sparse autoencoder is built to do.

Surrogate objective​

A stand-in objective that is optimized instead of the thing you actually care about, because the real thing cannot be differentiated, sampled, or trusted far from the current policy. In policy-gradient RL the quantity of interest is the expected return of the new policy, but the only data available was collected by the old one — so what gets maximized is E[ π_new(a|s)/π_old(a|s) · A(s,a) ], the importance-weighted advantage.

  • Why it matters: This expression is a good local approximation of the true improvement, and a terrible global one: push the ratio far enough and the surrogate reports a huge gain from a policy that is in fact broken. Everything distinctive about TRPO and PPO is a device for keeping the optimizer inside the region where the surrogate can still be believed — a hard KL constraint in TRPO's case, a clipped ratio in PPO's.
  • Analogy: Estimating how a recipe will taste with twice the salt by tasting the current version and extrapolating. Fine for a pinch more; useless for a handful.

Supervised learning​

Training a model from labeled examples — pairs of (input, correct answer) — by nudging it to reproduce the answer for each input; "supervised" because every example comes with the right answer attached, like a teacher's answer key. It is the most basic learning setup and underlies behavior cloning, which simply makes the (state → action the demonstrator took) pairs the examples and trains the policy to copy them. Contrast it with reinforcement learning, where no answer key exists and the agent must discover good actions from reward signals, and with self-supervised learning, which invents its own labels from raw data. Analogy: studying with flashcards that have the answer on the back, versus figuring things out by trial and error.

sglang​

An open-source LLM serving runtime that pairs fast inference (via RadixAttention prefix sharing) with first-class constrained generation — built-in regex / JSON / grammar constraints applied at decode time. Plays a similar role to vLLM but is the popular pick when reliable structured output (function calls, tool use, schema-conformant JSON) matters most.

Shape​

The size of a tensor along each dimension; the tuple returned by .shape

Sharding​

Splitting a dataset (or model) into many smaller pieces so they can be stored, loaded, or processed in parallel.

Shared memory​

A fast, on-chip, user-managed cache memory on a Streaming Multiprocessor (SM) that is shared by all threads within a single CUDA block. Shared memory has much lower latency and higher bandwidth than global memory (HBM), but it is extremely limited in size (typically 100–230 KB per SM) and must be explicitly managed in the kernel code via thread synchronization (like __syncthreads()).

  • Analogy: A shared whiteboard in a study room. Instead of each student (thread) walking back and forth to the library (slow HBM memory) to read their own copy of a book, one student copies the key pages onto the whiteboard. All students in the room can now read from the whiteboard simultaneously, which is much faster.
  • Example: In a tiled matrix multiplication kernel, threads load a tile of inputs from HBM into shared memory once, synchronize to ensure the load is complete, and then read those values multiple times from shared memory to perform dot products. This reuse avoids redundant HBM access and speeds up execution.

Shot list​

A structured plan that breaks a story into an ordered sequence of individual shots, each with its own short description of what happens in it — exactly the storyboard a film director writes before shooting. In video generation a large language model can act as the "director," expanding a one-line prompt into such a list (often as JSON), after which each shot is generated separately and the clips are stitched together. This is the planning step of hierarchical generation: deciding what happens in what order before any frames are made is what lets a long video follow a sensible narrative instead of drifting aimlessly. Systems like VideoTetris and MovieDreamer build on this idea.

SigLIP​

Sigmoid-loss CLIP — a CLIP variant that swaps CLIP's batch-wide softmax contrastive loss for a sigmoid loss, which scores each image–text pair on its own as an independent yes/no match. Judging pairs one at a time means it trains well even with small batches, where CLIP needs very large ones to gather enough negatives to compare against. SigLIP 2 (2025) extends it with better data and multilingual training.

Sigmoid​

The S-shaped squashing function σ(x) = 1 / (1 + e^(−x)): it maps any real number to a value between 0 and 1, with 0 mapping to exactly 0.5. The name is just Greek for "S-shaped" (sigma + -oid, "like the letter S"), after the curve it draws. It is the standard way to turn an unbounded score into a probability — a score of 0 means "50/50", large positive means "almost surely yes" — which is why it appears wherever a model must make a yes/no or A-vs-B judgment, such as the Bradley-Terry preference loss used to train reward models (there, the probability that answer A beats answer B is the sigmoid of their score difference).

Sigmoid loss​

A training loss that scores each example with one simple, independent yes/no question — "should these two things match?" — instead of making examples compete against each other. It runs the model's raw match score through the sigmoid function, an S-shaped curve that squashes any number into a probability between 0 and 1 (very negative → near 0, very positive → near 1, zero → 0.5), then rewards the model when a true pair lands near 1 and a mismatched pair lands near 0. Like grading each true/false exam question on its own merits, instead of ranking every student in the room against one another — which is what softmax-based losses such as CLIP's InfoNCE do. How it is computed: for each pair take the label y (1 if they truly match, else 0) and the predicted probability p = sigmoid(score), then add up the cross-entropy of that single decision, −[y·log p + (1−y)·log(1−p)], independently over every pair. Because each pair is judged alone rather than against a whole batch, training still works with small batches, unlike softmax losses that need many examples per batch to compare against. This is the loss behind SigLIP.

SiLU​

Sigmoid Linear Unit — just another name for Swish, the activation x · σ(x). The two words mean the exact same function: you will see "SiLU" in code (PyTorch's nn.SiLU) and "Swish" in papers.

SIMD​

Single Instruction Multiple Data — a computer architecture execution model where a single instruction operates on multiple data points simultaneously using wide vector registers (such as AVX-512 on CPUs or NEON on ARM). Analogy: A fitness instructor leading an aerobics class. The instructor calls out a single instruction ("Raise your left arm!"), and all fifty participants in the room perform that exact action at the same moment on their own bodies, processing fifty data points (arms) with one command.

SIMT​

Single Instruction Multiple Threads — an execution model used in GPUs where a single instruction is executed across multiple independent threads (a warp of 32 threads). Unlike SIMD where the programmer must think about vector registers and packing data explicitly, SIMT allows writing code for a single thread, while the hardware handles executing it across parallel threads and masking out inactive ones during divergence. Analogy: A school classroom where thirty students are taking the same exam. The teacher reads out a single instruction ("Answer question 3 on page 5"). Every student follows that instruction, but they write their answers independently. If some students finish early, they sit quietly (masked off) while the others catch up.

SmoothQuant​

An accurate and efficient post-training quantization (PTQ) method that enables both weights and activations of LLMs to be quantized to int8 precision.

  • Why it matters: While quantizing only weights is relatively easy, quantizing activations to int8 is difficult because LLM activations contain "outliers"—extremely large values in specific channels that span a wide range. Quantizing these wide-range activations uniformly to int8 squeezes the normal-range values into too few integer bins, ruining model accuracy. SmoothQuant solves this by mathematically sharing the quantization difficulty: it divides the activations by a smoothing scale factor to shrink the outliers, and multiplies the weights by the same factor to keep the overall mathematical output of the layer unchanged. This transfers the "outlier difficulty" from the hard-to-quantize activations to the easier-to-quantize weights.
  • Analogy: Imagine two dancers performing a lift. One dancer (the activations) has to make a sudden, high jump (an outlier spike), which is very difficult to do smoothly. SmoothQuant is like adjusting the choreography so that the other dancer (the weights) bends down (multiplied by the scale factor) while the jumping dancer does a much lower, smoother hop (divided by the scale factor). The overall lift height (matrix multiplication result) stays exactly the same, but it is much easier for both to perform.
  • Example: Using SmoothQuant to serve an LLM allows both weights and activations to run in int8 matrix multiplication, accelerating inference throughput on GPUs that support integer Tensor Cores.

Sim-to-real​

The process of transferring robotic control policies or models trained in simulation (sim) to physical hardware (real). Because simulators cannot perfectly replicate real-world physics, friction, contact dynamics, or sensor noise (a gap known as the reality gap), direct transfer often fails, requiring techniques like domain randomization or domain adaptation to ensure the policy generalizes to physical hardware.

  • Analogy: Imagine playing a flight simulator game using a keyboard for 100 hours. If you are suddenly placed inside a real airplane cockpit, the real controls will feel slightly different, and there will be wind gusts and physical vibrations you didn't experience. Sim-to-real techniques prepare you (or the robot) for those differences so you don't crash.
  • Example: Training an in-hand manipulation policy in MuJoCo with randomized friction coefficients, mass properties, and link dimensions, so that the policy is robust enough to run successfully on a real robotic hand.

Skill discovery​

Learning a set of reusable, distinct behaviors ("skills") before and independently of any specific task reward — a form of unsupervised reinforcement learning. Each skill is a policy (or one policy conditioned on a skill code) that reliably produces its own recognizable behavior, and the learned set acts as a vocabulary of moves a later, reward-driven controller can pick from instead of starting from scratch. The best-known method is DIAYN, which makes skills distinct by maximizing the mutual information between the skill code and the states it visits. Like a dancer drilling a handful of clearly different steps on their own, so that when the music finally plays they can string the steps together rather than inventing motion from nothing.

Skip-and-add logic​

A design pattern in neural networks where a signal bypasses a layer unchanged and is then added back to the layer's output. Think of it like a chef tasting a soup that already has a good base flavor (the "skip" part, where the main base is kept), and deciding to just stir in a pinch of salt (the "add" part) to improve it, rather than throwing the soup out and cooking a new one from scratch. Because the main signal flows straight through, the layer only has to figure out the small correction (the residual) needed to make it better. This keeps information flowing easily in very deep networks.

SLA​

Service Level Agreement — a formal, often contractual promise a service provider makes to its customers about how well the service will perform, plus what happens (refunds, credits, penalties) if that promise is broken. For an LLM API, an SLA might say "99.9% of requests will succeed and 95% will start responding within 500 ms; if we miss that, you get a partial refund."

  • Why it matters: When you serve a model, you cannot just chase the highest possible throughput — every user is also waiting on a response, so there is usually a hard ceiling on acceptable latency (for example, TTFT must stay under 500 ms). That ceiling is the "SLA budget," and it caps how large you can grow the batch of requests served together before individual requests get too slow.
  • Analogy: Think of a pizza shop that advertises "delivery in 30 minutes or it's free." That public guarantee is the SLA. The shop can save money by batching many orders into one delivery trip, but if batching makes any single order arrive late, it has to give that pizza away — so the 30-minute promise sets a hard limit on how much it can batch.
  • How it relates to nearby terms: The SLA is the high-level customer-facing contract. The specific numeric target engineers design toward is the SLO (Service Level Objective), the actual measured number is the SLI (Service Level Indicator), and the amount of allowed failure before the SLA is at risk is the error budget.

SLAM​

Simultaneous Localization and Mapping (SLAM) is the process where a robot builds a map of an unfamiliar environment while tracking its own position within that map at the same time.

  • Why it matters: A robot cannot navigate a new space without knowing where it is (localization) and what the space looks like (mapping). However, to build a map it needs to know where it is, and to know where it is it needs a map. SLAM solves this chicken-and-egg problem by doing both concurrently.
  • How it works: As the robot moves, it uses sensors (like cameras, LiDAR, or IMUs) to track key landmarks in the environment. It estimates its motion from these landmarks while simultaneously updating the landmark positions on its map.
  • Analogy: Imagine walking into a dark, unfamiliar maze with a flashlight. As you walk around, you note landmarks (like a statue or a distinct door) to sketch a map on a notepad. By looking back at those landmarks, you also keep track of where you are in the maze.
  • Example: Vacuum cleaning robots use SLAM to map out your house so they don't miss spots or bump into the same walls repeatedly, and self-driving cars use it to navigate city streets in real time.

SLI​

Service Level Indicator — the actual measured number for how well a service is doing, such as the real percentage of requests that succeeded or answered within 500 ms. The SLO is the target; the SLI is the measurement you compare against it — like the speedometer reading (SLI) versus the posted speed limit (SLO).

Sliding-window generation​

A way to make a video longer than a model's trained clip length without retraining it: generate a series of short clips that overlap by a few frames, then blend the overlapping frames so the joins are seamless. Sliding a fixed-size window forward a little at a time is where the name comes from. Its weakness is long-range coherence — the only information tying distant parts together is whatever the small overlap can carry forward, so the scene slowly drifts, with colors and details creeping away from the opening. It is the cheapest long-form trick because it wraps any existing text-to-video model; FreeNoise and Gen-L-Video are named methods that refine how the overlaps are blended (for example, reusing a shared noise pattern so neighboring windows stay consistent).

SLO​

Service Level Objective — a specific, measurable promise about how a service should perform, such as "p95 TTFT under 500 ms" or "99.9% of requests succeed." It is the target you design toward and get alerted on, like a delivery company promising most parcels arrive within two days. The measured reality you check it against is the SLI, and the slack it allows for failure is the error budget.

SM​

Streaming Multiprocessor — the real worker unit inside an NVIDIA GPU. A single GPU isn't one giant brain; it's a collection of dozens (sometimes well over a hundred) of these SMs, and each SM packs together many CUDA cores, a few specialized Tensor Cores, a slice of fast on-chip memory, and a scheduler that decides what to run next. When you launch GPU work, it gets chopped into thousands of threads; those threads are distributed across the SMs, and each SM runs them in lockstep bundles of 32 called warps (the SIMT model). Within an SM the 32 threads of a warp are each executed by a CUDA core, so one warp instruction lights up 32 CUDA cores at once — and the moment a warp stalls waiting on memory, the SM instantly switches to another ready warp to keep its cores busy (which is why high occupancy hides memory latency). Analogy: If the whole GPU is a giant factory, an SM is one self-contained workshop on the floor. It has its own crew of ordinary workers (CUDA cores), a couple of heavy-duty specialist machines (Tensor Cores), its own supply shelf (on-chip memory), and a foreman (the scheduler) who keeps everyone productive by handing out the next batch of work the instant one crew gets blocked. The GPU's enormous speed comes from running many of these workshops at the same time.

SMILES​

Simplified Molecular Input Line Entry System (SMILES) is a chemical notation system that represents the 3D structures of chemical molecules as simple, one-dimensional text strings of letters and symbols.

  • Why it matters: Molecules are complex 3D structures, but computers process text best. SMILES allows chemists and AI models to represent chemicals like aspirin (CC(=O)Oc1ccccc1C(=O)O) as simple strings. However, standard language model tokenizers (trained on English text) do not recognize chemistry. They split SMILES strings into individual characters (like C, =, (, O, )), resulting in extremely long token sequences. Mining common SMILES fragments and adding them as custom tokens to the vocabulary compresses these representations, making chemical AI models faster and cheaper to run.
  • How it works: A SMILES string uses standard rules to translate a molecule's structure into a line of text: atoms are letters (like C for carbon, O for oxygen), double bonds are =, rings are closed with matching numbers (like 1 or 2), and branches are enclosed in parentheses (). In custom vocabulary extension, a programmer counts the most frequent multi-character substrings across a drug database and registers them as new single tokens (e.g., merging C, (, =, and O into a single C(=O) token).
  • Analogy: Imagine trying to write out a recipe, but your keyboard only has letters on it, not whole words. To write "tablespoon of sugar", you have to type out t-a-b-l-e-s-p-o-o-n-o-f-s-u-g-a-r (19 keystrores). If you upgrade your keyboard to have single keys for "tablespoon" and "sugar", you can type the same recipe in just a few strokes. Standard GPT-2 is like the letter-by-letter keyboard for chemistry; extending the vocabulary with SMILES fragments adds "tablespoon" and "sugar" keys.
  • Example: The SMILES string for aspirin is CC(=O)Oc1ccccc1C(=O)O. Using GPT-2's standard tokenizer, it takes 18 separate tokens because it is processed character-by-character. After adding 256 common SMILES fragments to the tokenizer and resizing the model's embedding matrix, the same molecule is encoded in only 4 tokens—a 4.5× compression boost.

Social navigation​

The ability of an autonomous robot to navigate through environments populated by humans while respecting human social norms, comfort, and personal space.

  • Why it matters: Standard collision avoidance treats moving humans as generic round obstacles and plans the shortest geometric path around them. This can cause robots to cut too close to people, block their path, or startle them. Social navigation ensures robots behave in a socially acceptable and predictable way.
  • How it works: It uses predictive collision avoidance where the robot predicts human walking trajectories (often using deep learning models or social force models) and plans its own path to maintain a safe "social comfort zone" (often implemented as dynamic costmap inflation layers that expand in front of moving humans).
  • Analogy: Walking down a busy sidewalk. You don't just dodge people like static signposts; you read their body language, predict which way they are walking, and step to the side early so both of you can pass each other comfortably without doing a "sidewalk dance."

Soft gate​

A multiplier that scales each value by some amount between fully off (0) and fully on (1), instead of the hard either/or of a switch that is only ever 0 or 1. Picture a dimmer knob rather than a light switch: a hard gate can only block a signal or let it through untouched, but a soft gate can pass 0.3 of it, or 0.8, dialing each value partly up or down. In a SwiGLU layer the gate amounts come from squeezing one projection of the input through a smooth activation function like Swish, whose output slides continuously rather than snapping between two settings — and that smoothness is what lets the network learn the right gate values from clean gradients.

softmax​

The function that turns a vector of scores into a probability distribution — each value squeezed into 0–1, and all of them summing to 1; the core of attention and classification heads. The name means a soft version of max: instead of the hard "winner takes all" of argmax, which hands the single biggest score 100% and the rest nothing, softmax gives most of the weight to the biggest score while still leaving a little for the others. That smoothness — a dimmer switch rather than an on/off toggle — is what lets the model be trained by gradients.

Sora​

OpenAI's text-to-video (T2V) model (2024), the release that made "DiT over 3D VAE latents, trained with flow matching" the default blueprint for a frontier video generator. Its two headline ideas were treating video as a long sequence of spatiotemporal patches (so the same model handles images and clips of different lengths) and variable resolution (generating many sizes and aspect ratios from one model). Sora 2 (2025) added synchronized audio and stronger physical consistency. OpenAI never released weights, so open replicas like OpenSora reconstruct the recipe from its technical report. Think of Sora as the proof-of-concept that reset everyone's expectations for what a single video model could do.

Sparse reward​

A task where the reward is almost always zero and only becomes non-zero on rare success — reaching a goal, winning a game, solving a puzzle — as opposed to a dense reward that gives continuous feedback every step. Sparse rewards are where exploration gets hard: with no gradient of hints to follow, an agent using only ε-greedy can wander at random for an astronomically long time before stumbling onto the first payoff, which is why intrinsic-motivation bonuses exist. Like searching a dark warehouse for one light switch: until you find it there is no "warmer / colder" signal telling you whether you are getting closer.

Spatiotemporal attention​

An attention pattern for video in which every token attends to every other token across both space and time at once — all positions in all frames mixed in a single shared attention operation. This is the most expressive way to model motion, because it can directly relate any pixel in any frame to any other, but its cost grows quadratically with the total number of tokens T×H×W, so it becomes very expensive as clips get longer or larger. Contrast (2+1)D, which splits spatial and temporal attention into two cheaper separate steps, and windowed attention, which restricts the joint attention to small local 3D windows. Sora-class models can afford full spatiotemporal attention only because a 3D VAE first shrinks T×H×W aggressively before attention ever runs — moving the expense out of attention and into the compressor.

Spatiotemporal patches​

The 3D version of image patches: instead of cutting one frame into flat 2D squares, a video is cut into little boxes that span a small image region and a few consecutive frames, so each box (also called a tubelet) captures appearance and motion at once. Each box becomes one token for a transformer, so movement is baked into the input from the start rather than reconstructed later from separate frames; TubeViT is a model built this way. Like cutting a flip-book into small columns that each go down through several pages — one cut shows how that corner of the picture changes over time. Example: a 16-frame clip cut into 2×16×16 patches (2 frames deep, 16×16 pixels wide) becomes a sequence of motion-aware tokens. The trade-off is that 3D boxes multiply the token count fast, raising compute — contrast plain patchification, which slices a single still image.

Special tokens​

Reserved vocabulary entries that mark structure rather than text — e.g. <bos>, <eos>, <pad>, and chat-boundary tokens like <|im_start|>

Speculative decoding​

A trick to make decode faster for free: a small, fast "draft" model guesses the next few tokens, and the big "target" model checks all of them in a single parallel pass, keeping every guess that matches what it would have produced and discarding the rest. Like an editor who reads a sentence a junior writer drafted and approves the part that is already correct rather than writing every word from scratch — the answer is identical to what the target alone would say, just reached in fewer slow steps. It works because decode is starved for memory bandwidth, so the GPU has spare compute to verify several guesses at once.

SRAM​

Static Random-Access Memory — a type of fast, volatile memory built directly into the silicon of a processor (such as a GPU or TPU). Unlike off-chip dynamic memory (HBM), SRAM does not need to be constantly refreshed and can be accessed with very low latency (typically 10–30 cycles). On a GPU, SRAM is physically used to implement shared memory and L1 caches. However, it requires 6 transistors per cell, making it physically large, power-hungry, and expensive, which is why GPU SRAM capacity is limited to a few hundred kilobytes per SM.

  • Analogy: Sticky notes stuck directly onto your desk. If you need to write down a quick number, writing it on a sticky note (SRAM) is instant and right in front of you. But you only have room for a few sticky notes on your desk, so if you have massive amounts of data, you must write it in a notebook (HBM) stored in a drawer.
  • Example: In FlashAttention, the key optimization is keeping the intermediate attention matrix tiles entirely within the SM's SRAM (shared memory/L1 cache) during computation. This prevents the large T × T attention grid from being written to and read back from the much slower HBM, boosting overall execution speed.

Stable Diffusion​

The best-known open-source diffusion model for turning a text prompt into an image (first released by Stability AI in 2022). Its key trick is to do the slow denoising work in a small compressed space (the latent space of a VAE) rather than on full-size pixels — like sketching a scene as a rough thumbnail first and only blowing it up to full resolution at the very end — which makes it light enough to run on a single consumer GPU. Because the weights were released publicly, it sparked a huge ecosystem of fine-tunes and add-ons such as LoRA, ControlNet, and DreamBooth.

Stable Video Diffusion (SVD)​

Stability AI's open-weights image-to-video (I2V) model (2023), the canonical baseline for turning a single still image into a short clip. It is built by temporal inflation: it freezes a pretrained Stable Diffusion image model and adds new time-aware layers that learn motion, so it keeps Stable Diffusion's strong sense of appearance and only has to learn how things move. Released in two variants — one tuned to generate 14 frames, one for 25 — it conditions on the input image (not text), which makes it the easiest strong model to run for hands-on I2V experiments. It also exposes a motion score input to control how much movement the clip contains.

State​

A complete, self-contained description of the environment at a single moment in time. In reinforcement learning, it holds all the information the agent needs to make an optimal decision, meaning that no history of past events adds any extra predictive power (this is the Markov property). State vs. Observation: A true state is a perfect, god-eye view of the entire system. In contrast, an observation (which can be linked to POMDP) is only the subset of information that the agent can actually see or measure. For example, if you are playing poker, the state includes all cards in the deck, your opponents' cards, and their chips. An observation is just your own cards and the chips on the table. When an agent cannot observe the full state directly, the problem is a POMDP. Analogy: Imagine playing a board game like Chess. The state is the exact arrangement of all pieces on the board right now. You don't need to know how the players moved their pieces to reach this point; you can make your next move looking only at the current board.

State dict​

A Python OrderedDict that maps every parameter and buffer name to its tensor value; the standard format for saving, loading, and transplanting PyTorch model weights

State-space representation​

A standard way to write down how a system evolves over time using two compact matrix equations: ẋ = Ax + Bu (how the state x changes given the state itself and the control input u) and y = Cx + Du (how the measured output y relates to the state). The state x is the minimal list of numbers that fully captures the system's situation at an instant — for a cart-pole, its position, velocity, angle, and angular rate. The matrix A says how the system drifts on its own, B says how your controls push it, and C says what your sensors can see. Real robots are nonlinear, but you can linearize them about a setpoint to get an A and B that hold nearby — which is exactly the form LQR and the Kalman filter are built to work with. Analogy: it is the system's "dashboard plus rulebook" — the dashboard (x) shows everything that matters right now, and the rulebook (A, B) says how today's reading plus your inputs become tomorrow's.

Static quantization (PTQ)​

A quantization method that converts both weights and activations to int8 before serving, using a calibration pass to fix the activation scales in advance.

STFT​

Short-Time Fourier Transform — a way to find which frequencies are present and when in a signal by chopping it into many short, overlapping windows (say 25 ms each) and running a Fourier transform on each one separately. A plain Fourier transform tells you the frequencies in a whole clip but loses all sense of when they happened; the STFT trades a little frequency precision for time precision by asking the question over and over on tiny slices. The output is a grid of (time × frequency) magnitudes — the raw material a mel spectrogram then refines. Like tapping out a song's rhythm window by window instead of blending the whole piece into one average chord.

Step response​

How a controlled system reacts when its setpoint is suddenly jumped from one value to another — the single most useful diagnostic plot in control. You command an abrupt step (e.g. "go to 30° now") and watch the measured value over time; its shape reveals everything about your tuning. Key features to read off: rise time (how fast it gets near the target), overshoot (how far it sails past before settling — a sign of too little damping), settling time (how long until it stops wobbling), and steady-state error (any permanent gap that remains). A sluggish controller crawls up with no overshoot; a too-aggressive one shoots past and rings; a well-tuned one rises briskly and settles with little or no overshoot. Analogy: flick a hanging door open and watch — does it glide to rest, slam past and bounce, or creep shut? That motion is the door's step response, and a PID controller's gains are tuned by reading exactly these features.

Stiction​

Short for static friction — the initial resistance that must be overcome before two touching surfaces will start to slide at all, larger than the friction once they are already moving. In a robot joint it means the motor can apply a small torque and nothing happens until the command crosses a threshold, at which point the joint suddenly breaks free and lurches. This dead band makes fine, slow positioning jerky and is a major reason real arms feel sloppier than their simulations, which usually model only smooth speed-dependent drag. It is the static piece that friction compensation measures and cancels. Analogy: shoving a heavy box across a floor — you push harder and harder against nothing, then it abruptly gives and slides, and keeping it moving takes noticeably less effort than starting it did.

Stereo vision​

Recovering depth from two cameras mounted a known distance apart (the baseline), exactly as two human eyes do. The same scene point projects to slightly different pixel positions in the two images; that shift is the disparity, and turning it into distance by triangulation yields a per-pixel depth map. The catch is that depth precision falls off with the square of distance — doubling the range quarters the accuracy — because a far-away object produces a vanishingly small disparity that the pixels can no longer resolve. Analogy: hold a finger at arm's length and blink each eye in turn; the closer it is, the more it jumps between the two views, and that jump is the depth cue stereo vision reads.

Stitching​

The ability of an offline RL algorithm to combine the good parts of several mediocre recorded episodes into a single policy better than any one of them — producing behavior that appears nowhere in the dataset. Suppose the data contains one trip from A to B and a separate trip from B to C, but nobody ever travelled A to C. A method that learns values can still discover the A→C route, because the Bellman equation propagates the value of C backwards through B and into A — it reasons about the pieces, not the trips. Behavior cloning cannot do this by construction: its whole objective is "produce the actions the data produced," so it can only ever reproduce trips someone actually took. Stitching is therefore the concrete reason to pay for the extra machinery of CQL or IQL instead of just cloning, and it is why the gap between offline RL and cloning is widest on random and medium data (many mediocre fragments to recombine) and narrowest on expert data (the trips are already optimal, so there is nothing left to improve by recombining them). How you know it happened: the learned policy scores higher than the best single episode in its own dataset. Copying cannot exceed what was copied, so anything above that line was never demonstrated to it.

Stochasticity​

Randomness, uncertainty, or probabilistic behavior in a system, where the next state is not completely determined by the current state and action. In reinforcement learning, a stochastic environment is one where taking the same action in the same state can lead to different outcomes (e.g., transition probabilities are between 0 and 1, rather than being strictly 0 or 1). Analogy: Rolling a die or flipping a coin is stochastic: even if you follow the exact same motion (the action), the result is random. In contrast, a game like Chess is deterministic because moving a piece to a square always results in that exact board layout. Why it matters: Stochasticity makes learning and planning much harder for agents. In curiosity-driven exploration, pure stochasticity can trap naive agents because they confuse unpredictable, unlearnable noise (like random static in the noisy-TV problem) with genuine novelty.

STOMP​

Stochastic Trajectory Optimization for Motion Planning (STOMP) is a gradient-free, stochastic trajectory optimization algorithm. Unlike CHOMP, which requires computing the gradients of the collision cost (which can be difficult or noisy for complex obstacles), STOMP generates paths by sampling random variations (noisy perturbations) around a candidate trajectory, evaluating their costs, and combining the lowest-cost variations to update the trajectory. Analogy: Imagine trying to walk down a mountain in a thick fog. Since you cannot see the slope (cannot compute the gradient), you take a few tentative steps in different random directions (sampling perturbations). You then step toward the direction that felt the most downward, repeating this until you reach the bottom. Example: Planning a trajectory for a humanoid robot walking through a cluttered room. STOMP generates random variations of the joint trajectory, evaluates how close they get to obstacles, and combines the best trajectories to produce a collision-free motion path.

Stop-string​

A user-supplied substring that tells the server "as soon as the generated text contains this, stop." Matched on the decoded text, not the raw token IDs, because the same letters can land in different BPE tokens depending on what came before — so the matcher has to keep a small rolling window of recent output and check for the string at every step.

Storage​

The 1-D buffer that a tensor is a view into

Symlog​

A squashing function, symlog(x) = sign(x) · log(1 + |x|), that shrinks large values while leaving small ones almost untouched (and, unlike a plain log, handles negatives and zero). DreamerV3 trains its reward and value heads to predict symlog(target) instead of the target. The reason is the whole point of DreamerV3: one set of hyperparameters has to work on a game paying rewards of 0.01 and a game paying 10,000. Under a plain squared-error loss the second game's gradients would be millions of times larger, and any learning rate that survived one game would explode or stall on the other. Symlog compresses that scale difference away, so a single learning rate fits both. Its inverse, symexp, converts predictions back to real reward units.

Straight-through estimator​

A trick for training through a step that has no usable gradient — such as the nearest-codebook-entry lookup in a VQ-VAE. On the forward pass the hard, non-differentiable operation runs as usual; on the backward pass the model simply pretends that step was the identity and passes the gradient straight through unchanged. It is like sketching along a ruler and then erasing the ruler's marks: the rough step shapes the result, but learning flows as if it were never there.

Streaming​

Sending the model's reply to the client one piece at a time as it is generated, instead of waiting for the whole answer and then returning it in a single response. Over HTTP this is usually done with Server-Sent Events (SSE) or chunked transfer encoding; the connection stays open and the server flushes each new token as soon as it is sampled. Like a waiter who brings each course out as it leaves the kitchen rather than holding the whole meal until dessert is ready — the user sees TTFT drop dramatically even though total generation time is the same.

Streaming video generation​

Producing a video frame-by-frame (or chunk-by-chunk) and emitting each piece as soon as it is ready, conditioning every new chunk on the ones already made — instead of computing the whole clip before showing anything. This is what makes real-time and open-ended (potentially infinite) video possible: you see frames immediately and generation can run as long as you keep asking. It typically reuses a KV cache so each new chunk does not recompute the attention over all earlier frames, and pairs with distillation into few-step models for speed. CausVid, Self-Forcing, and StreamingT2V are examples. It is the video analog of how a chatbot streams words out one at a time rather than waiting for the full answer.

Stride​

The number of storage elements that must be skipped in the underlying flat 1-D memory array to move by one element along a specific dimension of a tensor.

  • Why it matters: Computers store all data in a single, continuous flat line of memory. However, neural networks require multi-dimensional arrays (like matrices or 3D images). Stride allows deep learning libraries like PyTorch to represent these complex shapes without copying or rearranging the data in memory. By simply changing the stride (along with shape), PyTorch can perform operations like transpose or slicing instantly.
  • How it works: Each dimension of a tensor has a corresponding stride value. When you index into a tensor (e.g., accessing tensor[row, col]), the computer calculates the actual memory location by multiplying the indices by their respective strides: memory_location = (row * row_stride) + (col * col_stride).
  • Analogy: Imagine a row of 12 cupcakes lined up on a narrow bakery tray. If you decide to mentally treat them as a 3x4 grid (3 rows, 4 columns):
    • To get to the next cupcake in the same row (e.g., column 1 to column 2), you just move to the very next cupcake (stride of 1).
    • To get to the cupcake directly below it in the next row, you must jump over 4 cupcakes (stride of 4). Your navigation strides for this layout are (4, 1). If you decided to rotate the grid (transpose it), you could simply change your rules to "rows have stride 1, columns have stride 4" without moving a single cupcake.
  • Example: In PyTorch, a 2D tensor of shape (3, 4) stored in standard row-major order has a stride of (4, 1). If you transpose it to shape (4, 3), PyTorch does not rearrange the numbers in memory; it just updates the stride to (1, 4).

StyleGAN​

A family of GANs (StyleGAN, StyleGAN2, StyleGAN3) famous for photorealistic faces — the models behind sites like thispersondoesnotexist.com. Instead of forcing random noise directly into a rigid spherical shape (which tangles attributes together), it first passes the noise through a mapping network to "iron out" the warped space into an intermediate W latent space. It then injects this unwarped style code into every generation layer through adaptive instance normalization. This design "disentangles" the latent space, so moving in one direction smoothly changes a single attribute (hair, age, lighting) while leaving the rest completely untouched.

StyleGAN2​

An improved version of StyleGAN that fixes visual artifacts like waterdroplet-like blobs. It does this by redesigning how the adaptive instance normalization (AdaIN) is applied, moving it outside the convolutions. Think of it as upgrading from a good camera that sometimes leaves dust spots on the lens to a professional one that takes perfectly clean photos every time.

StyleGAN3​

The third generation of the StyleGAN family, which focuses on fixing "texture sticking" — a problem where textures like hair or wrinkles would stay glued to the screen coordinates even as the face moved. It achieved this by making the entire network "alias-free," ensuring that when the underlying features move, the generated pixels move perfectly with them, like a seamless video rather than a sequence of loosely connected frames.

Sum-tree​

The data structure that makes prioritized experience replay fast. It is a binary tree where each leaf holds one transition's priority and every internal node holds the sum of its two children, so the root holds the total priority of the whole buffer. To draw a sample with probability proportional to priority, you pick a random number between 0 and that total and walk down from the root — at each node going left or right depending on whether the number falls inside the left child's summed priority — reaching a leaf in O(log n) steps instead of the O(n) it would take to scan and normalize every priority. Updating a changed priority is equally cheap: fix that one leaf and the handful of parent sums above it. Analogy: a roulette wheel whose slot sizes are proportional to each transition's priority, so one spin lands on a transition with exactly the right odds — but built so you can resize a slot without repainting the whole wheel.

Super-resolution​

Turning a low-resolution image or video into a higher-resolution one by inventing the missing fine detail — not merely stretching the pixels (which only blurs them) but hallucinating plausible texture and sharp edges that were never in the small version. A diffusion-based super-resolution model is trained by taking sharp images, shrinking them, and learning to reconstruct the originals while conditioned on the small input. Like an artist handed a thumbnail and asked to repaint it at poster size, filling in detail consistent with what the thumbnail implies. It is the upscaling stage in a cascaded diffusion pipeline, and "super" simply means resolving detail finer than the input's resolution seemed to allow.

SWE (Software Engineering)​

Short for Software Engineering — the discipline of building, testing, and maintaining software systems. In the AI/LLM context, "SWE" usually appears in compound terms like SWE-bench or "SWE-style agent," meaning an agent that does the kind of work a human software engineer does: reading code, diagnosing bugs, writing fixes, and running tests.

SWE-bench​

Short for Software Engineering Benchmark — a benchmark of real GitHub issues paired with the code changes that fixed them; an agent is judged by whether its edits make the project's test suite pass, which makes it the standard test of coding agents.

Sweep​

Training the same model many times while changing one setting across a range of values, then comparing results to pick the best — for example trying ten different learning rates and keeping the winner. Like tasting a sauce as you add salt in small steps to find the amount you like, rather than guessing the whole spoonful at once. A sweep is how you turn a hyperparameter hunch into a measured choice.

SwiGLU​

The activation used in most modern transformer MLPs: a GLU gate whose non-linearity is Swish, written (xW) · Swish(xV). In plain terms, the input is projected two ways — one path is the content, the other is squeezed through Swish to become a soft gate — and the two are multiplied so the gate dials each value up or down. It replaced plain ReLU feed-forward layers because, for the same size, it tends to learn a little better; it is the default FFN in Llama-style models.

Swish​

A smooth activation function, x · σ(x), also called SiLU. It does roughly the same job as ReLU — squashing large negatives toward 0 and passing positives through — but with a gentle curve instead of a sharp corner (and it even dips a little below 0 for small negatives before recovering). Think of a soft-closing drawer that eases shut instead of slamming at exactly zero: that smoothness gives the network cleaner gradients to learn from. It is the non-linearity used as the gate inside SwiGLU.

Synthetic captions​

Replacing an image's original web alt-text — which is often missing, keyword-spammed, or unrelated to the picture — with a fresh, detailed caption written by a VLM that actually looks at the image and describes it ("a golden retriever catching a frisbee on a beach at sunset"). Also called recaptioning. Training a text-to-image model on these cleaner descriptions dramatically improves how faithfully it follows prompts — it is the single biggest reason DALL·E 3 became so good at composition. Like re-cataloguing a library where half the books were shelved under the wrong title: once every spine is relabeled to match its contents, readers (here, the model) finally learn which words map to which pictures. Example: an image whose alt-text was "IMG_2025.jpg" gets a full descriptive sentence before it is used for training.

SynthID​

Google DeepMind's watermarking tool that hides an invisible, detectable signal inside AI-generated content — images first, later audio, text, and video — so it can be identified as machine-made without any visible change to the picture. Rather than stamping the pixels afterward, it can weave the mark into the generation process itself, which helps it survive cropping, resizing, and JPEG compression. Like a secret ink woven into the paper of a banknote: you can't see it, but the right detector lights it up instantly. It is one practical answer to the safety problem of telling real photos from synthetic ones as AI images flood the web.

System identification​

The process of building mathematical models of a physical system's dynamics by analyzing its actual inputs and outputs, rather than relying solely on first-principles physics equations. In robotics, it typically involves exciting a robot's joints (such as using chirp signals) to measure the resulting motion and torque, then using optimization or curve-fitting techniques (like L2-regularized least squares) to estimate parameters like link mass, center of mass, inertia, joint friction, and actuator damping. Analogy: Trying to figure out the weight and center of balance of an unmarked package by gently shaking and rotating it in your hands; the way it resists your force tells you its internal physical properties without you opening the box. This is crucial for high-performance control algorithms (like impedance control or computed-torque control) which fail or become unstable if the model's parameters do not match the real robot's hardware.

System prompt​

A message placed at the very start of a chat conversation that tells the model how to behave — its role, tone, rules, and the tools it can call — before the user's first turn ever arrives. Like a stage director's note to an actor before the curtain rises: "You're a polite customer-support agent who answers only refund questions." System prompts are usually long and shared across many requests, which is why caching their KV state (see prefix cache) saves so much repeated work.

Systolic array​

A grid of tiny, identical "multiply-and-add" units wired directly to their neighbors, used to do the huge matrix multiplications (GEMMs) at the heart of neural networks. Instead of each unit fetching its numbers from memory, doing one calculation, and writing the answer back, the numbers march through the grid step by step — every clock tick, each unit grabs whatever just arrived from the cell to its left and the cell above, multiplies them, adds the running total, and passes the values along to the next cell. The data flows through the chip in rhythmic pulses, which is exactly where the name comes from: "systole" is the medical word for a heartbeat pumping blood, and here the data gets pumped through the array the same way.

  • Why it matters: Reading and writing memory is the slow, power-hungry part of computing (see memory bandwidth). A normal processor reloads operands from memory for almost every multiply. A systolic array loads a number once at the edge of the grid and then reuses it across an entire row or column of units before it ever leaves — so for a big matrix multiply it can do thousands of multiply-adds per number fetched. That reuse is why it's so fast and energy-efficient, and why Google's TPU is built around one (its 2D grid of multiply-add cells, fed from fast on-chip SRAM).
  • Analogy: Think of a bucket brigade fighting a fire. Nobody runs back to the well for each bucket; people stand in a line and pass buckets hand to hand, so water flows continuously. A systolic array is a 2D bucket brigade for numbers: each cell hands its result to the next, and the matrix "flows" across the grid without anyone running back to memory.
  • Example: To multiply two 256×256 matrices, a TPU streams one matrix's rows in from the left edge and the other's columns in from the top. As the values cross paths inside the grid, each of the ~65,000 cells accumulates one entry of the answer. The whole multiplication happens in one smooth wave of data, instead of tens of millions of separate fetch-multiply-store trips to memory.

T2V​

Text-to-Video: generating a video clip from a text prompt alone, with no image to start from — the model must invent both what the scene contains and how it moves. This makes it harder than image-to-video (I2V), where the first frame is given, and it needs paired text–video training data, which is scarce. Sora, Veo, and Kling are well-known T2V systems.

T5​

A text transformer (Google's "Text-to-Text Transfer Transformer") that reads a sentence and produces rich embeddings of its meaning. Unlike CLIP's text encoder, which was trained only to match images to short captions, T5 was trained on general language tasks, so it captures long, detailed prompts and word order more faithfully — which is why models like Imagen, SD3, and Flux feed it (often the large "T5-XXL" variant) into cross-attention for better prompt adherence.

Tail latency​

The latency of the slowest requests (for example the p95 or p99 percentiles) rather than the median (p50); it is what users notice most.

Talking head​

A model that takes a single portrait photo plus an audio clip and generates a video of that person speaking the audio — lips, jaw, and head moving in sync with the sound. The audio drives the motion while the photo fixes the identity, so the same face can be made to say anything. The hardest part is lip sync: the mouth has to form the right shape for each speech sound at the right instant, which is why these systems extract audio features (often with a speech encoder such as wav2vec) and align them to facial motion frame by frame, then enforce temporal consistency so the face does not flicker or drift. Named systems include EMO, Hallo, SadTalker, and V-Express. Like a ventriloquist's puppet driven by a recording instead of a hand.

Tanh squashing​

The standard way a continuous-action policy (as in SAC) keeps its actions inside the environment's legal range. The network first outputs an unbounded Gaussian — a mean and spread that could in principle suggest any real number — and then each sampled value is passed through tanh, a smooth S-shaped function that squashes any input into the interval (−1, 1), after which it is rescaled to the action's true limits. This guarantees the action is always valid no matter what the network emits. The subtlety: bending a distribution through tanh changes its probability density, so the action's log-probability — which SAC needs for its entropy term — must include a correction (subtract Σ log(1 − tanh²(u)), the log of how much tanh locally stretches or compresses the axis). Skip or mis-sign that correction and SAC silently optimizes the wrong entropy. Analogy: tanh is a funnel that forces any value into a fixed-width chute; the correction accounts for how the funnel crowds probability mass near the walls, so you still know how likely each outcome was. The same value is computed with the reparameterization trick so gradients flow through the sampled action.

Target model​

In speculative decoding, the big, accurate model whose output you actually want — it checks the small draft model's guesses and has the final say on every token. Like the senior editor who must approve the assistant's draft: slow and expensive to consult, so the trick is to bother it as rarely as possible while still letting it decide the real answer.

TCP​

Tool Center Point — the configurable point on a tool whose pose tracking controls

Target network​

A slowly-updated copy of the DQN Q-network, used only to compute the learning target r + γ·maxₐ Q_target(s′, a). The problem it solves: if the same network being trained also produces its own target, then every gradient step moves the goalpost it is aiming at, and the feedback loop can spiral out of control. Freezing a copy and refreshing it only every few hundred steps (a hard update) — or nudging it a tiny fraction toward the live network on every step (a soft update, also called Polyak averaging) — gives training a stable target to chase for a while. It is one of the two stabilizers (with experience replay) behind DQN's success. Analogy: aiming at a parked car is easy; aiming at one that lurches forward every time you take a step is not.

Target policy smoothing​

One of TD3's three stability fixes: when building the learning target, add a small clipped random noise to the next action before the critic scores it. Without this, the critic can develop a sharp, narrow spike — an action whose value it has wildly overestimated — and the deterministic actor will happily steer straight into that spike. Averaging the target over a little noise around the action smooths those spikes out, so the critic only rewards actions that are good across a small neighborhood, not a single lucky point. It is a form of regularization built on the common-sense prior that similar actions should have similar values. Analogy: grade a student on the average of several nearby answers rather than one cherry-picked response, so a single fluke cannot inflate the score. Compare delayed policy updates and twin critics.

TD error​

δ_t = r_t + γV(s_{t+1}) − V(s_t) — the gap between a bootstrapped one-step estimate of a state's value and the value you currently hold for it. It is the core learning signal of temporal-difference learning: a positive δ means things turned out better than expected, so nudge V(s_t) up; a negative δ means worse, so nudge it down. Every TD method — SARSA, Q-learning, and TD(λ) with eligibility traces — is some rule for which states to apply this error to.

TD-MPC2​

A model-based RL algorithm that plans over a short horizon with a learned latent dynamics model and then bootstraps from a learned value function past the planning horizon — so short-term decisions come from planning (precise) and long-term consequences are filled in by the value estimate (cheap). The "TD" is for its temporal-difference value learning; "MPC" for the model predictive control planner, which uses the Cross-Entropy Method for action search. Its headline result is mastering the entire DeepMind Control Suite with a single set of hyperparameters. Analogy: a chess player who calculates the next few moves exactly, then judges the resulting position by intuition rather than calculating all the way to the end of the game.

TD3​

Twin Delayed DDPG — DDPG with three fixes that turn a fragile algorithm into a reliable one. (1) Twin critics: learn two Q-networks and use the smaller of the two as the learning target, so the actor cannot exploit one critic's lucky overestimate. (2) Delayed policy updates: update the critic several times for each actor update, letting the value estimate settle before the actor chases it. (3) Target policy smoothing: add a little noise to the action used in the target, so the critic cannot overfit to a razor-thin peak in its value estimate. Together these tame the overestimation and instability that plague DDPG, and on most continuous-control benchmarks TD3 lands well above it. It is a strong, simple baseline; SAC usually matches or beats it by adding an entropy bonus on top.

Teacher forcing​

The training convention behind next-token prediction: at every position the model is fed the true previous tokens from the training data, not the tokens it would have predicted itself. This is what lets a whole sequence be trained in a single forward pass — because every position's input is known in advance, all the next-token losses can be computed at once, in parallel, rather than one step at a time.

  • Why it matters: It is the reason pretraining is fast. Combined with the causal mask, a length-T sequence yields T supervised prediction problems from one pass. The catch is exposure bias: at inference the model must feed on its own generated tokens, a distribution it never saw during teacher-forced training, which is part of why generation can drift.
  • Analogy: Learning to play a song by always reading the next note off the sheet music, even after you fumble a note — you never have to recover from your own mistakes during practice, so the first live performance (no sheet music) is a different, harder task.
  • Example: To train on "to be or not", the model is simultaneously asked to predict "be" from "to", "or" from "to be", and "not" from "to be or" — always conditioned on the correct prefix, never on its own guesses.

Teleoperation​

The remote control of a robot by a human operator, often used as a mechanism to collect high-quality demonstration data for imitation learning and behavior cloning. The human operator directs the robot's motions using interfaces such as virtual reality (VR) controllers, leader-follower rigs (where the human moves a passive "leader" robot and the "follower" replicates the movement), or smart-glove interfaces.

  • Why it matters: Designing rewards or programming complex manipulation behaviors from scratch is extremely difficult. Teleoperation allows humans to demonstrate the task directly, recording state-action pairs (such as camera images, joint angles, and motor commands) that can then be used to train policies.
  • Analogy: Imagine trying to teach someone how to write a signature by writing a detailed manual of math equations describing the force and angle of the pen at every millisecond (hard). Instead, you grab their hand and guide it through the signature a few dozen times (easy). Teleoperation is that physical hand-guiding process for robots.
  • Example: Using a dual-arm follower setup like ALOHA, a human operator guides the leader arms to pick up a sponge and wipe a table. The follower robot replicates the movements, and the sensor logs are saved to train a behavior cloning policy.

Temperature​

A sampling knob that scales the model's scores before softmax: low temperature (e.g. 0.2) sharpens the distribution so the model plays it safe and repeats the likeliest words, while high temperature (e.g. 1.5) flattens it so rarer, more surprising words can win. Think of it as a creativity dial — turn it down for factual answers, up for brainstorming. The same knob appears in contrastive learning (written τ): there the similarity scores are divided by τ before the softmax, so a small τ (CLIP learns one starting around 0.07) sharpens the contest and forces the model to focus on its hardest negatives, while a large τ softens it — too small destabilizes training, too large and even the true pair is barely preferred. In maximum-entropy RL, temperature (written α) scales the entropy bonus: a high temperature encourages the agent to explore widely and act randomly, while a low temperature lets the agent focus purely on maximizing rewards.

Temporal attention​

Attention applied only along the time axis of a video: each spatial position (say, the pixel at row 10, column 20) looks at that same position across all the frames and decides how its value should change from frame to frame. It is the half of a (2+1)D block that handles motion, added on top of ordinary spatial attention (which works within each frame separately), and in temporal inflation it is exactly the new layer dropped into a pretrained image model to teach it movement. Like tracking one fixed spot on a flip-book through every page to see how it animates, while ignoring the rest of each page. It is far cheaper than spatiotemporal attention because each position only compares itself across the T frames, not against every other position as well.

Temporal consistency​

The property of a generated or edited video where each object keeps a stable appearance, position, and identity across frames instead of changing slightly every frame. It is what separates real video from a flip-book of independently produced images: get it wrong and you see temporal flicker, crawling textures, or a character whose face morphs shot to shot. Because a model that processes frames one at a time has no built-in reason to keep them aligned, video methods deliberately share information across time — through temporal attention, a 3D VAE that compresses several frames together, or by reusing features and noise between frames during editing. It is the single recurring obstacle in video-to-video, control, and long-form generation. Like a team of animators agreeing on exactly how a character looks so it does not subtly change every drawing.

Temporal flicker​

The shimmering, pulsing, or boiling look you get when a video is processed one frame at a time with no coordination across frames. Each frame is reconstructed (or generated) slightly differently from its neighbors — tiny independent errors in texture, color, or brightness — and because the real scene barely changed, your eye reads those frame-to-frame differences as unwanted motion. It is the classic failure of running an image VAE per frame, and the main reason video needs compressors and models that span the time axis (a 3D VAE or temporal attention) rather than treating a clip as a stack of unrelated stills.

Temporal inflation​

The dominant trick for making a video model out of an existing image model: take a pretrained 2D network, insert new layers that operate along the time axis (temporal convolutions or temporal attention), and usually initialize them as an identity (pass-through) so that, at the start of training, the inflated model behaves exactly like the original image model run frame by frame. You then fine-tune so the new layers gradually learn motion while the spatial layers keep everything they already knew about appearance. "Inflation" captures the picture of taking a flat 2D model and puffing it out into the third (time) dimension. Stable Video Diffusion, AnimateDiff, and Make-A-Video all use variants of this; the 2024+ frontier (Sora-class models) instead trains spatiotemporal models from scratch.

Temporal-difference learning​

The central learning idea of RL: update a value estimate from a single step of experience by bootstrapping — using your own current estimate of the next state's value as a stand-in for the rest of the future. The one-step version, TD(0), moves V(s) toward the target r + γ V(s′); the gap between that target and the old estimate is the TD error. Unlike a Monte Carlo update, TD does not wait for the episode to finish, so it learns online, step by step — trading a little bias (the target leans on a still-imperfect estimate) for much lower variance. Q-learning, SARSA, and the eligibility-trace method TD(λ) are all temporal-difference algorithms; the name refers to learning from the difference between two estimates made at different times.

Tensor​

A tensor is a multi-dimensional grid of numbers. It is the fundamental data structure used to store and manipulate data in deep learning.

  • Why it matters: Tensors are the universal language of neural networks. Everything—whether it is an image, a sound wave, a word, or the internal weights of a model—is represented as a tensor. By organizing data into these structured grids, computer chips like GPUs can perform millions of mathematical operations on them simultaneously.
  • How it works: A tensor is a generalization of arrays to any number of dimensions. Under the hood in frameworks like PyTorch, a tensor is represented by bookkeeping numbers (its metadata, such as shape, stride, offset, and dtype) that views a single continuous block of 1-D computer memory (the storage buffer).
  • Analogy:
    • 0-D Tensor (Scalar): A single number, like a temperature reading (e.g., 72).
    • 1-D Tensor (Vector): A list of numbers, like hourly temperatures for a day (e.g., [72, 74, 71]).
    • 2-D Tensor (Matrix): A grid of numbers, like a spreadsheet of daily temperatures for a week, where rows are days and columns are hours.
    • 3-D Tensor: A stack of grids, like a book of spreadsheets where each page is a different city's temperature chart. You can keep stacking these grids into higher dimensions (4-D, 5-D, etc.) to represent complex data.
  • Example: A color image is typically stored as a 3-D tensor with the dimensions [Height, Width, Channels]. The third dimension (Channels) has a size of 3, representing the Red, Green, and Blue intensity values for every pixel in the grid.

Tensor Core​

A Tensor Core is a specialized hardware unit inside modern NVIDIA GPUs (introduced in the Volta architecture) designed to perform high-speed matrix math.

  • Why it matters: Matrix multiplication is the heaviest mathematical task in deep learning. While standard CUDA cores calculate math one number at a time, Tensor Cores are hardwired to process entire grids of numbers (matrices) at once. This hardware acceleration is what makes training and running large models, like LLMs, feasible.
  • How it works: A Tensor Core specializes in a single combined operation: multiplying two small matrices (typically 4×4 or 16×16) and adding a third matrix to the result, all in a single clock cycle. It achieves this by using low-precision formats (like float16 or bfloat16) to speed up the multiplication, while accumulating the result in higher precision (float32) to prevent errors.
  • Analogy: Imagine a classroom of students trying to solve math problems. A standard CUDA core is like a student doing one multiplication at a time on scratch paper (e.g., 3 × 5 = 15). A Tensor Core is like a student using a custom rubber stamp: they press it down once, and it instantly calculates and prints a whole grid of multiplication results on the page.
  • Example: When you train a model in PyTorch using Automatic Mixed Precision (AMP), PyTorch automatically detects operations that can be run in 16-bit precision. It then routes those matrix multiplications directly to the GPU's Tensor Cores, often speeding up the computation by 4× to 10× compared to standard CUDA cores.

Tensor parallelism (TP)​

Splitting each layer's weights across several GPUs so they each do part of the math, then combining their partial results with an all-reduce. "At attention/MLP boundaries" means that combining happens at two natural seams in every transformer block — once at the end of the attention sublayer and once at the end of the MLP sublayer — because within a sublayer the GPUs can work independently, but at its edge their pieces must be added back together before the next step can start. Like four cooks each preparing part of a dish and merging everything at two fixed points before it moves on.

TensorRT​

TensorRT is NVIDIA's high-performance deep learning inference optimizer and runtime library. It takes a trained neural network, analyzes every layer, and rewrites the computation graph into the fastest possible form for a specific NVIDIA GPU — fusing adjacent operations, choosing optimal data layouts, selecting the best kernel implementations, and optionally quantizing weights and activations to lower precision (FP16, INT8, or FP8).

  • Why it matters: A model trained in PyTorch typically runs much slower at inference than it could because it executes each operation individually with Python overhead. TensorRT eliminates that overhead by compiling the model into a single optimized engine tuned for the exact GPU it will run on, often delivering 2–10× faster inference compared to an unoptimized PyTorch forward pass.
  • How it works: You export your trained model (usually via ONNX), then feed it to TensorRT's builder. The builder profiles every layer on the target GPU, selects the fastest kernel for each operation, fuses layers where possible (e.g., combining a convolution, batch normalization, and activation into one kernel call), and optionally calibrates INT8 precision using a representative dataset. The output is a serialized "engine" file that loads and runs directly on the GPU with minimal overhead.
  • Analogy: Imagine you have a recipe (your trained model) written in a general cookbook. TensorRT is like a professional kitchen consultant who rewrites that recipe specifically for your kitchen: combining steps that can happen simultaneously, pre-measuring ingredients into single containers, and swapping bulky equipment for compact, faster tools — so the dish comes out the same but in half the time.
  • Example: Deploying a YOLOv8 object detection model on a Jetson Orin Nano: exporting the PyTorch model to ONNX, running trtexec to build an INT8 TensorRT engine calibrated on sample images, and achieving real-time detection at 30+ FPS on a 15-watt edge device.

Tenstorrent​

An AI hardware and processor company that designs specialized computing platforms optimized for machine learning models, particularly transformers. Unlike traditional GPUs that use complex warp scheduling, Tenstorrent's architecture (such as Grayskull and Wormhole) is built around a network-on-chip (NoC) connecting a grid of tiny RISC-V processor cores, each with its own local memory. This design allows computation to be modeled as a pipeline of data flowing through the grid of cores, matching the natural graph structure of deep learning models. They provide an open-source software development kit (SDK) called tt-metal for programming these accelerators.

  • Analogy: Imagine a large postal sorting office. Instead of having a few giant sorting desks that constantly retrieve packages from a central warehouse (like GPU cores fetching from memory), Tenstorrent is like a large grid of individual mail clerks standing in rows. Each clerk has their own tiny desk (RISC-V core with local memory) and passes packages directly to the next clerk in line (data flowing through the network-on-chip), making the overall flow of mail (tensors) extremely smooth and fast without warehouse traffic jams.
  • Example: Compiling a neural network onto a Tenstorrent Wormhole card using the open SDK, where the compiler maps different layers of the model to different physical regions of the grid of RISC-V cores and streams the input data through them.

TF32​

TensorFloat-32 — a proprietary 19-bit math format introduced by NVIDIA for accelerating single-precision workloads on Tensor Cores. It uses the same 8-bit exponent as FP32 (to preserve dynamic range) and the same 10-bit mantissa as FP16 (to reduce hardware footprint), allowing the GPU to run matrix multiplications faster without requiring code changes to scale gradients.

Analogy: A shipping box that is standard height and width on the outside (so it fits on all standard conveyor belts and delivery trucks without modifications), but is shallower on the inside to save material and weight. It fits the same sorting system (the FP32 data pipeline) but handles the cargo (precision math) with less internal volume (fewer mantissa bits).

Example: When training a model on an A100 GPU with automatic TF32 enabled, the GPU automatically reads FP32 inputs, converts them to TF32 to accelerate matrix multiplications on the Tensor Cores, and writes the output back in standard FP32. This provides up to a 10× speedup on matrix math compared to standard FP32 training, with virtually no impact on model accuracy.

TFLOPs​

Tera (10¹²) floating-point operations per second

TGI​

Short for Text Generation Inference — Hugging Face's open-source LLM serving engine, similar in role to vLLM. It implements continuous batching, PagedAttention, and quantized inference behind a simple HTTP API, and is one of the two engines most commonly used to put an LLM in front of real users.

Thinking budget​

A cap on how many tokens a reasoning model is allowed to spend thinking before it must give an answer — like telling a student "you have ten minutes of scratch work, then write your answer." It lets a serving system trade accuracy for cost and latency: a bigger budget usually means better answers on hard problems but slower, pricier responses.

Text encoder​

The part of a model that turns a piece of text into numbers — a list of embeddings that capture what the words mean — so the rest of the system can work with language as math. Think of it as a translator that reads your prompt and rewrites it in the only language a neural network understands: vectors. In a text-to-image model the text encoder reads the prompt once and the diffusion model then keeps glancing at those vectors (via cross-attention) to decide what to draw. It is one half of a two-part model like CLIP — the side that reads words, paired with an image encoder that reads pictures — but the term is more general: any model that maps text to embeddings (CLIP's text tower, T5, a BERT-style encoder) is a text encoder, which is why it deserves its own name rather than being conflated with CLIP as a whole.

Text rendering​

The ability of an image generator to draw legible, correctly-spelled words inside the picture — a shop sign that actually reads "OPEN," not "OPNE" or wavy gibberish. It was for years the field's most visible failure, because a model trained only to match overall image statistics has no spelling checker: it learns the shape of letters but not that "the exact order of letters matters." Modern models (Imagen 3, Flux, Ideogram) largely fixed it with dedicated training data and stronger text encoders. Like a painter who can flawlessly reproduce the look of handwriting in a language they cannot read — beautiful strokes, but misspelled words until they are taught the alphabet itself. Example test: prompt "a neon sign that says 'DIFFUSION'" and check whether all nine letters appear, in order, spelled right.

Text-to-image​

A model that turns a written prompt into a brand-new picture — you type "a corgi astronaut floating in space" and it paints one from scratch. Under the hood a text encoder reads your words into numbers, and a generator (usually a diffusion model) uses them to decide what to draw, glancing back at the prompt the whole time via cross-attention. Like a sketch artist who never sees the scene and draws purely from your verbal description — the richer and clearer your words, the closer the result. Famous examples include Stable Diffusion, DALL·E 3, Imagen 3, and Ideogram.

Textual Inversion​

A personalization method that teaches a frozen diffusion model a new subject by learning a single new word for it — nothing in the model itself changes. Concretely it optimizes one fresh vector added to the text encoder's embedding matrix (the lookup table of word embeddings) so that this invented "word" makes the model draw your subject. Because only that one vector is trained, the saved file is a few kilobytes — the smallest personalization artifact there is — but its capacity is limited: one vector can pin down a recognizable look yet cannot match the fidelity of LoRA or DreamBooth, since the frozen model can only render what it already knows how to draw. Like adding one new entry to a shared dictionary: you define the word just once, and from then on that single word stands in for your whole subject — but, just like a dictionary, it can only ever be explained using words and ideas the model already understands.

Thermal throttling​

The automatic reduction of a processor's clock speed (and therefore performance) when it detects that its temperature is approaching a dangerous limit. The chip's built-in sensor triggers this slowdown to prevent physical damage — silicon transistors can degrade or fail if they get too hot.

  • Why it matters: Developers running multi-GPU workstations or edge devices under heavy model training or inference loads often hit thermal throttling without realizing it. The system silently slows down, making training or inference take longer while drawing the same power. Understanding and preventing it — through better cooling, undervolting, or power-limit configuration — is essential for sustained peak performance.
  • Analogy: Imagine running full speed on a treadmill in a hot room. As your body temperature rises, you instinctively slow to a jog to avoid overheating. Thermal throttling is the chip doing the same thing: it slows itself down to cool off, even though it could run faster if it were cooler.
  • Example: Two identical RTX 4090 GPUs running a training job — one in a well-ventilated case with good airflow stays at 70°C and maintains its full boost clock, while the other in a cramped case hits 83°C and throttles down by 15%, taking 15% longer to complete each epoch despite being the same hardware.

Thompson sampling​

An exploration rule that keeps a belief — a probability distribution over how good each option might be — then, at decision time, draws one plausible world from that belief and acts as if it were true. Options the agent is uncertain about occasionally get sampled at a flattering value and therefore get tried; options it has ruled out with confidence almost never are. Each trial sharpens the belief, so the exploration switches itself off exactly where the data is already conclusive. Compared with ε-greedy — which explores by throwing away its knowledge at random moments — Thompson sampling explores in proportion to how uncertain it actually is, which is why it comes with strong theoretical guarantees in bandit problems. Bootstrapped DQN is the usual way to approximate it with neural networks. Analogy: instead of flipping a coin about which route to take to work, imagine one convincing version of "today's traffic" consistent with everything you have seen, and drive the route that is best in that imagined world.

Thread​

In parallel computing (especially GPU programming like CUDA or Triton), a thread is the smallest unit of execution that runs a program (or kernel) on a single set of data.

  • Why it matters: Modern GPUs achieve massive throughput by executing thousands of threads simultaneously. Instead of running a loop sequentially on a CPU, a GPU schedules a separate thread for every single data element (like a single pixel or a single entry in a matrix), performing the entire calculation in parallel.
  • How it works: When a GPU kernel is launched, it starts a grid containing millions of threads. Each thread receives a unique identifier (such as its thread or block index) that it uses to calculate which memory address to read and write. Threads are grouped into blocks, and further scheduled in hardware as warps of 32 threads that execute instructions in lockstep.
  • Analogy: Imagine a large office where you need to check and sign 1,000 separate documents. A CPU is like a single highly efficient manager who reads and signs each document one after the other. A GPU is like hiring 1,000 temporary workers (threads) and giving each worker exactly one document to sign: they all sign their document at the exact same moment, completing the work in a fraction of the time.
  • Example: In a vector addition kernel (C[i] = A[i] + B[i]), each thread uses its global thread index i to load A[i] and B[i], add them together, and store the result in C[i].

Three-point turn​

A standard driving maneuver used to turn a vehicle around in a narrow road by driving forward while steering one way, reversing while steering the other way, and then driving forward again.

  • Why it matters: It is a classic demonstration of how to navigate a system with nonholonomic constraints (like a car that cannot slide sideways). Because the vehicle's direction of travel is constrained by the rolling direction of its wheels, it cannot simply pivot in place or slide sideways to change direction. It must execute a sequence of forward and backward motions.
  • Analogy: Imagine trying to turn yourself around while walking inside a very tight sleeping bag. You cannot simply pivot on your heels like you normally would. Instead, you have to wiggle forward, twist, shimmy backward, and twist again, using multiple steps to redirect your body.
  • Example: A car-like robot navigating a dead-end hallway. If the hallway is narrower than the robot's turning circle, the path planner must generate a trajectory that includes backing up and steering in reverse—a three-point turn—to orient the robot back toward the exit, rather than attempting a smooth, continuous U-turn.

Throughput​

The total volume of work a system completes in a given time period. For training, it is measured in processed examples or tokens per second; for inference serving, it is measured in requests or total tokens generated per second across all users.

  • Why it matters: High throughput is the key to cost-effective scaling, as it determines how many concurrent workloads or users can be served by a single piece of hardware.
  • Analogy: A highway's capacity. If a multi-lane highway can move 2,000 cars past a toll booth every minute, it has high throughput, even if individual cars experience traffic delays and take longer to reach their destination (high latency).
  • Example: Implementing continuous batching in serving engines like vLLM increases throughput by dynamically packing multiple requests together, keeping the GPU's tensor-core compute units busy.

Tic-Tac-Toe​

The familiar 3×3 pencil game where two players alternately place X's and O's, trying to get three in a row. Because it is tiny — only a few thousand possible games, and a guaranteed draw under perfect play — it is a favorite first testbed for search-and-learning agents like MuZero, where you can verify the whole policy/value/dynamics loop almost by hand before scaling up. Think of it as the "hello world" of board-game RL.

Tiling​

A memory-optimization technique that breaks a large computation (like multiplying two huge matrices) into smaller blocks, or "tiles," that can fit entirely inside a processor's fast on-chip memory (like SRAM or shared memory).

  • Why it matters: Accessing the main off-chip memory (HBM or DRAM) is extremely slow compared to the processor's calculation speed. Without tiling, threads would constantly wait for data to arrive from slow main memory, making the operation memory-bound.
  • How it works: Instead of loading entire massive arrays from slow memory, the GPU loads a small "tile" of data into its fast local SRAM once. The threads perform all the required calculations on that tile locally at maximum speed, and then write the final result back to main memory.
  • Difference between a Tensor and a Tile:
    • A Tensor is the entire, logical multi-dimensional grid of numbers (e.g., a massive 10,000 × 10,000 matrix). Because it is so huge, it has to live in the GPU's slow main memory.
    • A Tile is a small, physical slice of that tensor (e.g., a 16 × 16 block of numbers). It is temporarily copied into the GPU's fast, local on-chip memory so the processor can work on it immediately.
  • Analogy: Imagine you are a researcher studying a massive, multi-volume encyclopedia set (the Tensor) located on bookshelves far away (slow main memory). Your desk represents fast local memory (SRAM). Instead of walking back and forth to read one sentence at a time, you carry a single volume (a Tile) to your desk, extract all the information you need from it, and only go back to the shelves when you need the next volume.
  • Example: In a tiled matrix multiplication kernel, two large tensors are divided into smaller sub-matrices (e.g., 16×16 tiles). A block of threads loads these tiles into shared memory, multiplies them, and accumulates the results before moving to the next tile.

TinyTapeout​

TinyTapeout is an educational project and platform that makes custom silicon chip design accessible and affordable for hobbyists, students, and individual developers. It allows multiple designs to be combined onto a single physical microchip, which is then manufactured via a shared multi-project wafer (MPW) run.

  • Why it matters: Designing and manufacturing a custom ASIC is historically prohibitively expensive, costing millions of dollars for a single manufacturing run (tapeout). This makes custom silicon impossible for individuals. TinyTapeout solves this by dividing a single silicon chip into hundreds of tiny slots; developers submit their digital designs (in hardware description languages like Verilog), and TinyTapeout aggregates them onto a single wafer, reducing the cost of custom silicon to a few hundred dollars.
  • How it works: TinyTapeout uses the open-source SkyWater 130nm process PDK. Designers create digital logic circuits using web-based tools (like Wokwi) or standard hardware description languages. The platform automatically tests, routes, and packages these designs. After fabrication, participants receive a physical chip mounted on a development board, where they can toggle inputs and read outputs to test their custom logic.
  • Analogy: Imagine wanting to print a custom book, but the printing press requires a minimum order of 10,000 copies, which is too expensive. TinyTapeout is like a community anthology where hundreds of authors each write a single page. The printing press prints the combined book, and every author gets a copy of the whole book, but they can easily flip to their own page to read their work.
  • Example: Submitting a simple 8-bit hardware multiplier or a small hardware neural network activation unit to a TinyTapeout run, then receiving the physical silicon months later and testing it on your workbench.

Token (visual/audio)​

A discrete code that stands for a small piece of an image or sound, produced by a VQ-VAE or neural codec. Just as a tokenizer chops text into word-pieces, an image tokenizer turns a picture into a grid of these codes drawn from a fixed vocabulary — so a transformer can model images (or audio) with the same machinery it uses for language.

Tokenizer​

The mapping from string to integer IDs; trained, frozen, part of the model contract

Tokens per byte​

A measure of tokenizer efficiency: how many tokens it emits per byte of input text; higher means the same text costs more tokens

Tool call​

When an LLM, instead of answering directly, emits a structured request to run an external function — search the web, query a database, run code — and then continues once it sees the result. Like a person pausing mid-task to look something up or use a calculator, it is the basic action an agent takes in its loop.

Top-k​

A sampling rule that keeps only the k most likely next tokens and draws from those, throwing away the long tail. With k=1 it always takes the single best word (greedy decoding); with k=50 it chooses among the top 50 — like ordering only from a menu's 50 most popular dishes instead of the whole cookbook.

Top-p​

Also called nucleus sampling: instead of a fixed count like top-k, it keeps the smallest set of top tokens whose probabilities add up to p (e.g. 0.9), then samples from them. The shortlist automatically grows when the model is unsure and shrinks when it is confident — always keeping just enough candidates to cover 90% of the model's belief.

TOPP (Time-Optimal Path Parameterization)​

The process of taking a geometric path (a sequence of positions in space) and assigning a time to each point to find the fastest traversal that respects the robot's physical limits (such as joint-velocity, acceleration, and torque limits). A common modern algorithm for this is TOPP-RA (Time-Optimal Path Parameterization by Reachability Analysis), which formulates the problem as a sequence of linear programs. Analogy: Imagine drawing a curvy race track on paper (the geometric path). TOPP is the process of deciding exactly how fast a race car should drive along that track (the time parameterization) so that it crosses the finish line as quickly as possible without sliding off the track or blowing the engine (respecting acceleration and torque limits). Example: A factory sorting robot. Once a sampling-based planner finds a collision-free geometric path from the pickup bin to the drop box, TOPP calculates the optimal velocity profile along that path, allowing the robot to move at its physical limits without damaging its motors.

torch.compile​

The PyTorch 2.x API that JIT-compiles PyTorch code into optimized CUDA or Triton kernels. It traces the model execution to capture a computation graph, performs optimization passes like kernel fusion and dead-code elimination, and generates fast compiled kernels, speeding up standard eager mode code. Analogy: A translator who translates a book page-by-page as you read it (eager mode), versus a translator who reads the entire chapter ahead of time, optimizes the sentence flow, merges redundant phrases, and prints out a clean, polished copy for you to read fast (compiled mode). Example: Wrapping a model with compiled_model = torch.compile(model) invokes the Inductor compiler, which fuses adjacent element-wise operations (like linear bias + ReLU) into single compiled kernels to speed up execution.

torch.export​

The modern PyTorch API that captures a model into a standalone graph; the foundation for deployment paths like ExecuTorch and AOTInductor.

torch.multinomial​

The PyTorch function that draws a random sample from a probability distribution: hand it a list of probabilities and it rolls a weighted die, returning the index it lands on. A token with probability 0.6 comes up about 60% of the time. It is the "roll the dice" step at the end of sampling — the opposite of argmax, which never gambles. On the GPU each call is its own kernel launch, which is why folding it into the rest of the sampling math can speed up decode.

torchrun​

PyTorch's launcher command that starts one process per GPU and sets the RANK, LOCAL_RANK, and WORLD_SIZE environment variables those processes need to find each other.

TorchScript​

The legacy serialization/IR for PyTorch; superseded by torch.export

TPU​

Tensor Processing Unit — a custom application-specific integrated circuit (ASIC) designed by Google specifically for machine learning workloads. Unlike general-purpose CPUs or parallel GPUs, a TPU is built around a matrix multiply unit (MXU) that uses a systolic array, where data flows directly through a 2D grid of multiply-add units without constantly reading and writing back to a cache. Analogy: An automated sorting machine in a factory. A GPU is like a team of extremely fast packers, but they still have to pick up items from a warehouse shelf (cache/registers) and put them down. A TPU is like a conveyor belt system where items (data) flow continuously through the machines and get processed as they move, requiring no storage steps in between.

TPU pod​

A large-scale cluster of Tensor Processing Units (TPUs) physically connected by a dedicated, high-speed, custom network interface to act as a single supercomputer.

  • Why it matters: Training modern large language models (LLMs) requires massive computational power that far exceeds the capacity of a single processor. Splitting a model across normal servers causes a bottleneck because standard internet connections are too slow to keep the processors synced. A TPU pod solves this by linking hundreds or thousands of TPU chips with a specialized network, allowing them to share training data and model parameters with virtually zero communication delay.
  • How it works: In a TPU pod, TPU chips are arranged in a multi-dimensional grid (like a 2D or 3D torus) and connected via custom inter-chip interconnects (ICIs). This allows data to bypass the host CPUs entirely and flow directly between the TPU chips. Software libraries like JAX compile code to distribute calculations across the pod seamlessly using data-parallel and model-parallel paradigms.
  • Analogy: Imagine a single math genius (one TPU chip) working at a desk. They can solve equations incredibly fast, but a massive project (like training a frontier AI) is too big for one person. A TPU pod is like placing hundreds of these geniuses in the exact same room and giving them a set of ultra-fast pneumatic tubes (the high-speed interconnects) to pass notes back and forth instantly. Instead of acting like individual workers, the entire room functions as one giant, coordinated super-brain.
  • Example: Google's TPU v4 pods connect 4,096 individual TPU v4 chips together, allowing researchers to scale up a training job to exaFLOPs of performance. JAX programs can utilize jax.device_put or pmap to automatically partition data and coordinate training steps across the entire pod.

Trained or just prompted​

A choice in how you create a small helper model (like a router model). You can either "train" it (by fine-tuning its weights on thousands of examples, like sending someone to medical school) or "just prompt" it (by taking an existing smart model and simply giving it a written instruction like "You are a router, decide if this question is hard or easy," like handing a smart assistant a checklist). Training takes more effort upfront but is faster and cheaper to run; prompting is quick to set up but costs more per request since you process the instructions every time.

Trajectory sampling​

How PETS propagates uncertainty when it imagines the future. Instead of rolling a plan forward through the average of its ensemble of dynamics models, it rolls many independent particles forward, each one stepped by a randomly chosen ensemble member and each one drawing a random sample from that member's predicted distribution. The spread of where the particles end up is then an estimate of how uncertain the future really is — combining both epistemic doubt (which member you asked) and aleatoric noise (the randomness that member believes in). Averaging the models into one confident prediction instead would throw away precisely the disagreement the ensemble was built to produce, and the planner would then trust the model most in the places it knows least.

Trajectory optimization​

The process of formulating a robot's motion planning problem as a mathematical optimization problem. It finds a trajectory (a sequence of states and control inputs over time) that minimizes a cost function (such as execution time, energy consumption, or joint jerk) while satisfying constraints (such as obstacle avoidance, torque limits, and the robot's physical dynamics). Unlike purely geometric path planning, which only finds a collision-free path, trajectory optimization considers how the robot moves, ensuring the resulting motion is physically executable and smooth. Analogy: Imagine a bobsled team planning a run down an icy track. A simple path planner just tells them a line that stays inside the walls. A trajectory optimizer calculates exactly when they should steer, lean, and brace so that they navigate the curves at maximum speed without flipping over (violating physical dynamics) or exceeding their physical strength limits (torque constraints).

TrajOpt​

An optimization-based motion planning framework that formulates trajectory planning as a non-convex optimization problem and solves it using sequential convex programming. Unlike CHOMP, which uses gradient descent and repels the trajectory from obstacles, TrajOpt uses inequality constraints to strictly enforce collision avoidance while optimizing a cost function like path length or control effort. Analogy: Imagine planning a walking route where you want to minimize travel distance but have a strict rule that you must never step on the grass. Instead of just trying to stay away from the grass by feeling a repulsive force, you build a mathematical rule (constraint) that says "distance to grass >= 0" and solve for the shortest path that satisfies this rule. Example: A robotic arm assembly task where the gripper must move close to a fixture without colliding. TrajOpt optimizes the trajectory to minimize joint effort while enforcing constraints that keep all robot links at least 2 cm away from the fixture.

Trajectory Transformer​

A close relative of the Decision Transformer that also casts offline RL as sequence modeling, but models the whole trajectory — states, actions, and rewards — as one long token sequence, then plans by running beam search over the possible futures the transformer predicts, like searching for the highest-scoring continuation of a sentence. Where the Decision Transformer just emits the next action given a target return-to-go, the Trajectory Transformer can imagine and score several rollouts before committing, making it closer to a learned model used for planning. Analogy: instead of blurting the next move, it sketches several whole game plans in its head and picks the best.

Transformer​

The decoder-only / encoder-only / encoder-decoder architecture built from attention + MLP blocks

TransformerEngine​

NVIDIA's open-source library that automates safe FP8 training and inference on Hopper and Blackwell GPUs — it picks per-tensor scales each step so the low-bit math stays numerically stable. Like a thermostat for low-precision arithmetic: as values drift toward overflow or underflow, it nudges the scale to keep them inside the safe range. Drop-in transformer layers wrap your model and turn FP8 on without the user having to manage the scaling manually.

Transition function​

The part of an MDP that describes how the world moves, P(s' | s, a): the probability of landing in next-state s' given that you took action a in state s. It captures the environment's dynamics, including randomness — like a slippery floor where "go right" actually moves you right only 80% of the time. Also called the transition probabilities, the dynamics, or the model. When these probabilities are known, you can plan exactly with value iteration; when they are unknown, the agent must learn from sampled experience, which is what most of RL is about. Example: in a deterministic gridworld, P simply maps "in cell 5, move up" to "now in cell 2" with probability 1.

transpose​

The operation that flips a matrix across its main diagonal — every row becomes a column and every column becomes a row. A matrix M with m rows and n columns becomes Mᵀ with n rows and m columns, and the entry at row i, column j moves to row j, column i. In robotics, the Jacobian transpose Jᵀ maps forces at the end-effector back into equivalent joint torques — the mathematically dual direction of what J does (joint velocities → end-effector velocity). In PyTorch, .T and torch.transpose perform this flip by rewriting memory strides rather than copying data, so the result shares storage with the original array but may be non-contiguous.

Tree-of-Thoughts​

A reasoning method that explores several partial solutions at once as branches of a tree, scores them, and expands only the promising ones — like working through a maze by trying multiple paths and backing out of dead ends instead of committing to the first turn.

Triage​

Sorting cases by what each one needs, borrowed from emergency-room medicine where a nurse classifies arriving patients by severity before any doctor sees them. In LLM evaluation, hallucination triage means sorting model answers into useful buckets — correctly answered, correctly abstained ("I don't know"), confidently wrong (hallucination) — so each rate can be measured separately, instead of collapsing everything into one "accuracy" number that hides which failures are dangerous.

Triangular weights​

The triangle-shaped set of multipliers each filter in a mel filterbank uses to blend nearby STFT frequencies into one band. A filter's weight rises linearly from 0 up to 1 at its center frequency and falls back to 0 at its edges, so frequencies near the center count fully and those at the edges barely count — and neighboring triangles overlap so no frequency is dropped. Like a dimmer switch that is brightest in the middle of each band and fades to off at the borders, smoothly handing off to the next band. Example: a triangle centered at 500 Hz might weight 480 Hz at 0.8, 500 Hz at 1.0, and 520 Hz at 0.8, while 400 Hz and 600 Hz get 0; the band's value is the weighted sum of those frequencies' energies.

Triangulation​

Pinpointing a 3D location by intersecting two (or more) lines of sight to it. Each camera that sees a point defines a ray — "the point lies somewhere along this line" — and where two such rays from different viewpoints cross is the point's position in space. It is the geometric step that turns a matched pair of pixels in a stereo image into an actual distance. Analogy: two lighthouses each give you a compass bearing to the same ship; draw both bearings on a chart and the ship is exactly where the two lines meet. The name comes from the triangle formed by the two viewpoints and the target.

Tripwire​

A cheap, fast check whose only job is to sound the alarm the instant something goes wrong — named after the thin wire that, when stepped on, sets off a trap or flare. In model deployment a quick metric like perplexity is used as a tripwire: it won't tell you what broke, but it spikes the moment quality drops, so it catches a bad build before the slower, fuller tests even run.

Triton​

An open-source, Python-like programming language and compiler developed by OpenAI for writing highly optimized GPU kernels. Triton allows developers with no C++/CUDA expertise to write high-performance GPU code by abstracting away warp scheduling, register allocation, and memory coalescing, while still providing control over block-level memory staging. Analogy: A block-building game where you design rooms (block-level operations) and let the computer handle laying the individual bricks and wiring (warp scheduling and registers). Example: Writing a softmax kernel in Triton allows you to load rows of data into block pointers and use simple APIs like tl.sum, which Triton's compiler compiles into highly optimized GPU machine instructions.

Triton Inference Server​

NVIDIA's production server for hosting models behind an HTTP/gRPC API, with batching and multi-model support; unrelated to the Triton kernel language despite the shared name.

TRPO​

Trust Region Policy Optimization — the policy-gradient algorithm that PPO later simplified. Its goal is to take the largest improving step the data justifies while never moving the policy so far that performance collapses, and it enforces this with a hard trust region: the new policy must stay within a fixed KL-divergence budget of the old one. Solving that constrained step exactly is expensive — it needs a conjugate-gradient solver to approximate the curvature plus a backtracking line search to make sure the constraint actually holds — which is why TRPO works but is awkward to implement. PPO keeps the same "don't step too far" idea but swaps all that machinery for a one-line clip of the importance ratio, achieving comparable results with far less code, which is why TRPO is now mostly of historical and pedagogical interest.

Trust region​

The region around your current solution that you trust a simple approximation to be accurate within — so you only allow your next step to land inside it. In policy-gradient RL the worry is that a single large update, taken on a noisy estimate, overshoots and wrecks a working policy; the fix is to cap how far the policy may change per update. TRPO does this with an explicit constraint on the KL divergence between the old and new policy (the formal trust-region radius), while PPO approximates the same effect by clipping the importance ratio so updates outside a small band earn no further reward. Analogy: trusting your map only out to a short radius and refusing to march beyond it in one go — past that edge the map may be wrong, so you re-survey before stepping again.

TTFT​

Time to produce the first token — the elapsed time from when a request arrives at the server until the model returns its first output token, dominated by prefill plus any queue wait. Like a restaurant's "time until your drink arrives" — felt separately from the rest of the meal, and the first thing the user actually notices.

Twin critics​

A fix for the overestimation bias that plagues value-based continuous-control, introduced by TD3 and adopted by SAC. Instead of one critic Q(s, a), learn two independent ones and, when forming the learning target, use the smaller of their two estimates (min(Q₁, Q₂)). Why the minimum: a single critic's random errors sometimes make an action look better than it is, and a greedy update will chase that inflated value; requiring two independently-trained critics to both rate an action highly makes a shared overestimate far less likely, and taking the min deliberately leans pessimistic. It is the continuous-action cousin of Double DQN, which splits action selection from evaluation for the same reason. Analogy: trust a price only if two independent appraisers both agree it is high — and when they disagree, believe the lower one.

U-Net​

An encoder-decoder network whose name comes from its U shape: the left arm shrinks the image down to a small, abstract summary while the right arm builds it back up to full size, with skip connections that hand each down-sampling layer's detail straight across to its matching up-sampling layer. Those skips are what let it keep fine pixel detail while still reasoning about the whole image, which is why it became the standard backbone for diffusion models (before DiT brought in transformers). It was originally invented for medical-image segmentation.

UCB​

Upper Confidence Bound — the classic optimistic exploration rule. For every option you track an average payoff and a confidence interval around it (roughly, "the true value is somewhere in this range"), then always pick the option whose interval reaches highest — its optimistic upper end — rather than the one whose average is best. The usual form is mean(a) + c·√(log t / N(a)): the second term grows when an option has been tried few times (N(a) small) and shrinks as evidence accumulates, so a rarely-tried option is automatically treated as promising until it proves otherwise. That is optimism in the face of uncertainty written as one line of arithmetic, and the 1/√N shape is exactly the one that reappears in count-based exploration bonuses. Analogy: choosing between a dish you have ordered twenty times (you know it's a 7/10) and one you tried once (it might be a 9) — UCB says try the unknown one, because being wrong costs one meal and being right buys many.

UCF-101​

A widely used action-recognition video dataset from the University of Central Florida of about 13,000 short YouTube clips spanning 101 human-action categories (playing guitar, applying makeup, bench-pressing, and so on); the "101" is simply the number of action classes. It is small and low-resolution by modern standards, which is exactly why it became a common testbed for early video generation — you can train on it without a data-center. It shows up constantly in pre-diffusion video-GAN papers as the dataset everyone reported numbers on.

UKF​

Unscented Kalman Filter (UKF) is an advanced version of the Kalman Filter that handles nonlinear systems without needing to calculate derivatives or Jacobians. Instead of linearizing a nonlinear function (like the EKF does), the UKF represents the probability distribution using a small, carefully chosen set of sample points called sigma points. It passes these sigma points directly through the true nonlinear equations and computes the mean and covariance of the transformed points to form the new state estimate. This capture of nonlinearity is mathematically accurate up to the third-order Taylor series expansion, making it significantly more accurate than the EKF for highly nonlinear systems while sharing the same computational complexity.

Analogy: Estimating how a crowd of people walking through a winding maze will spread out. An EKF assumes the maze is a straight line locally and projects the crowd's path. A UKF places a few representative "scouts" (sigma points) at the front, middle, and edges of the crowd, has them walk through the actual winding maze, and then reconstructs where the rest of the crowd must be based on where the scouts ended up.

Underflow​

A condition in computer arithmetic where the result of a calculation is a non-zero number that is smaller than the smallest value the floating-point format can physically represent, causing the value to be rounded to zero. This is a common issue when training deep learning models in lower precision (like float16), where extremely small gradients round to zero and halt learning. Analogy: A ruler that can only measure down to millimeters. If you try to measure something that is a fraction of a micrometer, the ruler cannot resolve it and reads it as exactly zero. Example: In FP16 training, if gradient values fall below the minimum representable limit (~6e-8), they round to zero due to underflow, stalling the optimizer's updates. This is typically fixed using loss scaling (via GradScaler).

Undervolting​

Undervolting is a hardware management technique where the operating voltage of a processor (like a CPU or GPU) is lowered below the manufacturer's default settings while keeping its clock frequency unchanged.

  • Why it matters: Modern processors are supplied with more voltage than they strictly need to ensure stability across all manufactured chips (binning). However, higher voltage increases heat generation and power consumption exponentially. By carefully reducing the voltage, developers running multi-GPU workstations can dramatically reduce power draw (by 50 to 100 watts per card) and operating temperatures, preventing thermal throttling and allowing stable, continuous model training and inference without exceeding power or cooling limits.
  • How it works: Chip manufacturers set a voltage-frequency curve that defines how much voltage is delivered for any given clock speed. Undervolting offsets this curve downward. As long as the voltage is sufficient to keep the transistors switching reliably at that frequency, the processor behaves identically but runs cooler and consumes less power. If the voltage is set too low, the system becomes unstable and crashes, requiring fine-tuning to find the optimal stable voltage.
  • Analogy: Imagine a restaurant that keeps its kitchen thermostat set very high to guarantee all food is cooked rapidly, wasting a lot of energy and heating up the room. Undervolting is like lowering the temperature to the exact level needed to cook the food perfectly: the dishes are prepared just as fast, but the kitchen stays much cooler and the energy bill is significantly lower.
  • Example: Using command-line tools like nvidia-smi to set a power limit (e.g., restricting an RTX 4090 from 450 watts down to 300 watts) or shifting the voltage-frequency curve. This reduces heat and power consumption by ~30% while losing less than 5% of peak compute throughput.

Unicycle model​

The simplest useful model of a wheeled robot: a single point that can drive forward at a commanded speed and spin in place at a commanded turning rate, but can never slide sideways. Its state is just position and heading (x, y, θ), and two short equations turn the speed and turn-rate controls into how that state changes. The "cannot move sideways" rule is a nonholonomic constraint—the same reason a car has to do a three-point turn instead of gliding into a tight parking spot. It is the go-to model for differential-drive robots (two independently driven wheels) and the starting point for path-tracking controllers; its cousin the kinematic bicycle model adds an explicit steering angle for car-like vehicles. Analogy: an office chair you can only roll straight ahead and pivot — to reach a spot beside you, you must turn first, then roll, never drift across.

Update-to-data ratio​

How many gradient updates an algorithm performs per environment step collected — often abbreviated UTD. It is the dial that converts compute into sample efficiency, and it is the single biggest structural difference between the two families of deep RL. SAC and TD3 run at UTD ≈ 1 (one update per step, replaying old transitions from the buffer), so they squeeze a lot of learning out of few samples but pay a full backward pass every single step. PPO sits far below 1 — it collects thousands of steps, then does a few dozen updates on that batch and discards it — so each sample teaches it less, but it moves through samples enormously faster. Turning UTD up (to 10 or 20, as REDQ and DroQ do) buys still better sample efficiency at a steep compute cost, and eventually the critic starts overfitting the replayed data.

  • Analogy: Two students with the same textbook. One re-reads each chapter ten times before moving on (high UTD: learns a lot per page, slow to finish the book); the other skims each chapter once but gets through ten books (low UTD: learns little per page, sees far more pages).
  • Example: On a real robot, where every sample costs wall-clock seconds and wear on the hardware, you want high UTD. In a fast simulator you can fork across 64 cores, low UTD wins on wall-clock time even though it "wastes" samples.

URDF / MJCF / USD​

The standard text file formats that describe a robot to software: its links (the rigid pieces), joints (how the pieces connect and move), shapes, masses, and sensors. URDF (Unified Robot Description Format) is the ROS world's XML format; MJCF is MuJoCo's richer XML, with better contact and actuator modeling; USD (Universal Scene Description) is NVIDIA/Pixar's format used by Isaac for photorealistic scenes. Analogy: a URDF is the robot's blueprint — read it once and a simulator or kinematics library knows the whole skeleton without you hand-coding a single joint. They differ mainly in how much physics detail they capture, which is why the same robot is often shipped in more than one format.

User turn​

One message a user sends in a chat conversation, paired with the model's reply (the assistant turn). A back-and-forth between user and assistant is a sequence of alternating turns, all under the same opening system prompt. In typical traffic, the system prompt is long and fixed while each user turn is short and varies — which is exactly the pattern a prefix cache exploits.

V2V​

Video-to-Video: transforming an existing video into a new one while keeping its motion and timing — for example restyling it into a cartoon, or re-rendering it conditioned on per-frame depth or pose. The hard part is temporal consistency: editing each frame independently makes the result flicker, so V2V methods share information across frames. Contrast with image-to-video (one image in) and text-to-video (text only).

Vanishing gradients​

A problem during training where gradients become extremely small, effectively preventing the weights from changing their value and stalling the learning process.

VAE​

Variational Autoencoder — an autoencoder whose encoder outputs not a single point but a small cloud of possibility (a mean and a spread) for each input, and whose decoder samples from that cloud to rebuild the image. Training on the ELBO presses those clouds to fit neatly under one standard bell-curve shape, so afterwards you can draw a brand-new point from that shape and decode it into a fresh image the model has never seen. That sampling ability is what makes a VAE a generative model rather than just a compressor.

Validation loss​

The loss measured on held-out data the model was not trained on; the honest signal of how well training is generalizing.

Value clipping​

Applying PPO's clipping trick a second time, to the value function: the critic's new prediction is not allowed to move more than ε away from what it predicted when the batch was collected. It is detail #9 of the 37 PPO implementation details, mirroring the policy's clipped surrogate objective and motivated by the same worry — that several epochs of gradient steps on one batch will drag the network too far.

  • Worth knowing: Of all the details in the list this is the one whose usefulness is most in doubt. The paper that catalogued the 37 could find no consistent benefit from it, and later ablation studies have reported it hurting as often as helping. It survives in reference implementations largely because it was in the original OpenAI code, which is an honest and slightly uncomfortable summary of how a good deal of deep-RL practice is transmitted.

Value function​

How much total future reward you should expect, used as a score for situations. The return it averages is the discounted sum of all rewards from now on (r₀ + γr₁ + γ²r₂ + …, each step shrunk by the discount factor γ). Two flavors: the state-value V(s) is the expected return starting from state s and following your policy; the action-value Q(s, a) is the expected return if you first take action a and follow the policy afterward. The difference Q(s, a) − V(s) is the advantage — how much better that one action is than the policy's average. The Bellman equation is the recursive rule these values must satisfy.

Value iteration​

A dynamic-programming algorithm for solving an MDP when the transition function and reward function are known: start with any guess of the value function, then repeatedly apply the optimality Bellman backup — set each state's value to the best action's "reward now plus discounted next-state value" — until the values stop changing. Because that backup is a contraction mapping, the values are guaranteed to converge to the optimum V*; reading off the best action in each state then gives the optimal policy. It differs from policy iteration by never evaluating a policy to completion — it folds improvement into every sweep. Unlike policy iteration, which perfectly measures a policy's value before changing it, value iteration takes a tiny step of improvement after every single evaluation sweep. Think of finding the best route to work: policy iteration is like driving one specific route every day for a month to know its exact time before trying a new one, while value iteration is like driving a route once and immediately updating your guess for the best path.

Value network​

The helper network (the "critic") in some RL algorithms that estimates the value function — its best guess of how much future reward a situation is worth — so the policy can tell whether an action turned out better or worse than expected. PPO trains one alongside the policy, which roughly doubles the networks held in memory; GRPO skips it entirely by comparing each sampled answer to the group's average instead, which is what makes it cheaper.

Vanilla​

The plain, unmodified, baseline version of a model or algorithm — no special improvements or extra tricks, just the original idea as first described. Like ordering plain vanilla ice cream with no toppings: it is the default flavor before anyone adds anything extra. In machine learning, "vanilla VAE" means the original VAE from the 2013 Kingma & Welling paper, before later work added hierarchical latents, β controls, or other refinements. Comparing the vanilla version to improved variants is the clearest way to measure what each addition actually buys.

Variable resolution​

The ability of one trained model to generate clips at many different sizes, durations, and aspect ratios, instead of being locked to the single resolution it trained on — the headline claim of Sora. It is possible because a DiT processes a sequence of spatiotemporal patches rather than a fixed-size grid, so feeding it more or fewer tokens naturally yields a taller, wider, or longer video; 3D RoPE makes this work because it encodes each token's position as a rotation that extrapolates to lengths and shapes never seen in training, instead of a fixed lookup table that would have no entry for a new position. In practice the model is trained across several aspect-ratio buckets so it has seen a range of shapes. Like a printer that can lay the same document out on A4, letter, or a wide banner without re-typesetting it.

VBench​

A comprehensive open benchmark suite for text-to-video models that, instead of boiling quality down to one number, scores generated clips along many separate dimensions — subject consistency, motion smoothness, aesthetic quality, text–video alignment, and a dozen more — and reports each one. The idea is that "is this video good?" has several independent answers, so a single score hides whether a model is, say, beautiful but jittery; splitting the score apart tells you exactly what to fix. To produce each number it runs many generated clips through purpose-built detectors (for example, a tracker to check an object stays the same shape, an optical-flow measure for smooth motion, a CLIP-style match for text alignment) and averages the results into a per-dimension percentage. It is the closest thing the field has to a standard video-generation leaderboard.

Vectorized environment​

Running many independent copies of an environment at once and stepping them in lockstep, so that a single batched action array drives all of them and a single batched observation array comes back. It is detail #1 of the 37 PPO implementation details and the reason on-policy algorithms are practical at all.

  • Why it matters — and it is not mainly speed: The obvious benefit is that the network's forward pass is amortized over N environments instead of one. The deeper benefit is statistical. A gradient step assumes its batch is a sample of the state distribution, but consecutive states from a single environment are the same situation a fraction of a second apart, and are therefore nearly identical: a 128-step rollout from one copy is closer to one state photographed 128 times than to 128 samples. Stepping 8 copies gives a batch spanning 8 genuinely unrelated situations, which decorrelates the gradient in exactly the way experience replay does for off-policy methods — by a completely different mechanism, since an on-policy algorithm is not allowed to keep old data at all.
  • Note: "Synchronous" versions step the copies one after another in a single process; "asynchronous" ones put each in its own process. The distinction is about wall-clock, not about learning.

Verifier​

A program that automatically checks whether an answer is correct — running unit tests, or comparing to a known math result — giving the exact, unhackable reward that RLVR trains on.

Very Deep VAE​

A hierarchical VAE (Child, 2021) that scales to dozens of stacked latent variable groups — far more layers than earlier models. Each group only handles a thin slice of the work, with residual-like parameterizations keeping gradients flowing through the depth. Like adding so many floors to a building that no single floor needs to bear much weight, it achieved strong image generation quality, showing that deeper hierarchies can capture richer structure than shallow ones.

Video-CFG​

Applying classifier-free guidance (CFG) to a video model that has more than one condition — typically a text prompt and a conditioning image — by giving each condition its own guidance scale instead of one shared dial. You can then push text adherence and image faithfulness independently: strong text guidance to match the prompt, separate image guidance to stay locked to the conditioning frame. The catch unique to video is that turning either scale too high amplifies per-frame detail at the cost of smooth change between frames, so the clip's motion begins to flicker or its colors over-saturate — guidance strength trades against temporal smoothness. This is why production video models expose several guidance knobs rather than the single one image models use.

Video codec​

The set of rules for compressing video into a small file and decompressing it back into frames — "codec" is short for coder–decoder, which is literally what it does. Raw video is enormous (a few seconds can be hundreds of megabytes), so almost all real video is stored compressed; codecs exploit the fact that neighboring frames barely change. Analogy: a codec is like shorthand for a movie — instead of writing every frame in full, it writes "same as the last frame, but this corner moved." Examples include H.264 (the universal default) and AV1 (smaller files, slower to decode); the codec lives inside a media container like .mp4.

view​

A no-copy alias that shares storage with its source; requires a contiguous-compatible layout

VIO​

Visual-Inertial Odometry (VIO) is a state-estimation method that fuses visual data from one or more cameras with inertial data from an IMU to estimate how a robot moves through space. By combining the high-rate, short-term accuracy of the IMU (which measures acceleration and angular velocity) with the lower-rate, long-term stability of the camera (which tracks visual features across frames), VIO provides a highly robust, high-frequency estimate of the robot's trajectory. This fusion resolves the scale ambiguity inherent in monocular cameras and constrains the rapid quadratic drift of pure dead reckoning.

Analogy: Walking down a dimly lit hallway. If you close your eyes, you can feel your steps and turns for a moment, but you quickly become disoriented and lose track of your exact position (pure inertial navigation drift). If you open your eyes but look at a blank, textureless wall, you cannot judge your speed or distance. By keeping your eyes open and tracking visual landmarks (doors, frames) while feeling your body's motion, you can walk smoothly and precisely (visual-inertial fusion).

Video GAN​

A GAN adapted to produce short video clips instead of single images: the generator outputs a whole stack of frames at once and the discriminator judges whether the motion, not just each individual frame, looks real. The early family — VGAN, TGAN, MoCoGAN, DVD-GAN, and StyleGAN-V — produced only short, low-resolution clips and suffered badly from mode collapse (the generator falling back on a few safe outputs). Each pushed one idea: MoCoGAN separated content from motion, DVD-GAN was the first to reach plausible quality, and StyleGAN-V applied StyleGAN's latent-space tricks to video. The whole approach was largely abandoned around 2023 once diffusion proved both sharper and far more stable to train at scale.

ViT​

Vision Transformer — a transformer that classifies or encodes images by first chopping them into a grid of small square patches (patchification), turning each patch into one token, and then treating the picture exactly like a sentence of words. Because a plain transformer has no built-in notion of "next to" the way a CNN does, a ViT adds a learned positional embedding to each patch (a small vector that says "I am the patch at row 3, column 5") and usually prepends a CLS token whose output becomes the whole-image summary. Like reading a mosaic tile by tile, left to right, instead of taking in the whole wall at once — and, given enough data, this beats CNNs because the model can relate any tile to any other from the very first layer instead of only neighboring pixels. The "B/16" in a name like ViT-B/16 means a Base-size model with 16×16-pixel patches.

VLA​

Vision-Language-Action (VLA) models are multimodal neural networks that map visual inputs (camera images) and natural language instructions directly to continuous or tokenized robot control actions.

  • Why it matters: Traditional robot learning systems require separate modules for object detection, high-level planning, and motor control, which can lead to compounding errors. VLAs unify these components into a single transformer network, allowing robots to understand complex commands and respond to visual cues in real-time.
  • How it works: A VLA is typically initialized from a pretrained Vision-Language Model (VLM) that already understands images and text. It is then trained on large-scale robotic datasets of trajectory demonstrations (such as Open X-Embodiment) where the action commands (like gripper translation and rotation) are represented as additional tokens in the model's vocabulary. The model takes a prompt like "pick up the yellow block" along with the current camera image, and generates the action tokens autoregressively.
  • Analogy: Imagine an experienced video gamer who is watching a livestream of a game. A natural language prompt is given by the chat, like "open the chest." The gamer looks at the screen (vision), reads the instruction (language), and presses the exact sequence of controller buttons (action) to perform the task. A VLA behaves like this gamer, directly translating sight and instruction into physical control.
  • Example: Robotics Transformer 2 (RT-2) is a VLA model trained on diverse manipulation datasets. When shown a kitchen counter with a toy dinosaur and a toy building block, and commanded to "move the dinosaur to the block," RT-2 processes the visual scene and language command, then outputs joint velocity trajectories to guide a robot arm's gripper to pick and place the dinosaur.

vLLM​

An open-source, high-throughput large language model serving engine designed for fast and memory-efficient inference.

  • Why it matters: LLM token generation is heavily memory-bound, bottlenecked by memory transfer speeds. vLLM addresses this by optimizing how key-value data is managed in memory, dramatically increasing the number of users a single GPU can support.
  • How it works: vLLM implements PagedAttention to partition the KV cache into non-contiguous blocks, eliminating memory fragmentation. It also uses continuous batching to dynamically schedule incoming and outgoing requests at each token generation step.
  • Analogy: A library using a dynamic catalog. Instead of reserving a huge, fixed block of empty shelves for every visitor who might borrow books (which wastes massive space), the library stores books in small boxes wherever there is space and looks them up using a dynamic catalog map.
  • Example: Deploying a 70B parameter model with vLLM allows a service to process dozens of concurrent user prompts with high throughput and low latency, preventing out-of-memory errors.

Visual odometry​

Estimating how a camera has moved through space using only its own image stream, by tracking features from one frame to the next and computing the camera motion that best explains how they shifted. "Odometry" means measuring travel — as a car's odometer counts distance — and here the camera is the odometer. Because each frame's motion is measured relative to the one before, errors compound into drift, so over a long path the estimate slowly diverges from the truth unless loop closure corrects it. Fusing the camera with an IMU gives visual-inertial odometry, which drifts far less. Analogy: finding your way across a field by watching how fixed landmarks slide past as you walk, rather than counting your steps.

Visuomotor policy​

A control policy that maps raw visual inputs (such as camera images, depth maps, or pixel masks) directly to robot motor actions (such as joint velocities or end-effector forces), without using an intermediate step of explicit 3D object reconstruction. Visuomotor policies are typically trained end-to-end using imitation learning (e.g., diffusion policies) or reinforcement learning.

  • Analogy: When catching a ball, you do not measure its 3D coordinates, calculate its parabolic trajectory, and then plan your hand's path. Instead, you look at the ball and adjust your arm's movement in real-time based on where the ball appears in your field of vision. This direct connection between sight and action is a visuomotor policy.
  • Example: A robot arm in a simulator trained via deep reinforcement learning to pick up a toy. The input to the neural network is the raw camera image, and the output is the direct motor command to the joints.

VLM​

Vision-Language Model — a model that takes an image (usually plus a text question) in and produces text out, such as a caption or an answer. The standard build is middle fusion: a pretrained image encoder turns the picture into feature vectors, a small projector maps those into the token space of a pretrained language model, and the language model then "reads" the image alongside the words. LLaVA is the canonical open example; Qwen2-VL and Gemini are larger ones. Analogy: a sighted assistant describing a photo to a brilliant writer who cannot see it — the encoder does the looking, the language model does the talking. Unlike a native multimodal model, a plain VLM only outputs text; it cannot generate images.

Vocabulary​

The fixed set of tokens a tokenizer can produce, each with an integer ID; its size trades tokens-per-document against embedding matrix size

Volta​

NVIDIA's 2017 GPU architecture (V100) and the first generation to ship Tensor Cores, the dedicated matmul units that made deep-learning training dramatically faster. Subsequent generations — Turing, Ampere, Hopper, Blackwell — kept Tensor Cores and added support for ever-lower-precision formats. Named after the Italian physicist Alessandro Volta.

VP / VE SDE​

The two standard ways to define the forward noising process of a diffusion model, each written as an SDE. Variance-Preserving (VP) — the family DDPM uses — shrinks the original signal as it adds noise so the total variance stays around 1 the whole way. Variance-Exploding (VE) — used by the early score-based models — leaves the signal untouched and simply piles on ever-larger noise, so the variance grows without bound. They are mathematically interconvertible and reach similar quality, but differ in numerical conditioning and in which samplers behave well.

VP9​

A royalty-free video codec built by Google as a free alternative to the patent-licensed H.264. It compresses noticeably better than H.264 — smaller files at the same quality — and is the codec behind most YouTube streams and many .webm files, though it has since been largely overtaken by the newer, even-smaller AV1. Analogy: VP9 is to H.264 what a tighter, license-free ZIP format is to an older paid one — it squeezes the video smaller with no license fee, at the cost of more work to decode it back into frames. Example: a clip saved as a VP9 .webm is usually a good bit smaller than the same clip as an H.264 .mp4, but slower to unpack into frames during training.

VQA (Visual Question Answering)​

The task of answering a natural-language question about an image — "How many people are in this photo?", "What color is the car?" — where the model must read the picture and the words together to respond. It is the classic benchmark for multimodal understanding: unlike captioning, which can lean on generic descriptions, a question pins the model to one specific detail it cannot fake. Think of an open-book exam where the "book" is a photograph and each question forces you to actually look. Most VLMs are evaluated on VQA datasets, and it is the natural small task on which to compare fusion methods like concatenation versus cross-attention.

VQ-GAN​

A VQ-VAE trained with two extra signals so its reconstructions look sharp instead of blurry: a perceptual loss that compares images by their high-level features rather than exact pixels, and a patch discriminator — a small critic from the GAN world that scores whether each local region of an image looks real. The combination pushes the decoder to commit to crisp, specific details. This is the recipe Stable Diffusion's VAE descends from.

VQ-VAE​

Vector-Quantized VAE — an autoencoder whose latent code is forced to be discrete. Instead of letting the encoder output any continuous numbers, each patch of the image must be described using an entry chosen from a small fixed codebook, like painting only with the colors in a numbered paint set. Turning an image into a grid of these code indices lets you treat it as a sequence of tokens and generate it with the same tools used for language. It is trained with a straight-through estimator so gradients can flow through the non-differentiable lookup.

VRAM​

Video Random Access Memory (VRAM) is high-speed dedicated memory located on a graphics card (GPU) that is used to store the model's parameters (weights), inputs, activations, and optimizer states during computation.

  • Why it matters: Accessing system RAM (via PCIe) is far too slow for deep learning computations, which perform billions of operations per second. VRAM resides directly on the GPU card, providing the extreme memory bandwidth (e.g., up to 8.0 TB/s on a Blackwell GPU) required to feed data to the Tensor Cores without stalls. The total capacity of VRAM is the primary constraint on the size of the models you can train or serve on a single device.
  • How it works: When you load an AI model, its weights are copied from your storage disk into VRAM. During the forward pass, the intermediate calculations (activations) are also stored in VRAM so they are ready to be used in the backward pass to calculate gradients. If the combined size of the weights, activations, and optimizer states exceeds the total VRAM capacity, the system crashes with an "Out of Memory" (OOM) error, requiring optimizations like quantization, gradient checkpointing, or paged optimizers.
  • Analogy: Imagine a chef working in a kitchen. System RAM is like a walk-in freezer down the hall (slow to access). VRAM is the chef's countertop (instant access). If the chef can fit all the ingredients for the meal on the countertop, they can cook extremely fast. If they run out of space on the countertop (run out of VRAM), they must slow down to fetch ingredients from the freezer one-by-one or stop cooking altogether.
  • Example: Quantizing a 70-billion parameter model from 16-bit to 4-bit reduces its weight footprint in VRAM from ~140 GB to ~35 GB, allowing the model to be served on a single workstation with two 24 GB GPUs instead of requiring a cluster of datacenter GPUs.

W and W+ latent spaces​

The editable latent spaces inside StyleGAN that dictate how images are generated and controlled. W (The Master Remote): The intermediate space the input noise is first mapped into. Because StyleGAN's training thoroughly disentangles it, it acts like an intuitive master remote control—turning a single "dial" in W smoothly changes one specific attribute (like age) without altering the rest, making it perfect for editing. W+ (The Individual Room Panels): A relaxed version of W where each layer gets its own independent W code instead of sharing just one. Like abandoning the master remote for highly detailed control panels in every single room, it is harder to tweak one simple trait, but it can represent and reconstruct a specific, complex image much more precisely. This is the space GAN inversion usually targets when trying to match a real-world photo.

Walker2d​

A MuJoCo continuous-control task: a two-legged "walker" robot, confined to a vertical plane, that must learn to walk forward without falling — unlike HalfCheetah, it can topple, which makes it noticeably harder. It has 6 motorized joints (continuous torques) and a state of joint angles and velocities. It sits in the middle of the standard MuJoCo difficulty ladder, above HalfCheetah and below Ant and Humanoid. Provided as Walker2d-v4 in Gymnasium.

Warmup​

The opening phase of training where the learning rate ramps up from near zero to its peak, stabilizing the first noisy updates

Warp​

A group of exactly 32 threads that an NVIDIA GPU runs together in perfect lockstep — every thread in the warp executes the same instruction at the same moment, just on its own piece of data. This is the SIMT model, and a warp is the smallest unit of work an SM actually schedules: the hardware never runs a lone thread, it always dispatches them 32 at a time. Analogy: Picture a rowing boat with 32 rowers who must all pull on the same stroke called by one coxswain. The command is a single "row!" (one instruction), but each rower drives their own oar through their own patch of water (their own data). They're fast precisely because nobody steers independently — one call moves all 32 in sync. Why the lockstep matters: Because the whole warp shares one instruction, an if-else that sends some threads one way and the rest the other forces the GPU to run both paths in turn, with the idle threads sitting out each time — the slowdown known as warp divergence. Keeping all 32 threads on the same path is a key trick for fast GPU code. And while one warp waits on slow memory, the SM swaps in another ready warp to stay busy, which is exactly what occupancy measures.

Wasserstein GAN (WGAN)​

A GAN variant that replaces the original loss with the Earth Mover's Distance between the real and generated image distributions. The original loss gives almost no gradient once the discriminator wins, stalling training; the Earth Mover's Distance stays informative even when the two distributions barely overlap, so the generator keeps learning. It requires the critic to obey a Lipschitz constraint, enforced in the popular WGAN-GP version by a gradient penalty.

Watchdog​

A safety utility (implemented in hardware or software) that monitors the health of a system by expecting a periodic heartbeat signal—commonly called "kicking" the watchdog. If the system hangs, crashes, or fails to send this heartbeat within a predefined timeout window, the watchdog triggers a fail-safe action, such as shutting down motors, applying brakes, or rebooting the controller. Analogy: A dead man's switch on a train. The operator must hold or periodically press a lever to prove they are conscious; if they release it (meaning they are incapacitated), the train immediately applies emergency brakes to prevent a crash. In robotics, a software watchdog is crucial to intercept control software lockups and safely trigger an emergency stop before the robot causes physical harm.

Watermarking​

Hiding an invisible, machine-detectable signal inside a generated image so software can later confirm "this was made by AI" without changing how the picture looks to a human. The signal can be stamped into the pixels after generation (a faint patterned perturbation) or baked into the model's own sampling — Google's SynthID nudges pixel values in a learned pattern, and Tree-Ring plants a ring-shaped mark in the initial noise that survives diffusion and is recovered by inverting the generation process. A matching detector then reads the mark back out and reports a confidence score. Like the watermark pressed into a banknote: invisible in normal use, obvious under the right lamp, and hard to forge or scrub off. The built-in tension is robustness vs invisibility — a mark strong enough to survive cropping and JPEG compression is harder to keep imperceptible. Example: generate 1,000 images, run the detector, and it should flag nearly all of them while leaving real photos unflagged.

WBC​

Whole-Body Control (WBC) is a control framework used in legged robots and manipulators that coordinates all available joints simultaneously to achieve multiple, sometimes competing, physical tasks (like keeping the body balanced while reaching for an object).

  • Why it matters: A robot dog has 12 or more motors, and a humanoid has 20-30+. Coordinating them individually leads to unstable, jerky movements. WBC looks at the robot as a single unified system, allowing it to maintain balance and respect safety limits (like joint ranges and foot friction) while executing complex movements.
  • How it works: It formulates control as a Quadratic Program (QP), a mathematical optimization problem solved hundreds of times per second (e.g., 500 Hz). The QP solver finds the exact joint torques that achieve high-priority goals (like not falling over) and lower-priority goals (like waving a hand) while satisfying hard constraints (like feet not slipping on the floor).
  • Analogy: Imagine standing on a moving subway train. To keep your balance, you don't just move your ankles; your knees bend, your hips sway, your core tenses, and you might reach out to grab a handrail. Your entire body coordinates automatically to keep you upright while you focus on reading a book.

WebDataset​

A library that streams training data directly from sharded .tar archives, avoiding the need to unpack millions of individual files.

Weight decay​

A regularization technique that shrinks model parameters toward zero at each update step, discouraging large weights and improving generalization

Weights​

The main, larger group of learned parameters in a layer — the W in y = xW + b — that decide how strongly each input affects each output. Think of the volume sliders on a soundboard: a big weight turns an input way up, a near-zero weight mutes it, and a negative weight flips it. During training the optimizer keeps nudging these sliders to lower the loss, and they make up the bulk of a model's size.

WGAN-GP​

Short for Wasserstein GAN with Gradient Penalty — the most popular and reliable recipe for training a Wasserstein GAN. A Wasserstein GAN only works if its critic obeys a Lipschitz constraint (its output can't change too fast). The original WGAN enforced that bluntly, by clipping every critic weight back into a fixed range after each step — a heavy-handed move that often crippled the model's quality. WGAN-GP replaces the clipping with a gentle gradient penalty that simply nudges the size of the critic's gradient toward 1, which keeps training far more stable. Like keeping a car at the speed limit with a smooth governor that eases off the gas, instead of a hard rev-cut that jerks the whole engine every time you nudge past it.

Whisper​

OpenAI's open speech-recognition model — an encoder-decoder transformer that turns a mel spectrogram of speech into text, trained on 680,000 hours of multilingual audio scraped from the web. Its encoder digests the audio into rich embeddings and its decoder writes out the words, so one model handles transcription, translation, and language identification. Because that encoder learned such general audio representations, people often reuse just the encoder — freezing it and training a small head on top — as a ready-made audio feature extractor (much like a vision linear probe). The name evokes catching even quiet, whispered speech.

Windowed attention​

A cheaper form of attention that lets each token attend only to others inside a small local neighborhood (a "window") rather than to the whole sequence. In video, windowed spatiotemporal attention applies full joint space-and-time attention but only within small 3D boxes of nearby frames and pixels, so cost grows with the window size instead of the full T×H×W. It is the middle ground between cheap (2+1)D attention and expensive full spatiotemporal attention: you keep some direct space-time interaction but give up reach across the whole clip. Like reading a document through a small sliding window that shows only a few lines at a time — fast, but you cannot see the whole page at once.

Worker processes​

Background subprocesses that a DataLoader spawns to load and preprocess data in parallel with GPU computation.

Workhorse​

The dependable, go-to method that does the bulk of the everyday work in a field — not the flashiest, but the one practitioners reach for by default because it reliably gets the job done. Just as a workhorse on a farm pulls the heavy loads day in and day out, PPO earned the title in RLHF and the PID controller earned it in robotics.

World consistency​

Whether a generated video keeps a single, self-consistent world as it plays — the same room keeps the same layout, a character keeps the same clothes and face, and the lighting and geography do not silently contradict themselves from one moment (or shot) to the next. It is a step beyond per-frame quality: each frame can look great while the world drifts, which is the same drift problem that makes long videos fall apart. Together with object permanence it is one of the world-behavior criteria Sora's report names as still unsolved, and it is a facet of the broader physical plausibility challenge.

World Model​

A generative model that predicts the next state of an environment given the current state and an action — in plain terms, a video generator that also takes an action input. Run it in a loop, feeding each predicted frame back in as the new current state, and it becomes a learned simulator you can act inside: a human can play it like a game, a policy can train inside it by imagining rollouts (as DreamerV3 does), or a planner can search through it. A plain text-to-video model is just the special case where the action is empty. Real examples include Genie (playable worlds from web video) and GameNGen (a neural DOOM).

World-model rollout​

World-model rollout is the process of using a learned world model to simulate and generate a sequence of future environment states or video frames in response to a hypothetical sequence of actions.

  • Why it matters: Running experiments on physical hardware is slow, wear-inducing, and potentially dangerous. By rolling out actions in imagination, a robot can safely test hundreds of action sequences in a fraction of a second, selecting the best one before executing a single physical movement.
  • How it works: The agent selects a starting state (like the current camera image) and a sequence of proposed actions. The action-conditioned world model takes the state and first action to predict the next state. This predicted state is then fed back into the model along with the second action, repeating the process for a fixed horizon. The final generated state is compared against a goal target (like a goal image) to evaluate the sequence.
  • Analogy: Imagine a chess player thinking three moves ahead. They do not physically move the pieces on the board to test their strategy. Instead, they simulate the moves in their head: "If I move my bishop here, they will move their knight there, and then I can take their rook." Each of these mental simulations is a rollout.
  • Example: A robotic manipulator is tasked with folding a cloth. It uses a video world model to roll out various pushing movements. The planning system generates 100 random action sequences, runs them through the world model to predict the resulting cloth shape for each, and executes the first action of the sequence whose rollout matches the desired folded configuration.

WSD​

Warmup-Stable-Decay — a learning-rate schedule that warms up, holds the rate constant for most of training, then decays sharply at the end.

XLA​

Accelerated Linear Algebra — a domain-specific compiler backend developed by Google that optimizes machine learning computations. It accepts execution graphs from frameworks like JAX, PyTorch (via torch_xla), or TensorFlow, and compiles them into highly efficient machine instructions tailored for specific hardware accelerators like GPUs and TPUs. XLA's primary strength is kernel fusion, which combines multiple separate mathematical operations into a single execution step to avoid writing intermediate results back to slow global memory.

  • Analogy: Imagine a painter who is asked to paint a wall blue, then paint yellow stars on top of it, and then paint a red outline around the stars. A traditional execution framework is like painting the whole wall blue, waiting for it to dry (saving it to memory), then walking over to get yellow paint, and so on. XLA is like the artist planning ahead, mixing the colors on the palette, and painting the star with its outline in one unified brushstroke, saving trips back and forth and speeding up the work.
  • Example: Using torch_xla to run a training loop on a Google Cloud TPU, where XLA automatically fuses a sequence of LayerNorm, linear projection, and activation operations into a single custom hardware instruction.

YaRN​

Yet another RoPE extensioN method — a context-extension scheme that rescales rotation frequencies unevenly across dimensions to reach long contexts with minimal fine-tuning

Yaw​

The rotation of a vehicle or object turning left or right in a horizontal plane (like a car steering left or right on a flat road).

  • Why it matters: Controlling yaw determines the heading or direction a robot is facing. For a drone or a ground robot, changing its yaw changes its navigation heading toward a goal.
  • How it works: Yaw is one of the three Euler angles used to describe 3D orientation. It is rotation around the vertical (up-and-down) axis. In a quadrotor, yaw is controlled by increasing the speed of the two diagonally-opposite rotors spinning in one direction relative to the other two, generating a net torque that spins the body in place.
  • Analogy: Imagine shaking your head "no" (turning your face left and right while keeping your chin level). Your head is yawing.
  • Example: An autonomous drone uses its camera to find a target object and rotates its body (adjusting its yaw) until the target is centered in the camera's field of view.

YOLO​

YOLO stands for You Only Look Once — a family of real-time object detection models that process an entire image in a single forward pass to predict bounding boxes and class labels simultaneously. Unlike earlier two-stage detectors (which first propose candidate regions, then classify each one), YOLO frames detection as a single regression problem: it divides the image into a grid, and each grid cell predicts bounding boxes and class probabilities in one shot.

  • Why it matters: YOLO's single-pass design makes it fast enough for real-time applications — video surveillance, autonomous driving, robotics — where latency matters more than squeezing out the last fraction of accuracy. Later versions (YOLOv5, YOLOv8, YOLO11) improved accuracy to rival two-stage detectors while keeping the speed advantage.
  • Analogy: Imagine a teacher scanning a classroom in one quick glance and instantly noting "two students raising hands in the back row, one near the window" — that is YOLO. A two-stage detector is like first circling every desk that might have a hand up, then going back to check each circle individually.
  • Example: Deploying a YOLOv8 model compiled with TensorRT on a Jetson Orin for a warehouse robot that detects and counts packages on a conveyor belt at 30 frames per second.

ZeRO​

DeepSpeed's parameter/gradient/state sharding scheme — comparable to FSDP

Zero-conv​

A 1×1 convolution whose weights and bias all start at exactly zero, used by ControlNet to bolt a new branch onto a pretrained model without disturbing it. At initialization a zero-conv outputs nothing, so the new branch adds zero to the original network and the model behaves exactly as before — yet because the layer still receives gradients, it can gradually learn how much signal to pass through. Like wiring in a new tap that is turned fully off at first, then opened slowly as training discovers how much to let flow. This is what lets ControlNet train a fresh control signal without damaging the base model's existing quality.

Zero-shot​

Doing a task the model was never explicitly trained for, with zero task-specific examples shown at test time. The classic case is CLIP zero-shot image classification: instead of training a classifier head on labelled images, you write each candidate label as a short sentence — a prompt template such as "a photo of a {label}" — encode every sentence with the text encoder, and assign the image whichever label sentence has the highest cosine similarity to its image embedding. The prompt wording matters: phrasing the label as a natural caption matches the style CLIP saw during training, and averaging several templates (prompt ensembling) lifts accuracy a little more. Like a quiz contestant who never studied your specific exam but has read so widely that, handed the answer choices written out in full, they can pick the best match anyway. Example: deciding whether a photo is a cat or a dog by checking whether "a photo of a cat" or "a photo of a dog" sits closer to the image in CLIP's shared space.

Ziegler-Nichols​

A classic step-by-step recipe for picking starting PID gains without a model of the system, published by John Ziegler and Nathaniel Nichols in 1942. The common version: turn off the integral and derivative terms, then slowly raise the proportional gain until the system oscillates with a steady, even rhythm; record that gain (the ultimate gain) and the oscillation's period, and plug them into a small table of formulas that spits out values for all three PID terms. It rarely gives the perfect tuning — the results tend to overshoot — but it lands you in a sensible ballpark to refine by hand, which is its real value. Analogy: a cook's rule of thumb like "a teaspoon of salt per cup of rice" — not gourmet-exact, but a reliable starting point you then taste and adjust.

ZMP​

Zero-Moment Point — classical biped balance criterion

Zoom​

A camera move that magnifies or shrinks the view without the camera physically moving — like using binoculars to pull a faraway sign closer while your feet stay planted. It narrows or widens the lens's field of view, so the whole frame scales in or out at once. Contrast it with a dolly, which changes the picture by actually rolling the camera nearer or farther. It is one of the moves a video model can be steered through with camera control.

β-VAE​

A VAE variant that multiplies the KL divergence part of the ELBO by an adjustable knob called β. Turning β up past 1 pressures the model to use its latent space more tidily, often making individual latent dimensions line up with meaningful features (like rotation or thickness) — but push it too far and the model stops reconstructing the input well. It is the simplest way to trade reconstruction quality against a cleaner, more interpretable latent.

σ-schedule (Karras)​

The EDM convention of describing each noise level by its standard deviation σ (a real number) rather than by a discrete timestep t (an index from 0 to ~1000). Because σ directly measures "how much noise is on the image right now," the math for training and sampling becomes cleaner and sampler step sizes are easier to choose. Like labeling oven settings by their actual temperature instead of an arbitrary dial number from 1 to 10.

License​

MIT License. See the LICENSE file for details. ://github.com/25621/ai-learning-guides/blob/main/LICENSE) file for details.