Know the language. Build with clarity.

414 defined terms · 16 topic areas · 50 essentials

Models, reasoning & training terms

36 matching terms.

• Inference-time compute Concept · Models, reasoning & training

  Computation spent solving a request rather than training the model.

• LoRA Concept · Models, reasoning & training

  Low-rank adaptation: learn low-rank parameter updates instead of updating all model weights.

• SFT Concept · Models, reasoning & training

  Supervised fine-tuning: train on examples of desired inputs and outputs.

• Active parameters Concept · Models, reasoning & training

  The subset of model parameters used for a given token or computation path.

• Attention Concept · Models, reasoning & training

  A learned mechanism for combining information from different input positions or representations.

• Base model Concept · Models, reasoning & training

  A pretrained model before a particular instruction or preference adaptation stage.

• Best-of-N Pattern · Models, reasoning & training

  Generate multiple candidates and select one using a scoring method.

• Chain of thought Concept · Models, reasoning & training

  Intermediate reasoning expressed in steps; visible explanations are not guaranteed faithful traces of internal computation.

• Constitutional AI Concept · Models, reasoning & training

  An alignment approach using explicit principles to guide critique and AI-derived feedback.

• Dense model Concept · Models, reasoning & training

  A model whose main parameters are generally used for each token, unlike sparse expert activation.

• Distillation Concept · Models, reasoning & training

  Train a student model using supervision obtained from a teacher model or system.

• DPO Concept · Models, reasoning & training

  Direct Preference Optimization: train on preference pairs without a conventional separate online RL optimization stage.

• Fine-tuning Concept · Models, reasoning & training

  Adapting pretrained model weights with task, domain, or preference data; supervised fine-tuning and parameter-efficient methods are distinct approaches.

• GRPO Concept · Models, reasoning & training

  Group Relative Policy Optimization: estimate relative advantages from groups of sampled responses for policy optimization.

• Instruction-tuned model Concept · Models, reasoning & training

  A model trained on examples of responding to instructions.

• Logits Concept · Models, reasoning & training

  Unnormalized model scores over candidate outputs, often transformed into probabilities.

• Mixture of experts Concept · Models, reasoning & training

  A model architecture routing computation through a subset of expert components.

• Open weights Concept · Models, reasoning & training

  Model weights are accessible under a license; training data and unrestricted usage rights are not implied.

• PEFT Concept · Models, reasoning & training

  Parameter-efficient fine-tuning: adapt a model by training a limited subset or added parameters.

• Post-training Concept · Models, reasoning & training

  Training after pretraining to improve instruction following, preferences, specialized behavior, or reasoning.

• PPO Concept · Models, reasoning & training

  Proximal Policy Optimization: a policy-gradient reinforcement-learning method using constrained updates.

• Pretraining Concept · Models, reasoning & training

  Broad training that establishes a model's initial representations and capabilities.

• QLoRA Concept · Models, reasoning & training

  Parameter-efficient adaptation using low-rank updates with a quantized base model.

• Reward model Concept · Models, reasoning & training

  A model that estimates preference or quality for training, evaluation, or candidate selection.

• RLAIF Concept · Models, reasoning & training

  Reinforcement learning using feedback generated by AI systems.

• RLHF Concept · Models, reasoning & training

  Reinforcement learning using reward signals derived from human feedback.

• RLVR Concept · Models, reasoning & training

  Reinforcement learning with verifiable rewards, such as outcomes checked by tests or objective graders.

• Self-consistency Pattern · Models, reasoning & training

  Aggregate results from multiple sampled reasoning attempts, often by answer agreement.

• SLM Concept · Models, reasoning & training

  Small language model: a relative size label with no single universal parameter threshold.

• Synthetic data Concept · Models, reasoning & training

  Generated examples used for training, evaluation, or augmentation.

• System 1 / System 2 Concept · Models, reasoning & training

  Informal metaphors for fast direct responses and deliberative processing, not standard model architectures.

• Temperature Concept · Models, reasoning & training

  A sampling parameter that reshapes relative token probabilities; its effect depends on the decoding setup.

• Tokenization Concept · Models, reasoning & training

  Convert input into units from a model-specific vocabulary.

• Top-p Concept · Models, reasoning & training

  Nucleus sampling: select from a probability-ranked token set meeting a cumulative probability threshold.

• Total parameters Concept · Models, reasoning & training

  The full parameter count of a model, including components not active on every token.

• Transformer Concept · Models, reasoning & training

  A neural architecture built around attention and learned transformations of token representations.

Review cards · Distinctions · Cheat sheets