Capability Elicitation
Techniques for discovering and activating latent capabilities in AI models — abilities that exist but are not obvious from standard testing or usage.
Why It Matters
Capability elicitation reveals what models can actually do, which may be more (or different) than what benchmarks show. It is critical for both safety and utility.
Example
Discovering that a model can solve complex mathematical proofs when given chain-of-thought prompting, despite failing the same problems with direct questioning.
Think of it like...
Like a talent show audition revealing that someone who applied as a singer is also an amazing dancer — the capability was there but needed the right conditions to appear.
Related Terms
Evaluation
The systematic process of measuring an AI model's performance, safety, and reliability using various metrics, benchmarks, and testing methodologies.
Red Teaming
The practice of systematically testing AI systems by attempting to find failures, vulnerabilities, and harmful behaviors before deployment. Red teamers actively try to break the system.
Prompt Engineering
The practice of designing and optimizing input prompts to get the best possible output from AI models. It involves crafting instructions, providing examples, and structuring queries to guide the model toward desired responses.
Emergent Behavior
Capabilities that appear in large AI models that were not explicitly trained for and were not present in smaller versions. Emergent abilities seem to appear suddenly at certain scale thresholds.
Benchmark
A standardized test or dataset used to evaluate and compare the performance of AI models. Benchmarks provide consistent metrics that allow fair comparisons between different approaches.