09-reasoning-slms-key-formulas

Question 1

What is the Hallucination Paradox?

Answer 1

Reasoning models (e.g., OpenAI o1, o3) hallucinate at HIGHER rates on simple factual benchmarks despite improved logical reasoning. Mechanism: more internal claims in the reasoning chain create more opportunities for error propagation. o3 had a 0.51 hallucination rate vs. 0.44 for o1 on SimpleQA.

Question 2

What are Reasoning Models (e.g., o1, o3)?

Answer 2

LLMs that generate explicit chains of thought (“System 2 thinking”) before producing a final answer. They improve performance on logic and coding but exhibit the Hallucination Paradox on factual benchmarks.

Question 3

When should you use a standard LLM with RAG instead of a Reasoning Model?

Answer 3

For applications requiring strict factual accuracy (e.g., benefits eligibility, regulatory citations). Reasoning models should be reserved for complex analysis where the logic can be verified (e.g., code vulnerability analysis, legal reasoning over provided statutes).

Question 4

What is a Small Language Model (SLM)?

Answer 4

An LLM with fewer than ~7 billion parameters (e.g., Phi-3). Advantages: runs on edge devices without cloud connectivity, dramatically cheaper inference, data never leaves the device (privacy), and lower latency for simple tasks.

Question 5

What are the key advantages of SLMs for federal use?

Answer 5

(1) Edge capability — run on laptops or tactical devices without cloud; (2) Cost — orders of magnitude cheaper; (3) Privacy — data never leaves the device; (4) Speed — lower latency for simple tasks; (5) Bypasses data center energy bottleneck.

Question 6

What is the formula for the Compound Mistake Problem?

Answer 6

Accuracy_total = (Accuracy_step)^n, where n is the number of steps. A 95% per-step accuracy over 10 steps yields only ~60% total accuracy (0.95^10 ≈ 0.598).

Question 7

What is RAG in one sentence?

Answer 7

Retrieve relevant documents from an external knowledge base, inject them into the model’s context window, and generate answers grounded in those documents to reduce hallucination.

Question 8

What is the Data Flywheel in one sentence?

Answer 8

A virtuous cycle where user interactions generate data that improves the AI system, which produces better interactions that generate more valuable data.

Question 9

What is LoRA in one sentence?

Answer 9

A parameter-efficient fine-tuning method that adds small trainable low-rank matrices (~100MB) to frozen model layers, avoiding full retraining of billions of parameters.

Question 10

What is AI-as-a-Judge in one sentence?

Answer 10

Using a strong LLM to automatically evaluate the outputs of another LLM for quality, faithfulness, or safety at scale.

Question 11

What is the Model Router in one sentence?

Answer 11

An abstraction layer that routes AI requests to different models based on complexity, cost, or capability — enabling vendor-agnostic, cost-optimized architectures.

Question 12

What is the Instruction Hierarchy in one sentence?

Answer 12

A prompt design pattern where system-level instructions take precedence over user-level inputs, enforcing behavioral compliance regardless of user attempts to override.

Question 13

What is Span-Level Verification in one sentence?

Answer 13

Requiring the model to cite the exact sentence in a source document that supports its answer, enforcing auditability and truthfulness.

Question 14

What is Hybrid Search in one sentence?

Answer 14

Combining dense vector search (semantic meaning) with sparse keyword search (BM25 for exact terms) to maximize retrieval recall in RAG pipelines.

Question 15

What is Reranking in one sentence?

Answer 15

A second-stage retrieval step where a cross-encoder model re-scores initial search results to maximize precision before context is passed to the LLM.

Question 16

What does Faithfulness measure in one sentence?

Answer 16

Whether a generated answer is derived ONLY from the retrieved context, with no hallucinated information introduced by the model.

Question 17

What is the Planner-Evaluator-Executor pattern in one sentence?

Answer 17

A decoupled agent architecture where a Planner proposes actions, an Evaluator checks them for safety and compliance, and an Executor carries out only approved actions.

Question 18

What is Evaluation-Driven Development in one sentence?

Answer 18

The discipline of defining success metrics and building the evaluation pipeline BEFORE building the AI system, so every design decision is tested against measurable criteria.

Question 19

What are the three memory types in Huyen’s hierarchy?

Answer 19

Parametric (model weights), Episodic (context window), and Semantic (vector database / external storage).

Question 20

What is PEFT in one sentence?

Answer 20

Parameter-Efficient Fine-Tuning methods (like LoRA) that adapt only a small subset of model parameters, dramatically reducing compute cost while preserving most fine-tuning benefits.