Day 91LLMOps · Caching, routing, cost
Prompt caching & semantic caching
- • Compare exact-match prompt caching vs semantic caching — when does each fit?
- • How would you measure semantic-cache safety — what's the false-hit failure mode?
Operate LLMs with discipline
LLMOps and Reliable LLM Products
Operationalize LLM applications with prompt and model versioning, eval gates, tracing, routing, safety, cost controls, and incident response.
Featured topics
6 topic cards built for interview prep
Each topic includes a summary, practical learning goals, representative interview prompts, and a suggested roadmap day.
Intermediate80 minDay 51
Prompt, Model, Tool, and Retrieval Versioning
Treat LLM behavior as a release artifact by versioning prompts, model snapshots, tool schemas, retrieval configs, and eval sets together.
Learning objectives
- • Define the release unit for an LLM product
- • Pin model versions and document upgrade criteria
Advanced95 minDay 53
LLM Evaluation Operations
Build operational eval suites with golden datasets, adversarial tests, trace grading, human review, and online feedback loops.
Learning objectives
- • Create eval datasets that represent real and adversarial usage
- • Grade retrieval, tool use, reasoning traces, final answers, latency, and cost
Advanced90 minDay 54
LLM Observability and Incidents
Instrument LLM calls, traces, retrieval, tool execution, refusal rates, hallucination reports, cost, and user outcomes.
Learning objectives
- • Design logs and traces that are useful without leaking sensitive content
- • Investigate hallucination, latency, tool, and cost incidents
Advanced95 minDay 55
LLM Cost, Latency, Caching, and Routing
Optimize LLM systems with model cascades, semantic caching, context budgeting, batching, fallbacks, and quality guardrails.
Learning objectives
- • Measure cost per request, feature, tenant, and user segment
- • Route requests across model tiers without silent quality loss
Advanced100 minDay 56
LLM Safety, Security, Privacy, and Red Teaming
Prepare for prompt injection, jailbreaks, data exfiltration, PII handling, policy enforcement, human review, and auditability.
Learning objectives
- • Threat model RAG, tool-using agents, and external integrations
- • Design red-team tests and human escalation paths
Advanced90 minDay 57
Fine-Tuning Operations and Adapter Management
Cover fine-tune vs RAG decisions, dataset curation, LoRA adapters, evaluation, rollout, and model governance.
Learning objectives
- • Choose between prompt changes, RAG, supervised fine-tuning, and adapters
- • Evaluate fine-tunes for generalization, safety, and memorization
Practice prompts
Daily-plan topics tied directly to this pillar
These are pulled from the same 133-day roadmap content used by Browse Questions.
Day 91LLMOps · Caching, routing, cost
Model routing: cheap-then-expensive cascades
- • How would you route requests across GPT-5, Claude 4.5, and a small open-source model?
- • Walk through how a verifier model gates the cheap-model output before falling back to the expensive one.
Day 91LLMOps · Caching, routing, cost
Production LLM harness: evals, traces, guardrails, feedback loops
- • A demo works perfectly but production fails. What does a robust LLM harness include beyond the prompt?
- • How would you make LLM behavior reproducible, debuggable, and regression-tested across prompt, tool, and model changes?
Day 91LLMOps · Caching, routing, cost
Inference servers: vLLM, TensorRT-LLM, SGLang
- • What does vLLM's PagedAttention do for throughput?
- • Compare vLLM vs TensorRT-LLM vs SGLang.
Day 91LLMOps · Caching, routing, cost
Local LLM serving: Ollama, quantized model files, laptop/server trade-offs
- • When would you run an LLM locally instead of calling a hosted API?
- • What changes when serving a quantized local model through Ollama compared with a GPU-backed vLLM service?
Day 91LLMOps · Caching, routing, cost
Latency debugging, rate limits, and concurrency controls
- • How would you diagnose high first-token latency vs high tokens-per-second latency?
- • How do rate limits, concurrency limits, queues, and retries interact in an LLM API gateway?
Day 98LLMOps · Caching, routing, cost
Prompt caching & semantic caching
- • Compare exact-match prompt caching vs semantic caching — when does each fit?
- • How would you measure semantic-cache safety — what's the false-hit failure mode?
Day 98LLMOps · Caching, routing, cost
Model routing: cheap-then-expensive cascades
- • How would you route requests across GPT-5, Claude 4.5, and a small open-source model?
- • Walk through how a verifier model gates the cheap-model output before falling back to the expensive one.