Operationalizing LLMs: From Prototype to Production
Yogiraj Awati shares how Instacart moves LLM applications beyond prototypes through retrieval, offline and online evaluation, guardrails, observability, and provider fallbacks. Two case studies cover recipe ingestion and Carebot support workflows with API-backed actions and human handoff.
Putting an LLM into production requires more than choosing a model. Yogiraj Awati breaks the application into model, retrieval, orchestration, evaluation, safety, and monitoring layers, then shows how those concerns appear in two Instacart use cases. A recipe-ingestion pipeline turns unstructured ingredients into normalized, catalog-compatible data, while Carebot combines help-center retrieval with API-backed actions, clarification, policy controls, and human escalation. The session also covers offline evaluation before model or prompt changes reach live experiments, matching models to workloads, caching stable support content, and preparing tested provider fallbacks.
More from the studio
1:17:55Tech TalksBeyond the DAG: Building Agentic Workflows That Loop, Branch, and Scale
Agentic workflows loop, branch, retry, and trigger new work instead of following a fixed DAG. Santosh Kumar Radha explains how AgentField uses typed functions, structured outputs, stopping and escalation patterns, event triggers, and controls for scaling, identity, and authorization.
Santosh Kumar Radha·Sep 17, 2026
48:04Tech TalksBuilding Trustworthy Financial AI: Governance, Bias, and Mechanistic Insights
Fabrizio Dimino examines why financial LLM recommendations can change when option order changes. He connects positional-bias tests and mechanistic interpretability with finance-specific red teaming, risk-sensitive scoring, model validation, and AI governance.
Fabrizio Dimino·Sep 17, 2026
1:05:16Tech TalksSecuring Code You Didn't Write: The AI Era's New Threat Model
Coding agents expand the software supply chain to include repositories, configuration, comments, connectors, and tool output. Rome Thorstenson examines reported attacks and outlines practical controls for permissions, secrets, prompt injection, and defense in depth.
Rome Thorstenson·Sep 17, 2026