Agent Observability & Tracing Debug Production LLM Systems

Instrument a real agent with OpenTelemetry, catch silent failures, judge live traffic, debug a 3-bug incident.

What you'll learn

⚡ Instrument a real tool-calling agent with OpenTelemetry: spans, attributes, events, nesting and context propagation
⚡ Stand up Arize Phoenix and Langfuse yourself with Docker, and compare both on one identical trace
⚡ Catch silent failures that never raise: empty results, hidden retries, graceful fallbacks and confidently wrong answers
⚡ Attribute latency and cost per span, per feature, per user and per tenant, and see why run-level averages hide regressions
⚡ Build an LLM-as-judge on sampled live traffic, wire its verdicts back into the trace, and catch the judge being wrong
⚡ Detect prompt injection and quality drift from trace patterns rather than from the prompt text
⚡ Debug an unfamiliar trace under time pressure, correlate it to a user complaint, and diff a good run against a bad one
⚡ Work a multi-cause incident end to end: three seeded bugs, one red herring, a verified fix and an honest postmortem
⚡ Price observability at real scale and decide what to sample using arithmetic instead of instinct
⚡ Roll this out on a team: ownership, code review for instrumentation, and a runbook built from a real incident

Requirements

❗ Comfortable Python: functions, classes, dictionaries, reading a traceback
❗ Any OpenAI-compatible API key. Total spend across the whole course is a few dollars
❗ Docker and Docker Compose installed, for the two tracing backends
❗ No prior OpenTelemetry or observability experience needed. It is built from the first span

Description

This course contains the use of artificial intelligence.
Your agent returned a confident, well formed, completely wrong answer. Every span in the trace says status OK. Your eval suite passed this morning. Nothing is red, and something is badly broken.
An eval score is a photograph. Production is a video. This course is seventy one lectures about closing that gap, and every number in it was measured on a real system rather than typed because it sounded right.
What you actually build. A real tool-calling agent (search, fetch, calculate, summarise) instrumented with OpenTelemetry from scratch. Two real self-hosted backends, Arize Phoenix and Langfuse, compared on one identical trace. An LLM-as-judge running on sampled live traffic. A review queue. A runbook. Around twenty five small diagnostic tools, each written to answer one question the traces raised.
The failures are seeded and real. A search tool that always succeeds and always returns nothing, so the model rephrases six times and burns its budget. A fetch that serves the right document with stale numbers, where the agent does flawless arithmetic and lands eighty nine percent low. A graceful fallback that hides the truth. A prompt injection arriving through a poisoned document rather than a user prompt. None of them raise an error.
Then a capstone incident. Three bugs of three different kinds, no hints. One is a red herring: the loudest thing in the latency data, and not your problem. You learn to abandon it properly, by measuring what it leaves unexplained. The third bug has no error, no slow span, no anomaly at all, and nothing in any dashboard this course builds will flag it. It is visible only by adding up numbers the system already gives you and noticing they disagree.
Including the parts that did not work. When I scored every detector built in this course against that capstone, seven of thirteen contributed nothing. The LLM judge did not merely miss the bug, it returned grounded equals true and was right to. That result is in the course rather than hidden, because your coverage is always narrower than it feels.
Measured, not asserted. Storing every span for a million agent runs a day costs four dollars and thirty two cents a month, which makes sampling traces to save money the wrong instinct. A judge flipped its verdict ten times out of ten purely because the evidence arrived in a different order. Every figure quoted on screen is read off a measurement ledger that ships with the course, so you can check me.
Ten sections, seventy one lectures, roughly seven and a half hours, built on one small system you can run end to end on a laptop.

Who this course is for

⭐ Engineers already shipping something with an LLM in it, who have been told it is broken and had no idea how to find out why
⭐ Anyone whose agent passes its evals and still misbehaves in production
⭐ Backend and platform engineers who know normal APM but have not traced a non-deterministic tool-calling loop
⭐ ML and AI engineers who write evals and want the live half of the picture
⭐ Not for beginners looking for a first Python course, and not a prompt engineering course

Homepage

https://www.udemy.com/course/agent-observability-tracing-debug-production-llm-systems

Rapidgator
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part1.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part2.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part3.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part4.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part5.rar.html
AlfaFile
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part1.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part2.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part3.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part4.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part5.rar
DDownload
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part1.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part2.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part3.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part4.rar
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part5.rar
KatFile
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part1.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part2.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part3.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part4.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part5.rar.html
FreeDL
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part1.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part2.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part3.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part4.rar.html
yshfk.Agent.Observability..Tracing.Debug.Production.LLM.Systems.part5.rar.html