As the software development landscape pivots from traditional, deterministic request-response architectures to complex, autonomous multi-agent systems, the methods we use to ensure reliability are undergoing a radical transformation. In this new era, AI agents are replacing human-led search tasks, performing intricate reasoning, and invoking external tools with minimal oversight. However, this autonomy comes at a cost: the rise of "hidden" failures—hallucinations, performance bottlenecks, and opaque reasoning chains that standard logging cannot capture.
To ensure these systems remain stable, observability has become the cornerstone of AI engineering. When an agent drifts into a hallucination or stalls during a complex tool invocation, relying on simple logs is like trying to troubleshoot a jet engine with a stopwatch. To maintain reliability, developers must shift toward granular, trace-based visibility.
The Shift: From Logs to Traces
In the world of Generative AI, a log is a static record that something happened. A trace, by contrast, is a dynamic narrative: it tells you what happened, in what order, how long each step took, and—crucially—what caused what.
In traditional software, we often rely on debugging tools or direct access to infrastructure to inspect errors. But modern AI deployments, particularly those using managed serverless runtimes like Amazon Bedrock’s AgentCore, often abstract away the underlying infrastructure. You cannot SSH into a managed runtime to inspect a process; you are limited to the telemetry you export. Because AI is fundamentally non-deterministic, reproducing a specific failure scenario without a complete trace is nearly impossible.
Case Study: The Multi-Agent Travel Orchestrator
To understand how to implement effective observability, consider a multi-agent travel planning application. This system is designed to take a simple user request—such as "Plan a 5-day trip from NYC to Tokyo in March"—and decompose it into a series of logical steps.
Under the hood, this application uses a multi-agent framework where a central orchestrator delegates tasks to specialist agents: a research agent for destination information, a weather agent, and dedicated tools for booking flights and hotels. Each of these agents is paired with a Large Language Model (LLM) optimized for its specific function. When this system is deployed on a managed environment like AgentCore, the lack of traditional debugging access makes comprehensive OpenTelemetry (OTEL) instrumentation not just a best practice, but a necessity for survival.
Chronology of Instrumentation: Configuring the Pipeline
Implementing observability in a Strands-based multi-agent app requires a multi-layered approach to ensure that every span of the execution chain is captured and correlated.
1. The Foundation: Configuring the Exporter
The initial phase involves setting up the OpenTelemetry exporter. Using the Strands SDK, developers can configure the StrandsTelemetry helper to handle traces and metrics. Because Strands adheres to GenAI semantic conventions, it automatically reads standard OTEL_* environment variables, allowing for a standardized setup.
The configuration requires defining the OTLP endpoint (e.g., New Relic’s OTLP ingest), setting the service name, and configuring metric temporality. Notably, developers must also bridge Python’s standard library logging to OTLP, ensuring that logs are correlated with the trace and span IDs generated by the AI orchestrator.
2. Capturing AI-Specific Semantics
To gain true visibility into AI behavior, one must capture the "GenAI" instrumentation. By setting environment variables like OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT and OTEL_INSTRUMENTATION_GENAI_CAPTURE_TOOL_INPUT, the system begins to record the actual content of the AI’s reasoning process. This allows developers to see exactly what the model "thought" and what parameters were passed to tools, providing a window into the agent’s decision-making process.
3. Implementing Guardrails for Data Privacy
While capturing input and output is vital for debugging, it poses security risks—especially when dealing with PII (Personally Identifiable Information). A robust observability strategy must include "gated" tracing. By creating a custom tracer class that intercepts events, developers can define filters that redact sensitive data before it is exported to the telemetry backend. This ensures that while engineers have the context required to debug, they are not inadvertently storing user credentials or private travel data in their logs.
4. Promoting Events to Attributes
A common pitfall in modern observability platforms like New Relic is the distinction between "Span Events" and "Span Attributes." By default, many AI frameworks write prompt, completion, and tool input data into Span Events. However, querying these via standard NRQL (New Relic Query Language) can be difficult, as queries often look for top-level attributes. To solve this, developers must implement logic to "promote" critical event data to Span Attributes, making the telemetry significantly easier to filter, search, and alert upon.
Supporting Data: Why Traces Unlock Reliability
When a travel agent fails, the root cause could be a latency spike in the weather API, a token limit error in the flight booking model, or a logical hallucination in the research agent.
By analyzing the traces, developers can visualize the "critical path" of the request. They can identify:
- Bottlenecks: Which specific agent or tool is responsible for the majority of the latency?
- Failure Points: At which point in the chain did the agent go rogue?
- Model Efficiency: Are certain agents using models that are too expensive or too slow for their specific task?
Without these traces, these questions remain unanswered. With them, the "black box" of AI becomes transparent, allowing for iterative performance tuning and rapid incident response.
Implications for the Industry
The shift toward agentic architectures implies a fundamental change in the role of the DevOps engineer. As agents take on more agency, the focus moves away from "uptime" and toward "correctness" and "reasoning efficiency."
- Deterministic Troubleshooting of Non-Deterministic Systems: Even though AI models are probabilistic, the infrastructure they run on must be governed by deterministic observability. Tracing provides the guardrails that allow developers to trust these systems.
- Standardization of GenAI Observability: The industry is moving toward a unified standard (OpenTelemetry) for AI. This prevents vendor lock-in and allows organizations to move their agentic workloads across different cloud providers and LLM vendors without losing their observability maturity.
- The Rise of "Observability-Driven Development": Just as Test-Driven Development (TDD) changed how we write code, observability-driven development will dictate how we build agents. If an agent’s behavior cannot be traced and analyzed, it should not be considered "production-ready."
Conclusion
Building multi-agent orchestrations is becoming easier thanks to advanced frameworks like Strands and managed runtimes like Amazon Bedrock’s AgentCore. However, the ease of building does not equate to the ease of maintaining. The complexity of these systems is hidden, and without proper instrumentation, that complexity can lead to costly, silent failures.
By implementing OpenTelemetry, carefully managing the capture of sensitive data, and optimizing how telemetry is queried, engineers can turn their AI systems into transparent, manageable, and reliable tools. The repository for this travel-planning case study is available for developers to clone, deploy, and experiment with. In the race to build the next generation of intelligent agents, the winners will be those who can see inside the box.
Disclaimer: The views expressed in this article are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered are environment-specific and not part of the commercial solutions or support offered by New Relic. For official support, please consult the New Relic Explorers Hub.
