In the modern enterprise landscape, data is the lifeblood of decision-making. Yet, for many engineering and product leaders, the promise of data-driven insight is frequently stifled by a persistent, costly barrier: the "ingestion tax." Organizations often find their telemetry data trapped within observability platforms while critical business context remains siloed in external databases like Snowflake, PostgreSQL, or even static Google Sheets.
Today, New Relic is fundamentally changing this dynamic with the General Availability of New Relic Lens. This new data federation capability marks a significant shift in how observability platforms interact with the wider enterprise data ecosystem. By allowing users to query, correlate, and analyze data exactly where it resides—without the need to move or ingest it—New Relic is positioning itself as the central nervous system for unified business and technical intelligence.
The Core Challenge: The "Double Tax" of Observability
To understand the significance of New Relic Lens, one must first recognize the "double tax" problem that has plagued IT operations for years. Historically, if an engineering lead wanted to correlate system performance—such as checkout latency on an e-commerce site—with business metrics like total shopping cart revenue, they faced a logistical and financial nightmare.
The business data usually resides in a relational database or a cloud data warehouse like Snowflake. To gain insight, teams have traditionally had to "ingest" that business data into their observability platform. This process incurs a double cost: the original cost of storing the data in the primary database, and a subsequent, often significant, cost for ingesting that same data into an observability tool.
Beyond the financial burden, this process introduces latency and complexity. Data pipelines must be built, maintained, and monitored. When data schemas change, the pipelines break, leading to "chaotic workarounds" where analysts manually export data into separate Business Intelligence (BI) tools to perform cross-platform correlations. This manual labor is not only inefficient; it often leads to stale data and siloed decision-making, where the "technical" story told by the observability platform and the "business" story told by the BI tool remain fundamentally disconnected.
Chronology of a Data Evolution
The journey to New Relic Lens is a reflection of the broader evolution of the observability market. Over the past decade, the industry moved from simple server monitoring to full-stack observability. However, as organizations scaled, the volume of telemetry data exploded, making traditional ingestion-heavy models unsustainable.
- Phase 1: The Monitoring Era: Focus was on "is the server up?"
- Phase 2: The Observability Boom: Focus shifted to "why is this request failing?" utilizing massive ingestion of logs, metrics, and traces.
- Phase 3: The Federation Era (Present): Focus has turned to "how does this system performance impact my bottom line?" without the overhead of moving petabytes of data.
New Relic Lens represents the culmination of this third phase. By introducing data federation, the company is moving away from the "all-in-one-place" mandate and toward an "access-anywhere" philosophy. This strategy allows users to keep their data in specialized systems optimized for that data type—Snowflake for analytics, PostgreSQL for relational business logic, and New Relic for high-cardinality telemetry—while providing a unified interface to query them all at once.
How New Relic Lens Works: Technical Architecture
New Relic Lens acts as a bridge between the observability UI and external data sources. It allows users to write standard SQL queries that reach out to external connectors in real-time.
The Power of Cross-Database JOINs
The standout feature of Lens is its ability to perform cross-database JOINs. For example, a user can query user metadata stored in Snowflake and join it with page view telemetry generated within New Relic.
Example Workflow:
Imagine a query designed to group page performance by customer segment:
SELECT sf.company, COUNT(*)
FROM telemetry.PageView nr
JOIN snowflake.users.user_metadata sf ON nr.enduser.id = sf.userId
GROUP BY sf.companyIn this scenario, the telemetry.PageView data is retrieved from New Relic’s internal storage, while the snowflake.users.user_metadata is fetched on-demand from the client’s Snowflake instance. The final correlation happens within the Lens engine, delivering an integrated report to the user in seconds.
This architecture removes the need for ETL (Extract, Transform, Load) processes. Because the data remains at the source, security teams can maintain governance over the external databases, and engineering teams avoid the latency associated with synchronizing datasets.
Supporting Data and Use Cases
The utility of Lens spans multiple departments within an organization:
- Product Management: By linking system latency with user churn data (stored in a CRM or SQL database), product managers can visualize the exact point where performance degradation begins to impact user retention.
- Engineering Leadership: Engineering leads can correlate deployment frequency with business revenue. If a specific deployment coincides with a dip in revenue, they can identify the bottleneck in minutes rather than days.
- IT Operations: Operations teams can ingest metadata from configuration management databases (CMDBs) to enrich alerts. An alert that says "Database X is slow" becomes "Database X, which hosts the payment processing service for our EMEA region, is slow."
Official Perspective: Driving Value Through Innovation
Alexander Jones, Principal Product Manager at New Relic and a key architect behind the language features powering Lens, emphasizes that this release is about more than just convenience—it’s about empowering users to do more with the data they already possess.
"We wanted to build a query experience that doesn’t just look at telemetry in isolation," Jones notes. "By enabling users to bring their own context—whether that’s from Snowflake, Google Sheets, or traditional databases—we are helping teams bridge the gap between technical performance and business outcomes. This is about making observability a strategic business asset."
Jones’s work on the underlying query language has been focused on performance and reliability, ensuring that federated queries do not degrade the experience for the user. By optimizing how New Relic handles these remote connections, the system ensures that complex joins are executed with high efficiency, maintaining the responsiveness that DevOps teams require during incident response.
Implications for the Industry
The introduction of New Relic Lens has profound implications for the observability market.
1. The Death of the Ingestion Tax
As organizations look to cut costs, the "double tax" of ingestion is becoming a primary target for CFOs. By providing an alternative, New Relic is setting a new standard for cost-effective observability. This feature may force competitors to rethink their pricing models, as the value proposition of "all-in-one-platform" shifts toward "best-in-class federation."
2. Democratization of Data
By allowing queries to be performed directly from the New Relic UI, the platform becomes accessible to non-engineers. Product managers and business analysts, who might be intimidated by complex telemetry logs, can now use standard SQL to answer their own questions using business data they are already familiar with.
3. Strengthening Data Governance
Because data never leaves the original database until it is queried in the UI, organizations can maintain stricter control over their data footprint. Security teams can configure Fine-Grained Access (FGA) to ensure that only authorized personnel can query specific external sources, providing a secure path to cross-functional data analysis.
Getting Started: Implementation and Availability
New Relic Lens is now available to all paid customers of the platform. The implementation is designed to be low-friction, requiring minimal configuration to link external databases.
Administrators manage access via the ‘organization_manager’ role by default, ensuring that sensitive data sources remain protected. From there, teams can configure connectors to their preferred data warehouses or databases. New Relic has provided comprehensive documentation to assist in setting up these connectors, emphasizing a "configure and query" approach that avoids long implementation cycles.
For teams looking to bridge the gap between their technical infrastructure and their business objectives, the message is clear: Stop moving data and start querying it. By breaking down the silos that have traditionally separated the engineering room from the boardroom, New Relic Lens is paving the way for a more unified, efficient, and data-driven future.
To learn more about implementing New Relic Lens within your environment, visit the official New Relic Documentation. For specific inquiries or community support, visit the New Relic Explorers Hub.
