New Relic: Are You Making These Costly Mistakes?

There’s a lot of misinformation floating around about how to get the most out of New Relic. Many users fall into common traps that prevent them from fully harnessing this powerful technology. Are you sure you’re not making these mistakes?

Myth 1: Default Dashboards Tell You Everything You Need to Know

The misconception here is that New Relic‘s default dashboards provide a complete picture of your application’s health. They don’t. While these dashboards offer a good starting point, they often present generic metrics that may not be relevant to your specific application or business goals. I had a client last year who relied solely on the default dashboards and missed a critical performance bottleneck in their payment processing system. By the time they realized the issue, they had already lost a significant number of transactions. Don’t be them.

The reality is that you need to customize your dashboards to monitor the metrics that truly matter to you. This requires a deep understanding of your application’s architecture and performance characteristics. For example, if you’re running an e-commerce site, you might want to track metrics like average cart size, conversion rate, and checkout time. Or, if you are running a containerized workload on AWS, you should be monitoring the CPU and memory usage of your pods. By creating custom dashboards that focus on these key metrics, you can gain much deeper insights into your application’s performance.

Consider a scenario where you’re running a web application that experiences slow response times during peak hours. The default New Relic dashboard might show you that overall response time is high, but it won’t tell you why. By creating a custom dashboard that tracks the performance of individual database queries, you might discover that a specific query is the culprit. You can then optimize that query to improve performance.

Myth 2: Alerting is a “Set It and Forget It” Feature

Many believe that once you’ve configured New Relic’s alerting system, you can simply sit back and wait for notifications. This is a dangerous assumption. Alerting, when not properly configured, can quickly lead to alert fatigue, where you’re bombarded with so many notifications that you start ignoring them altogether. I’ve seen teams completely miss critical production issues because they were desensitized to the constant stream of alerts. We had to completely revamp their alerting strategy.

Effective alerting requires careful planning and ongoing maintenance. You need to define clear thresholds for each metric you’re monitoring and ensure that those thresholds are appropriate for your application’s normal behavior. For example, setting a CPU utilization threshold too low can trigger false positives, while setting it too high can cause you to miss genuine problems. The key is to tune your alerts over time, based on your application’s performance and your team’s response capabilities. Consider integrating New Relic alerts with tools like Slack or PagerDuty to ensure that alerts are routed to the right people at the right time. A well-defined on-call schedule is critical, too.

Here’s a concrete example: a SaaS platform in the Perimeter Center area was experiencing intermittent database connection issues. They had alerts set up for database connection errors, but the thresholds were too high, and the team only received alerts when the entire database was down. By lowering the threshold and adding alerts for slow query execution times, they were able to identify and resolve the intermittent connection issues before they caused a major outage. It took them a week of tuning and experimentation to arrive at the right settings.

Myth 3: Data Retention Doesn’t Matter

A common misconception is that New Relic’s default data retention policies are sufficient for all use cases. This couldn’t be further from the truth. While New Relic offers various data retention options, the default settings may not provide enough historical data for in-depth analysis or long-term trend identification. Imagine trying to diagnose a performance regression that occurred three months ago, only to discover that the relevant data has already been purged. Not fun.

You need to actively manage your data retention policies based on your organization’s specific needs and compliance requirements. Consider increasing the retention period for critical metrics and events, such as transaction traces and error logs. This will allow you to perform more thorough root cause analysis and identify long-term performance trends. Also, be aware of the cost implications of increased data retention. New Relic charges based on data volume, so you’ll need to balance your data retention needs with your budget. If you need to retain data for longer periods, consider exporting it to a data warehouse or other storage solution.

We ran into this exact issue at my previous firm, a software company near the intersection of GA-400 and I-285. We were trying to analyze a security breach that had occurred several months prior, but the relevant audit logs had already been purged from New Relic. We ended up having to reconstruct the events from other sources, which was a time-consuming and error-prone process.

Myth 4: Ignoring the New Relic Query Language (NRQL)

Many users limit themselves to the pre-built dashboards and reports, failing to tap into the power of the New Relic Query Language (NRQL). They think it’s too complex or unnecessary. This is a huge missed opportunity. NRQL allows you to query your New Relic data in a highly flexible and granular way, enabling you to uncover insights that would be impossible to obtain using the standard interface.

Learning NRQL opens up a world of possibilities. You can use it to create custom visualizations, build complex alerts, and perform advanced data analysis. For example, you could use NRQL to calculate the average response time for a specific transaction type, or to identify the most common error messages in your application. It is also possible to correlate data from different sources to get a more holistic view of your system’s performance. While there are many monitoring tools out there, the ability to query the data in a flexible way sets New Relic apart. NRQL is better than some of the query languages I’ve seen in competing products.

Here’s what nobody tells you: NRQL is not as daunting as it seems. Start with the basics and gradually work your way up to more complex queries. New Relic provides excellent documentation and tutorials to help you get started. Experiment with different queries and visualizations to see what you can uncover. You might be surprised at the insights you gain.

Myth 5: New Relic is Only for Production Environments

Some believe that New Relic is solely a tool for monitoring production environments. This is incorrect. While New Relic is certainly valuable in production, it can also be used effectively in development and testing environments. Using New Relic throughout the entire software development lifecycle can help you identify performance issues early on, before they make their way into production. We’ve found that it saves our clients a lot of money to use New Relic in pre-production to catch issues early.

By monitoring your application in development and testing, you can identify performance bottlenecks, memory leaks, and other issues that might not be apparent during functional testing. This allows you to address these issues proactively, rather than reactively after they’ve already impacted your users. Consider integrating New Relic into your CI/CD pipeline to automatically monitor the performance of each build. This can help you ensure that new code changes don’t introduce any performance regressions. I had a client who used New Relic to monitor their load testing environment and discovered a critical scalability issue that would have caused a major outage in production. They were able to fix the issue before it ever impacted their users.

Furthermore, you should consider code optimization as a way to improve performance, which can then be verified using New Relic. It’s important to debunk app performance myths so you can get the best results with New Relic.

Don’t let these common myths hold you back from fully harnessing the power of New Relic. Take the time to customize your dashboards, tune your alerts, and master NRQL. Your application’s performance will thank you for it. Spend the next week reviewing your New Relic configuration — it’s an investment that pays off.