Ranger4 DevOps Blog

Seven Key Questions to Ask APM Vendors

Posted by Burhanuddin Sefudin on Tue, Aug 19, 2014 @ 15:08 pm

At Ranger4, we constantly notice the surprise and excitement of IT Teams and business application owners when they see AppDynamics in action and realise they can defy the conventional wisdom of application management.

Your application monitoring platform shouldn’t have to struggle to keep up with changing technologies; it should be ahead of the curve.

Over the last year, AppDynamics collected a short list of key questions that APM buyers should ask to ensure they get the solution that best fits their needs - this follows on from our previous blog Application Performance Management: What to Look for in a Vendor. For example, some products like profilers are best suited for development environments, but are a bad fit for production monitoring. The same distinction is true of APM products.

Many APM vendor solutions will work for you if you have a relatively simple environment. However, if your environment has any of the characteristics below, you’d be well served to dig deep into vendor capabilities:

Environment Characteristics

Here is a list of recommended questions to ask APM vendors:

1. Fit for Production Environment

  • What is the overhead in production if I want visibility into every class & method?
  • When setting up the system, am I forced to make compromises on level of visibility to be able to have an acceptable level of overhead? Why am I forced to make that compromise?
  • Do you recommend that users instrument every class & method? If not, why not? Can the system scale to support thousands of reporting agents to a single management server? 

2. Fit for Highly-Distributed or SOA environments

  • Has your user interface been designed for users to monitor the health of individual Application Servers or does it enable the monitoring of distributed Business Transactions?
  • How does the product automatically discover and visualise the relationship between services and the back-end tiers?
  • Can it measure and display the latency of each hop?
  • What technology approach is used to track & follow business transactions across the distributed environment and provide code-level diagnostics at each tier?
  • What percent of my business transactions will it discover out-of-the box?
  • Can the product identify communication problems between the tiers/services in a SOA environment?

3. Fit for Agile Environments

  • Does your product require me to manually instrument the code? If so, how long does this take per application?
  • When agile code is introduced, does the instrumentation have to be manually updated? If so, how much time does this typically take your clients?
  • Does your product allow me to easily analyze the how a new release may have degraded application performance and pinpoint the root cause down to the code level?  

4. Behaviour Learning

  • Does your product continuously learn the normal performance of the application or do I have to set manual thresholds?
  • Can it learn different load and response time patterns for different days or time of day?
  • Can it learn “normal” deviation patterns and take action automatically based on what it observes?
  • Does the system automatically adapt to dynamic changes in capacity in a virtualised or elastic computing environment? 

5. Reduce Mean-time-to-Resolution (MTTR) with Troubleshooting & Root Cause Diagnostics

  • When a problem occurs, is diagnostic data automatically captured or do I have to manually trigger a diagnostic session?
  • Are there code-level diagnostic automatically captured for every bad session?
  • How does the product distinguish one-time anomalies from consistent patterns of poor performance? What actions does the product take when it detects systemic failures?
  • Does the product provide specific diagnostics for each of the following:
    • Slow response
    • Errors
    • Stalls
    • Deadlocks
    • Malformed code
    • Slow SQL/DB access
    • Memory Leaks & Thrash
    • Distributed service communication issues
    • Synchronisation issues

6. Easy-to-Deploy & Use

  • Can I deploy the product myself in less than an hour? Or do you recommend that I have your consultant assist me in the deployment?
  • Does your product user interface require experts to interpret the data when problems happen and rely on them to determine root cause? Or, does the product pinpoint root cause automatically?
  • Does your product have an intuitive design that I can I access your product from a web browser? 

7. Total Cost of Ownership

  • What is the initial software license cost?
  • What is the total cost of hardware to support the APM solution deployment?
  • How many collectors? How many Managers of Managers?
  • What is the annual labor and consulting costs for APM Product Configuration / Instrumentation?
  • What are the costs for ongoing maintenance, application of software upgrades of the APM solution?

These questions are just here to help you get started. Feel free to get in touch with Ranger4 to further discuss your business and its application needs.


Topics: Application Performance Management, APM, Appdynamics