Ranger4 DevOps Blog

Application Performance Management: What to Look for in a Vendor

Posted by George Price on Fri, Jan 31, 2014 @ 13:01 pm

At a high level, APM tools must be selected with the five following strategic end goals in mind:   

                       

1. Simple to install and use

2. Able to go wide

3. Able to go deep

4. Able to understand business transactions

5. Built for cloud environments


Your APM tool must be able to cope with the change your business constantly experiences.


Gartner defined five functional dimensions of APM, let’s explore each a little further:


1) End-user experience monitoring

This is the capture and presentation of data about how end-to-end application availability, browser, network and server latency, execution correctness and quality appear to the end user. IT operations should be able to compare end user experience across geographies and browser types and advise development accordingly.


We believe business transactions are the only true measure of application performance and you should look for a tool that provides real-time business transaction monitoring with the capability to measure and track all transactions:


- Response Time

- Load

- Error Rate

- Slow Rate

- Stalls


2) Application topology discovery and visualization


You should look for a tool that handles the discovery of the software and hardware infrastructure components executed by the application, and the possible ways in which these components communicate to deliver the application and makes it possible to visualize and manage entire applications. Look for tools that make it possible to auto-discover and map application tiers and services, and see the relationships between them, thus eliminating blind spots.


You tool should enable you to follow the flow of your transactions across all tiers and services (even in a highly distributed environment) so you can identify bottlenecks super fast.


3) User-defined transaction profiling


The tracing of user-grouped events, which comprise a transaction as they occur within the application as they interact with components discovered in the second dimension (application topology discovery and visualization); this is generated in response to a user's request to the application.


4) Application component deep dive

                                           

Your Application Performance Management tool of choice should provide fine-grained monitoring of the resources consumed by, and events occurring within, the components discovered in the application topology discovery and visualization dimension. These may include both server and client side components, devices and interfaces.


You should want to be able to identify root cause with complete code diagnostics and have full visibility into code execution in production. Your APM tool should add very little overhead and require little or no manual configuration.


5) IT operations analytics


The burden of overseeing and managing applications has increasingly moved away from application architects and developers. Now, it is IT operations and infrastructure professionals who are often shouldering the responsibility for ensuring application health and performance.

       

This is particularly true with respect to distributed applications. A fundamental change in the way that applications are built, deployed, and scaled has caused web applications to shift from being monolithic in nature to many components distributed over multiple tiers and nodes, all connected through service-oriented architectures. This trend has contributed to IT operations and infrastructure professionals being the ones whose are alerted when applications fail to meet business SLAs.


You chosen tool should use a number of techniques including event processing, statistical pattern discovery and recognition, unstructured text indexing, search and inference, topological analysis, and multidimensional database search and analysis to discover meaningful and actionable patterns in the typically large datasets generated by the first four dimensions of APM. It should enable you to detect business impact and performance spikes and your tool should learn "normal" performance behavior and "normal" code execution paths for all Business Transactions and Application Services, at all hours, days and weeks of the year - so anomalies can be detected automatically.


Your tool of choice should ship with pre-built reports for your line of business people, dashboards for the operations teams and have a release comparison feature so that the development teams can see and understand the impact of their releases.


Questions to ask

Here are some questions to ask when talking to APM vendors:

                       

  1. What is the installation process up front?

  2. What kind of post-install work is required?

  3. How does the tool map the application?

  4. How does it display that mapping?

  5. Does it keep up with the needs of agile development?

  6. How much manual instrumentation is needed for complete visibility?

  7. Does your tool have insight into the entire distributed environment?

  8. Are there any blind spots in your current architecture?

  9. If it does map the environment, how does that data come back to you? Is it simple to view?                   

  10. Can your APM tool go deep as well as wide?

  11. How does it isolate problem areas at the code level? • How much overhead does it add? Less than 2%?                   

  12. How do you define “business transactions”?

  13. How does your tool represent them?

  14. How does your tool speak the “business language” of the application?

  15. Does the tool learn and react from performance baselines dynamically? If not, how are they set and maintained?

  16. Is your tool built for a cloud environment “ground up,” or has it been retrofitted?

  17. Can the tool manage application performance with respect to virtual and clouds environment? If so, how? • How many nodes can the tool monitor in the cloud?

  18. How does it adapt to the dynamic nature of the cloud?

  19. Can it dynamically provision resources across the cloud?

  20. What cloud-ready environments has it monitored in the past? What are the customer names?


    Topics: Application Performance Management, AppHappy, DevOps, Appdynamics