Ranger4 DevOps Blog

Forrester Study of Service Virtualization at a Major European Bank

Posted by Paul Hancock on Thu, Aug 7, 2014 @ 10:08 am

Forrester published an extremely powerful study into the Economic Impact of IBM Rational Service Virtualization and Test Automation Solutions at a large European bank. The customer received an ROI within 2 months and the cumulative financial benefit over a 3 year period of $6.2 million. This has been an extremely successful initiative for the bank given their investment of $433,149 over the 3 year period. It's also worth noting these savings continue for the bank at more than $2.5 million per annum.                   

The study highlights the benefits and costs of implementing IBM Rational Service Virtualization and Test Automation solutions. The customer, a large European bank with more than 30 million customers and 3,500 branches worldwide, implemented two of the IBM testing solutions — Rational Test Workbench (RTW) and Rational Test Virtualization Server (RTVS) — for one division’s software development group. The solutions were first implemented in 2009 by Green Hat Software, prior to its acquisition by IBM in 2012. 

Previous State

Prior to deploying these solutions, the bank's development group:

  • Had difficulty understanding and aligning around requirements
  • Lacked robust testing tools
  • Relied heavily on manual testing
  • Had a limited testing environment that constrained its project delivery capacity

These conditions led to a “high-risk” environment for implementing software changes, often leading to revenue-impacting outages for key applications maintained by the division’s development group.

Realised Benefits

  1. Increased project delivery capacity. The customer increased its project delivery capacity by 100% over the three-year analysis (scaling from 40 projects to 80 projects completed annually) as a result of an improved testing environment and reduction in the frequency of production incidents, which freed up resources to execute a larger volume of projects
     
  2. Incremental headcount avoided. The customer was able to avoid hiring three additional FTEs () into the development group over the three-year analysis as a result of using RTW and RTVS, enabling the organization to increase the scope, frequency, and consistency of continuous testing throughout the development process without substantially increasing its number of development resources.
     
  3. Decreased revenue loss from production incidents. The customer better protected its revenues by reducing the frequency of revenue-impacting production incidents. Prior to implementing RTW and RTVS, approximately 20% of projects had production incidents, which resulted in application outages and cost an average of $100,000 revenue loss per incident.
     
  4. Reduced costs for resolving production incidents. With integration testing performed earlier in the software development life cycle and fewer production incidents, the development team spent less time on bug fixing and design/integration rework, allowing developers to focus on new requirements and other high-value projects and initiatives.
     
  5. Reduced consulting and third-party testing fees. Equipped with a robust set of testing capabilities offered by RTW and RTVS, the customer reduced its reliance on external consultants and third-party integration testing services to supplement its own internal testing efforts, saving fees previously incurred for external support.
     

Return on Investment Over a Three Year period

Year 1 ROI

Year 2 ROI (Cumulative)

Year 3 ROI (Cumulative)

 Payback period

Total benefits (PV)

 Total costs (PV)

398%

860%

1,333%

 

< 2 months

 

$6,273,147

 

($433,149)

 

Key Success Factors

The bank has provided the following as critical success factors: 

  • Business sponsorship 
  • Well defined development methodologies and ensuring one size does not fit all - i.e. flexibility for Waterfall and Agile  
  • Focus on testing and test automation early in the development lifecycle is critical to a successful delivery
  • Financial investment and time to learn the development and testing tools 

Want to understand more about Integration Testing and Service Virtualization? Download the Ranger4 guide: 

page4image19112

Topics: GreenHat, Service Virtualization, test automation