Leading IT Analyst, Forrester, just published their The Forrester Wave™: Continuous Delivery And Release Automation, Q3 2017 analysing part of the DevOps toolchain. We know from experience how hard it is for organisations to choose the tools to include in their toolchain and that decisions made today often need to be changed tomorrow as the technology landscape moves so fast. We recently stood up a new LinkedIn group to facilitate discussion around DevOps toolchains as a result of our customers asking us for more help, blueprints and examples of existing toolchains - feel free to join!
Forrester have identified the 15 most significant vendors in the Continuous Delivery and Release Automation category to be: Atlassian, CA Technologies, Chef Software, Clarive, CloudBees, Electric Cloud, Flexagon, Hewlett Packard Enterprise (HPE), IBM, Micro Focus, Microsoft, Puppet, Red Hat, VMware, and XebiaLabs. Out of these, they identified the leaders in this space to be:
- Electric Cloud
- CA Technologies
The minimum set of capabilities Forrester used to assess each vendor was:
- Integration with continuous integration tools
- Automation capabilities
- Release package modeling
- Moving releases
- Pipeline management
- Pipeline governance
- Vendor usage
- API coverage
- Out-of-the-box third-party plug-ins
- Identification and resolution of vulnerable artifacts
You can see the Forrester Wave on page 10 of the report and on page 11 the breakdown of the scores (except Atlassian - the Wave says that there was incomplete vendor participation here).
This report is undoubtably useful for the granularity in the comparisons that it makes, but it does also raise a number of questions in our minds:
- There is no definitive list of categories for tools that make up a DevOps toolchain, but if there were, would CDRA be one?
- Where does deployment automation stop and release automation start? Or vice versa?
- Do some CI tools extend to CD and some CD tools rely on existing CI capabilities in separate tools? When do we talk about Continuous Deployment?
- Microsoft are listed as a contender as this was their first submission - our pure .Net customers use this platform extensively and report back positively on their experiences with it
Our Opinion: Whilst this tool category is an important one in a DevOps toolchain, we also emphasise the importance of considering the roles of the following tools and their relationship with the other toolchain components - particularly if you want full tracebility and to complete The DevOps LoopTM i.e. measure value from idea to realisation throughout your continuous release cycle:
- Collaboration and communication
- Artifact repositories
- Requirements/product backlog management and refinement
- Service desk and support ticketing
- Application performance management
Want to talk more about DevOps Toolchains? Go here.