Resources

The DORA Definition of Change Failure Rate is:

The percentage of production deployments that fail and need to be fixed (hotfix, rollback, roll forward, patch, etc).

What it measures

Change Failure Rate measures the quality of your delivery process in terms of how often you can successfully release changes to your users.

Change Failure Rate is not a measure of the severity the failures. A single critical failure may have greater business impact than a hundred minor failures.

Similarly, Change Failure Rate is not intended to correlate a failure with the change that introduced it, that is what your post incident review is for.

How it is measured

Your Change Failure Rate is calculated using the following factors:

• The number of production deployments attempted
• How many deployments experienced a failure in production.

Data sources

Calculating the number of deployments is obtained from your pipeline: the number of times that a production deployments was performed.

Failed deployments are calculated from your incident management system: when an incident is triggered, it marks the previous deployment as failed.

Our Calculation Methodology

We calculate change failure using 2 methods. Each method has it’s own benefits and drawbacks, and we are experimenting to see which is preferred overall.

We currently report metrics using time based calculations

Method 1: Deployment based

Percentage of the last n completed deployments to production that included a bug or fault that impacted service to the end users.

Method 2: Time Based

Percentage of completed deployments to production in the last n days that included a bug or fault that impacted service to the end users.

Classification

WayFinder calculates a classification tag for change failure rate to allow you to compare your performance with wider industry performance.

Classification tags are based on industry standards and DORA findings.