The QA Maturity Model Explained: Five Levels, Nine Dimensions

"How good is our QA?" is a question most engineering leaders answer with a feeling. The feeling is usually accurate about direction and useless about what to do next. A maturity model turns the feeling into a position on a map, so the next step is a specific place rather than "better".

This post explains the model behind our free QA maturity assessment: what the levels mean, what the nine dimensions measure, how the score is calculated, and what it deliberately does not claim.

Five levels, one idea

Maturity models for software processes go back to the Capability Maturity Model from the late 1980s. Ours borrows its five-level shape because the shape is right: each level describes who or what quality depends on.

Level 1, Ad hoc. Quality depends on individuals remembering to check things. It works until someone is on leave, the deadline moves, or the person who knew the system leaves.

Level 2, Repeatable. Testing happens consistently and is written down, but it is mostly manual and mostly at the end. Quality is a stage the work passes through rather than a property of how it is built.

Level 3, Defined. You have real automation, a pipeline, and agreed ways of working. The gaps now tend to be in the areas nobody owns: environments, non-functional testing, or what actually blocks a release.

Level 4, Managed. Quality is measured and enforced rather than assumed. You catch most defects before users do, and you can say why you believe a release is ready.

Level 5, Optimising. Quality is designed in, measured over time, and used to make release decisions. Defects are treated as signals about the process, not just as tickets.

The progression is from people to process to measurement. That is the whole idea. A team at level 2 that hires a brilliant tester has not moved up a level, because the practice still depends on that one person. A team at level 3 that loses a brilliant tester has not moved down, because the regression suite runs regardless.

The nine dimensions

A single level hides more than it reveals, so the assessment scores nine dimensions separately and derives the level from the total. Each dimension is one question with four answers, scored 0 to 3. The dimensions are the things that, in our experience, actually separate teams that ship reliably from teams that do not.

1. Regression. How regression testing is run before a release: by whoever is free, against a script, partly automated, or fully automated on every build.
2. Automation coverage. How much of the critical path is covered by automated tests, and whether those tests are maintained.
3. Pipeline gating. Whether a failing test blocks anything, and whether the block holds. A pipeline whose red builds get merged anyway gives you the cost of testing without the benefit.
4. Environments. Whether you have somewhere to test that is not production and not a single shared staging box with hand-made data.
5. Defect management. Whether bugs are logged, triaged on a schedule with agreed severities, and followed by a root cause review.
6. Performance and security. Whether non-functional testing happens never, after an incident, before major releases, or continuously with thresholds in the pipeline.
7. Release readiness. Whether "ready to ship" is a judgement call, a checklist, agreed exit criteria, or exit criteria enforced automatically with evidence attached.
8. When QA gets involved. After development, near the end, at sprint start, or during refinement helping shape acceptance criteria.
9. Metrics. Whether you track nothing, bug counts, escape rate and coverage, or trends over time that feed release decisions.

Notice that only three of the nine are about tests. The rest are about process, environment and decisions. Teams that fixate on test count tend to plateau at level 3, because the thing holding them back is not more tests.

How the score works

Nine questions, maximum three points each, 27 points in total. The level is a percentage band of that total:

• Level 1 up to 25 percent
• Level 2 from 26 percent
• Level 3 from 52 percent
• Level 4 from 74 percent
• Level 5 from 93 percent

The bands are not equal because the levels are not equally hard. Going from Ad hoc to Repeatable is mostly writing things down. Going from Managed to Optimising means almost every dimension is at its top answer, which very few teams achieve and even fewer sustain.

The assessment also reports your two weakest dimensions as gaps and your strongest as strengths. For most teams the gaps are more useful than the level. The level tells you where you are. The gaps tell you what to do on Monday.

Reading your result

A few patterns we see often enough to name.

High automation, low gating. Lots of tests, run on every build, and a failure blocks nothing. This team has paid for level 3 and is getting level 2. The fix is a policy change, not an engineering one, and it is the highest-return change on the whole list.

Good process, no metrics. Everything is written down and followed, and nobody can say whether it is working. These teams tend to argue about quality in retros because there is no number to settle it. Start by tracking escape rate, which is defects found in production divided by defects found in total.

Late involvement. Strong on everything except dimension 8. QA sees the feature when it is built, finds the problems that should have been caught in refinement, and the team concludes QA is slow. Moving one QA engineer into refinement sessions usually fixes it inside a quarter.

Environment-bound. Automation is solid but runs against one shared staging environment that is broken half the time. Every other dimension is capped by this one. Fix the environment before writing another test.

What the model does not claim

We built this as a conversation starter, not a certification, and we want to be clear about its limits.

It does not benchmark you against other companies. We could invent a percentile, and some tools do, but we do not have a dataset that would make it honest. The score compares you to the top answer on each question, nothing else.

It cannot see your code, your pipeline or your team. Nine questions answered by one person is a self-report, and self-reports are optimistic. If two people on the same team get different levels, that disagreement is itself useful information.

It does not say that level 5 is the goal for everyone. A two-person team shipping an internal tool does not need automated exit criteria. The right level is the one where the cost of the next defect no longer justifies the cost of the next improvement. For most product teams with paying users, that is somewhere around level 4.

Take it, then talk about it

The assessment takes about three minutes, needs no signup, and gives you a shareable link so your team can see the same result. Take it, send the link to whoever disagrees with you about QA, and compare answers. That conversation is the point.

If you want an outside view, book a 30-minute QA audit and bring your result. We will go through the two weakest dimensions and tell you what we would do first.