Watchtower: Why We're Building a Software QA Command Center

Before a release, someone always asks whether QA has finished testing. That should be a quick question. On most teams it isn't.

Spending enough time in software development, we've all probably heard the same conversation more than once.

A feature is ready to go. Everyone wants to release it. Then someone asks the question.

"Has QA finished testing?"

"Are we ready to release?"

The awkward part is that the answer is rarely just yes or no. And every now and then the answer is yes, we tested it, and a bug shows up two days after deployment anyway.

Not because someone missed something obvious, but because software rarely behaves exactly the way we expect it to.

It's not necessarily a sign that developers missed something or that QA didn't test thoroughly enough. It's just the reality of building software. The faster a team moves, the harder it becomes to know exactly where things stand.

We kept seeing the same pattern

At NorthQA, we've spent years working with different teams of different sizes. Different products. Different industries. Different ways of working. The tooling was different too, but the conversations were surprisingly similar.

The same questions kept coming up, regardless of the team or the product.

• Teams want to release faster without sacrificing quality.
• QA engineers want to spend less time repeating the same work.
• Developers want feedback while they're still on the feature, not days later.
• And everyone wants confidence before pressing deploy.

So we started building Watchtower

It didn't start with wanting to run more tests. It didn't start with wanting to run them faster either.

It started with something smaller. What if the answer to "are we ready to release?" was already in one place?

What if QA didn't have to piece that answer together from different tools, test runs, and conversations?

Watchtower is a QA platform, a QA command center. It doesn't replace the tools you test with, and it doesn't change how your team works. It sits above all of that and gives QA a view of its own: what's being tested, how it has been behaving, what's failing and for how long, and what's ready to go out.

Testing done by hand sits in the same place as the automated results, because both are QA's work and both answer the same question.

The goal is to make the entire testing process easier to understand, from what's being tested to what's been tested and what's ready to release.

What's in it?

The first thing Watchtower does is understand your project.

You connect your repository and the setup is essentially done. It finds your specs wherever they live and works out what they're called, so nobody has to describe the folder layout or map anything by hand.

Today, Watchtower works with Playwright and Cypress, giving teams a way to bring their existing tests into one place without changing how those tests are written or organized. The platform is designed to work across different testing setups as well, so support isn't tied to a single framework or a particular way of structuring your test suite.

If you're running both Playwright and Cypress, Watchtower can bring them together. And if your tests are spread across more than one repository, that works too.

After that there's one button that rescans, with a line underneath telling you when it last ran and which commit it saw.

Test Center

Every test your team has, gathered into readable areas instead of raw file paths, so a QA lead can see what's covered without opening the code or asking a developer.

Every test case maps back to a real file in your repo, so nothing in the list is guesswork.

Open one and you get its history rather than just today's result, which is the difference between "this failed" and "this has been failing since March."

Checks you run manually live here too, with their own status and comments, while anything automated takes its status from the runs, so nobody can quietly mark a failing test green.

And when a test moves in the codebase, you can point Watchtower at its new home and it keeps the history, comments, and status that came with it.

Environment, branch, last run, and the real spec file, all on one card.

Patrol

Sometimes you only want to check one feature before merging. Sometimes you want to run an entire area of the application.

Pick a folder or a file and run it. It goes off to your CI runners, and the results come back into Watchtower on their own.

Because Watchtower already understands your project, there's very little setup involved. You simply choose what you want to test and start.

CI runs

Most teams already rely on CI, so Watchtower works alongside it rather than replacing it.

As your pipelines complete, Watchtower updates automatically. You can trigger runs directly from the application if you want to stay there, or continue using your existing workflow.

One detail became surprisingly important while we were building it. If newer results exist than the ones you're looking at, Watchtower tells you. It sounds small, but stale information is often worse than having no information at all.

Go, Review, or No-Go

Most testing tools stop at reporting results. We wanted to go one step further.

Every completed run ends with a simple verdict.

• Go. The available test results provide enough confidence to move forward with the release.
• Review. Something needs another look before the team can confidently release.
• No-Go. The results show issues or gaps that make the release too risky to proceed.

A plain verdict, not a wall of output. Every run lands on one of three.

It's intentionally simple. The verdict isn't there to replace engineering judgement. It's there to give the team a clear starting point for the release conversation, instead of making everyone dig through a wall of logs to figure out where things stand.

Dashboard

The dashboard isn't just a collection of test metrics. It gives you the history behind them. You can see whether pass rates are actually improving, where failures keep coming back, and what changed between runs. Instead of looking at the latest result in isolation, you get the context needed to understand whether the test suite is becoming more reliable or whether the same problems are continuing to surface.

Every run adds a point, so a dip and its recovery are visible without opening a single log.

Two things make those numbers worth trusting. An automated test's status can only come from a run. There's no way to quietly turn a red number green before the release meeting. And if results arrive for a newer change than the one on screen, the dashboard tells you your view is behind instead of letting you read stale numbers. A chart that admits when it's out of date is rarer than it should be.

Looking beyond pass and fail

One feature we're particularly excited about is something we're calling Watchman.

While Watchtower focuses on the tests you've already written, Watchman looks at the changes themselves. It analyses new code, understands what's being introduced, and highlights whether your existing test suite actually covers that new behaviour.

Each stated criterion is checked against the change. Met ones cite the spec file that proves it, gaps say exactly what's missing.

It lands on the same Go, Review, or No-Go, except this time the call is about the work rather than the test results.

A hundred passing tests don't necessarily mean the latest feature was tested. That's the problem we're trying to solve next.

Watchman is still evolving, but it's a direction we're excited to keep exploring.

Where we go from here

One question continues to come up while we're building Watchtower. When a test fails every few runs, what's actually wrong?

Is the test unreliable? Is there a genuine defect? Or is something else happening entirely?

We don't think software should pretend it always knows the answer. Rather than guessing, we'd rather show the evidence and leave the judgement with the people who know the product best.

That's shaped a lot of the decisions we've made while building Watchtower. It's also shaped how we'll continue building it.

We'll keep listening to the teams we work with. We'll keep changing our minds when we learn something new. And we'll keep sharing what we build along the way.

Software changes constantly. Testing changes with it. We're hoping Watchtower makes understanding both just a little easier.

More soon.

---

Watchtower is still in development and will be out for beta testing soon. If you'd like to be among the first teams to use it, join the waitlist.