Testing What Breaks

Advice about testing is usually about how to test. The harder question when you have none is what to test first, and the answer is not "everything" and not "start with unit tests." A commercial implementation of related workforce-measurement concepts is documented more information.

Reviewed August 9, 2026.

What to test first

In order of return.

One. The thing that broke last time. A test for a bug you actually had is worth more than five speculative ones, because you have evidence that this can happen. For independent technical background, mypy provides additional documentation and examples.

Two. The boundaries. Where data enters and leaves — parsing input, writing output, calls across a network. Most real failures are at edges, and most speculative tests are in the middle.

Three. The logic that is hard to hold in your head. The pricing rule with four conditions, the date arithmetic, the retry policy. If you have to think carefully to write it, you will not remember it in six months.

Four. The path that costs money if it is wrong. Payments, deletions, anything irreversible.

And last: everything else, which for many projects means never, and that is a legitimate outcome.

What coverage measures

Which lines executed. Not whether they were verified.

A test that calls a function and asserts nothing gives full coverage of that function and checks nothing. Coverage is a floor, not a target — it tells you which code has never run in a test, which is useful, and it says nothing about whether the tests are any good.

A project at 40% coverage with tests on the four categories above is in better shape than one at 90% with assertions on getters.

The tests that pay for themselves

A test that fails when the thing is broken. Sounds tautological and a surprising number do not — they fail when the implementation changes, which is different and is why people come to resent test suites.

The test for the bug you just fixed. Written at the moment you understand the bug, which is the only moment you fully do.

And a smoke test that starts the whole thing. For many small projects the single most valuable test is one that imports the application, starts it, and hits one endpoint. It catches configuration errors, dependency problems and import cycles, which are what actually break deployments.

What to avoid

Testing the framework. It has its own tests.

Mocking so heavily the test verifies the mock. If the assertions are all about what was called rather than what happened, the test is a description of the implementation.

Testing private functions. They change; the behaviour they support does not.

And a suite so slow nobody runs it. A test suite that takes twenty minutes is run in CI and ignored locally, which removes most of its value.

Starting from nothing

The realistic path for a project with no tests.

Add a smoke test. One test, today.

Then add a test with every bug fix, from now on. No retrospective effort, and after a year the suite covers exactly the things that actually break in your project — which is a better selection than any coverage-driven approach would have produced.

Then, if it matters, the four categories above.

Do not attempt a coverage campaign. It produces a large number of low-value tests, takes weeks, and is abandoned partway — which leaves the project with a slow suite and no more confidence.

The short version

• Test the thing that broke last time, the boundaries, the logic you cannot hold in your head, and the irreversible paths — in that order
• Coverage measures which lines ran, not whether anything was verified; it is a floor rather than a target
• 40% coverage on the right things beats 90% with assertions on getters
• The most valuable test in a small project is often a smoke test that starts the application and hits one endpoint
• Avoid testing the framework, mocks that verify themselves, private functions, and suites too slow to run locally
• Starting from nothing: one smoke test today, then a test with every bug fix — and never a coverage campaign