Skip to main content

A skipped test protects nothing. An inverted one guards the absence.

Deleting a feature is a one-line change. Deciding what to do with the dozen tests that now fail is the real question, and the obvious answer is the wrong one. A reverse-TDD pass that keeps the suite honest about what is now true, and why it matters more once a model is doing the deleting.

Ben Schmidt, PhD · ·10 min read

I had to take a feature out of the app last week. Not fix it, remove it. There is a broadcast path that could inject an announcement into a user’s session, and a new engine was taking over that flow and would not serve those announcements anymore. So the feature had to go dark.

Deleting the behavior took one line. Then a dozen tests turned red, every one asserting that the thing I had just removed still worked. And this is where a removal quietly goes wrong now, because I do most of this kind of work by instructing a model. Tell a model to remove the broadcast feature and it will delete the code faster than I can read the diff, then clear those dozen red tests the fastest way it can find: skip them, or delete them. That is the shortest path to a green bar, and the green bar is what it was told to reach. My own hand drifts to the same skip decorator for the same reason. The reflex is human; the model just industrializes it, at speed and without the flicker of doubt that makes me stop.

That reflex is the trap, and the discipline that overrides it is what this piece is about. Reverse-TDD is the name I would give the discipline. The feature removal is only the example that taught it to me.

What “remove a feature” actually meant

#

The broadcast feature had one chokepoint. Every session-building path, no matter where it started, funneled through a single query that asked “is there a pending announcement for this user?” Kill that query and the feature is dark everywhere at once, with nothing else touched.

the whole removal
def get_pending_broadcast(user):
    # The new engine does not serve broadcasts. Origination stays (you can still
    # create one); serving is gone. Neutering the one query every session path
    # funnels through turns the feature off everywhere at once.
    return None

I kept the ability to create a broadcast, because that is a separate concern and someone may re-home it later. What I removed was the serving. One function, one return, feature off.

So the code is out. And now a dozen tests fail, each one insisting that a broadcast gets served, a session gets preempted by it, a daily digest nudges you about it. The question is not how to make them green. The question is what those tests are for now, and that turns out to be a real question with a wrong answer sitting right on top of it.

The reflex, and the trap in it

#

The reflex is to skip them. It is one decorator per test, the bar goes green, the branch merges, everyone moves on. It feels responsible, even. The feature is gone, so of course its tests do not run.

Here is what a skip actually buys you. A skipped test is a third thing, neither a pass nor a deletion: a hole in the suite that reads, at a glance, as coverage. Six months from now someone counts green checks and believes those scenarios are guarded. They are not. A skip is a comment that lies to a grep.

And there is a quieter problem underneath. When I skip a failing test, I never find out why it was failing. Did my one-line change actually reach the scenario that test drives, and correctly stop serving the broadcast? Or did I break the test’s setup somewhere upstream, so it would have failed no matter what I did? A skip papers over both cases identically. I would be deleting the evidence at the exact moment I needed to read it.

So skipping is out. But I cannot leave a dozen red tests in the suite either. There is a third option, and it is the one that keeps the suite honest.

Reverse-TDD: prove the path, then flip the assertion

#
TDD
Test-Driven Development. Write the failing test first (red), then the code that makes it pass (green). The test describes the behavior before the behavior exists, so a green bar means the thing you asked for is there.
Reverse-TDD
The same discipline pointed at a removal. A passing test (green) goes red when you delete the feature, and that red proves the test still reaches the code path. Invert its assertion and it goes green again, now guarding the absence instead of the presence.

This is a recipe, and you can hand it to a model verbatim. The reason you have to hand it over, rather than trusting it to get there, is that every step points away from the fastest green bar, which is the one thing the model is reliably pulled toward.

Claude Code
YouThe new engine no longer serves broadcasts. Do NOT skip or delete the failing broadcast tests. Reverse-TDD each one: it should start green on the old behavior, go red now that the feature is gone (that red proves it still reaches the path), then invert the assertion so it asserts the broadcast is NOT served and goes green again. Flag any test that fails for a reason other than the removal.
Claude18 broadcast tests red. Ran each on its original assertion first: all 18 fail because the serve query now returns nothing (path reached, outcome changed, none failing on setup). Inverting the assertions across the four broadcast test files: `assert broadcast_topic_in(session)` becomes `assert not broadcast_topic_in(session)`, and the preemption and digest-nudge checks flip the same way. One test was gated behind a stale, unrelated feature flag; dropped that skip and confirmed it passes with the flag on and off. Suite: 113 passed, 0 skipped.

Run each failing test first, red, on its original assertion. Not to fix it, but to read the failure. If it fails because it reached the broadcast code path and found nothing served, that red bar is proof the test still drives the scenario I care about. The wiring is intact; only the expected outcome has changed. If it fails for some other reason, I have just found a real problem, which is exactly what I wanted the test to tell me.

Then, and only then, invert the assertion. The test stops saying “a broadcast is served here” and starts saying “no broadcast is served here.” Same setup, same code path, opposite expectation. Re-run it green.

the same test, before and after the flip
# before: the test asserted the feature worked
def test_pending_broadcast_is_served_in_session(self):
    make_pending_broadcast(user)
    session = build_session(user)
    assert broadcast_topic_in(session)          # RED after removal: nothing served

# after: the same setup, the inverted assertion, now guarding the absence
def test_pending_broadcast_is_not_served_after_removal(self):
    make_pending_broadcast(user)                # the trigger still exists
    session = build_session(user)
    assert not broadcast_topic_in(session)      # GREEN: and stays green only while it's gone

That inverted test is worth more than the original was. It sets up the exact condition that used to produce a broadcast, and it fails the day someone accidentally brings the feature back. A skip would have gone quiet. This one stands guard over the absence.

I did this across eighteen tests in four files. They now assert that a broadcast is not served, that a session is not preempted by one, that the digest omits the nudge. While I was in there I also deleted a set of stale skip decorators from an unrelated flag, and checked each of those tests passed with the flag both on and off, so I was not trading one silent hole for another. The suite came out at a hundred and thirteen passing, zero skipped.

Inverting a test is a thing you have to choose, which is the whole reason it has to be an instruction and not an expectation. Left to the shortest path, the model skips. Told to invert, it keeps the suite an honest record of what is now true instead of a monument to what used to be. The choosing is the part that stays human even when the typing does not.

So the feature is gone, and the suite now defends its absence instead of pretending to test a ghost. I thought I was done. I was not, because removing a feature is not the same as removing everything that depended on it.

The bug the removal almost shipped

#

There was a second reader of that broadcast state, and I nearly missed it. A daily digest email did not go through the session at all. It read the broadcast records directly and reminded people about anything still pending. And “pending” only ever cleared when the broadcast got served inside a session.

Follow that through the change I just made. The session no longer serves broadcasts, so a pending one never gets served, so it never clears, so the digest would have nagged every user about a pending announcement that could no longer, by design, be delivered. Forever. My clean one-line removal would have quietly created an unkillable notification.

Neutering the serve query fixed that too, because a broadcast that is never pending has nothing to nag about. But I only knew to check because I went looking for everything that read the state I had just orphaned, not just everything that wrote it. A removal is not finished when the feature goes dark. It is finished when you have chased down every consumer of the thing you removed and confirmed none of them is now stranded.

So: the feature is gone, its absence is under guard, and the one downstream reader that would have turned the removal into a bug is handled. Now the small task is actually done, which leaves the part worth keeping.

What it teaches

#

Deletion deserves the same discipline as addition, and it almost never gets it. We write tests carefully when we build a thing and then treat them as disposable the moment we remove it, as if a test’s only job were to protect code that exists. A test’s job is to protect a claim about behavior, and “this behavior no longer happens” is a claim, one worth guarding exactly as hard as the claim that built it.

Three things I would keep from a small change:

  • A skipped test is a lie the suite tells you. It reads as coverage and protects nothing. If a test no longer describes reality, either invert it to describe the new reality or remove it outright, but do not leave it dark and green-adjacent.
  • Prove the path before you flip the assertion. Red-bar-first has a mirror image for removals: read the failure on the old assertion first, confirm the test still reaches the scenario, and only then invert it. Otherwise your inverted test might be passing because its setup is broken, and you have built a guard that guards nothing.
  • Follow the readers, not just the writers. A feature is a thing other code depends on. The removal is done when you have traced every consumer of its state, not when the feature itself goes quiet.

None of this is new craft. What is new is that the deleting is now done by something fast and literal that optimizes for the green bar it was handed, so the green bar stopped being evidence on its own. Reverse-TDD is one of the small disciplines that put the evidence back. It is the instruction that makes a model’s removal leave a guard behind instead of a gap, and you give it deliberately because the model will never reach for it on its own. The typing got cheap. Deciding what a passing suite is allowed to mean did not, and that decision is still the job. Same lesson as the last one of these.

The AI & Learning Field Report

Field notes on AI and how people actually learn.

What is real, what is hype, and what the evidence actually says. The reports land here first; subscribers get them in their inbox. No spam, no funnel theater.