OPEN HONEST FOUNDATION · METHOD

Apophatic engineering

A measuring instrument cannot report a gap it is structurally unable to see. Teams find only the defects they expect to find, and serious defects stay hidden until something breaks in production. More testing does not help.

Apophatic engineering is a discipline for widening what a team can know and act on about the software it builds. It names the limits of the instruments already in use, adds instruments that see past those limits, and then removes the mechanisms behind the failures those instruments find. The instruments are the test suites, monitors, audits, and type checkers that report on a system of production.

The Open Honest Foundation developed apophatic engineering from the epistemology set out in The Robot in the Dark, by the Foundation's founder, Adam Zachary Wasserman. That book sets out how to find an instrument's limits and stops there. Apophatic engineering adds the step of building past them.

The rule is elimination rather than mitigation. A defect that cannot occur needs no handling, no monitoring, and no debugging.

Phase one: understand the limits of your ability to observe

A QA engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd.

First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.

It is tempting to believe that we cannot know what we do not know, and that trying is futile. In fact there are techniques that do find what we do not know. Using them starts with understanding the limits of our current ability to observe, because no method can penetrate the darkness until we know where the darkness begins. Hazard and operability studies, mutation testing, and model checking all work this way.

All three use expectations from outside the system under examination. Expectations written by a system's own builders can only name failures those builders already knew about.

THREE FIELDS ALREADY DO THIS

Hazard and operability studies walk a fixed list of deviation words against every element: no flow, more flow, reverse flow, part of, other than. The list comes from how plants fail in general, not from this plant.

Mutation testing breaks the code on purpose to see whether anything notices. It attacks the checker rather than the thing checked.

Model checking takes an invariant from a person and hunts for a counterexample nobody imagined.

Phase two: make the failure mode inaccessible

The usual response to a failure is to add something: a check, a retry, an alert, a runbook. Each of those leaves the failure possible, and each adds code that can fail on its own.

Phase two removes instead. Understanding the mechanism is what makes the removal cover every case that shares it, rather than the one case that broke.

WHAT GETS REMOVED

Unbounded values. A value that can grow without limit gives a program an unlimited number of possible conditions, and no test suite covers them all. Limit it to a fixed set of cases and the program cannot reach a condition that nobody tested.

Inputs that cannot exist. A function states what kinds of value it accepts. One that accepts only a date on the calendar can never be given February 30, so the code inside never checks for it and that error cannot occur.

Shared values. Two parts of a program running at the same time can both change the same value, and the outcome depends on which one got there first. Take away the sharing, or take away the changing, and that bug cannot happen.

An Industrial Pedigree

Walter Shewhart's control chart tells a plant which variation has an assignable cause and which is chance.

W. Edwards Deming's third point is to cease dependence on inspection to achieve quality, and to eliminate the need for inspection by building quality into the product in the first place.

Toyota runs with little inventory, because a thin buffer forces problems into the open instead of hiding them. The line stops on an abnormality so nothing gets worked around. Asking why five times reaches the mechanism. Shigeo Shingo's poka-yoke, or mistake-proofing, comes last. A part is shaped so it fits only one way, and a fixture will not close unless every piece is in place, so the worker cannot make the error.

Two examples from the Slop Audit

Three indicators for one measure

Most coverage tools print one number, the percentage of lines of code that have been run at least once by a test. Some parts of the code need to be tested more than once, because that piece of code can produce more than one answer.

Slop Audit indicator L1.19 measures how much of the decision space the tests reach. A function with four yes-or-no choices has sixteen possible paths. If the tests exercise twelve of them, the number is twelve out of sixteen.

That number depends on knowing the sixteen. Apophatic engineering adds two more indicators to decide whether the sixteen can be known.

L1.18 asks whether the program's shared state can be sorted into a finite list of cases. Code that keeps a running count with no upper bound has no finite list, so there is no total to divide by. A coverage tool will print a percentage anyway.

L1.20 asks whether the tests give the same answer twice. A test suite that reports different results on consecutive passes has a defect that needs to be uncovered.

Seven measurements not yet in the standard

In October 2026 the Foundation checked an implementation of the Slop Audit against the standard. It found seven measurements that were not yet in the standard. The Foundation is adding all seven as candidate indicators. After it adds them, the standard will name what the implementation already measures.

Two of the seven come from apophatic engineering:

The Silence Index finds the branches that the test suite never follows. The Slop Audit then writes a test for each branch and runs it. When one of those tests passes, the branch worked and nobody had checked it. When one fails, the test suite had missed a real defect. The Slop Audit reports both counts.
A test runner cannot force two threads to collide in a chosen order. A model checker walks through those orders itself. The Slop Audit runs one over the code that two threads can reach at the same time, and reports how much of that code the checker covered.

How the standards use it

Both published standards are built this way.

Slop Audit

The audit reports a not-checked state rather than folding it into a pass, so a reader can tell the difference between a clean result and an absent one. Its dimensions came from outside the codebases it scores, drawn from named compliance frameworks rather than from the conventions of any one team. Every commit to the implementation has to pass the Slop Audit.

Honest Framework

The specification removes categories of defect rather than catching them later. A function whose signature precludes a class of failure cannot exhibit that failure, so there is nothing to test for and nothing to monitor. Mutable state is made finite and deterministic so that every state a program can enter is known in advance and can be tested. The framework also generates tests from the specification itself. Types are checked three times: when the architecture is declared, when the code is committed, and while it runs. Every commit to the framework has to pass the generated tests, all three type checks, and the Slop Audit.

Both are on the Standards page.

Where the word comes from

Scientists have put the word to work for some time. Bengtsson, Weis, and Życzkowski took an apophatic approach to the geometry of quantum states, describing the set by simple features it does not have. Bridewell and Isaac named a method in consciousness science apophatic, validating computational models as negative data about what consciousness is not. The word itself is borrowed from negative theology, where the divine is described by what it is not.

What is set out here applies the same principle to the building of software. Three fields arrived at the same move. Neither of the earlier two anticipates the engineering phases above, and neither needs to.

Bengtsson, I., Weis, S., and Życzkowski, K. (2011). Geometry of the set of mixed quantum states: an apophatic approach. · Bridewell, W., and Isaac, A. M. C. (2021). Apophatic science: how computational modeling can explain consciousness. Neuroscience of Consciousness.