OPEN HONEST FOUNDATION · METHOD
Epistemology
Every measuring instrument has a bounded cone of light. Honest knowledge begins by naming where that cone ends.
Asking what an instrument can and cannot settle is the discipline the Foundation works by. Epistemic humility of this kind governs everything the Foundation undertakes, across three domains: the structural correctness of code, the quality and compliance posture of enterprise software, and the linguistic foundations of artificial intelligence.
The cone of light
An instrument shows the structure it was built to show and says nothing about structure outside its cone. A microscope cannot settle a question about galaxies, and a telescope cannot settle a question about cell membranes. The same is true of instruments nobody thinks of as instruments.
A TEST SUITE
It shows the cases its author thought to write. It says nothing about the cases its author did not think of.
A COMPLIANCE AUDIT
It shows what its dimensions were built to expose. It says nothing about properties outside those dimensions.
A BENCHMARK
It shows differences along the axes it measures. It says nothing about differences along the axes it does not.
Many defects, bugs, mistranslations, and misunderstandings are artifacts of the cone rather than features of the problem. A problem that resists more effort inside one instrument sometimes dissolves when a different instrument is brought to bear, or when the limits of the first instrument are named out loud. Refusing to make that distinction produces a pattern the field knows well: more measurement, more testing, and more compute fail to close a gap that a change of instrument would close easily.
CONSCIOUSNESS
It shows whatever reaches awareness, and awareness is a small, edited portion both of the world and of what the mind itself is doing. People report reasons for choices they did not make, and they miss changes in plain sight. Much of what does reach awareness arrives already shaped by language. Almost nobody treats it as an instrument with a cone at all.
Overlapping cones and neighboring instruments
Two instruments can be pointed at the same object from different places. Where the cones they cast overlap, each one reports on the same region under assumptions the other does not share. The Foundation calls an instrument in that position a neighboring instrument, and the term is used in that sense everywhere on this site.
A neighboring instrument is not a rival. It reports within limits of its own, and those limits are different limits. Disagreement from a neighboring instrument tells you where your own cone ends. It does not overturn what your instrument measured inside that cone, and your measurement does not overturn the neighbor's either. The two readings are compared at the boundary where the cones meet.
WHY THE TERM MATTERS
Without it, a finding and an objection from another discipline look like a contest one of them has to lose. Phenomenology, theology, philosophy of mind, and law use words such as intentionality and agency in senses a transformer experiment never touches. Treating those senses as refutations of a measurement, or treating the measurement as a refutation of them, mistakes a boundary for a disagreement.
AI: obscure by design
Frontier AI systems are the highest-stakes instruments in use today, and nobody knows which questions they can answer reliably. Five things about the way they are built make that impossible to find out from outside the company that built them.
Confounded
A frontier model is trained by changing many things at once: the data, the scale, the architecture, the tokenizer, and the training procedure, all tuned together to raise a benchmark score. Because everything varies together, no capability can be traced to any one cause.
Private
The training data, the exact procedure, and much of the architecture are undisclosed for competitive reasons. Outside parties cannot check or reproduce what they are not allowed to see.
Nothing to refute
Nobody commits in advance to a prediction that reality could prove wrong. The model is built until it works, and the explanation of why it works is written afterward.
Three systems, not one
What a user talks to is a trained model, plus a round of reinforcement learning and fine-tuning that reshapes its answers, plus a harness of ordinary hand-written code that decides what the model sees and what comes back. Companies put at least as much work into the harness as into the training. Outside researchers never get the three separately, so no behavior can be assigned to any one of them.
Always changing
The model, the fine-tuning, and the harness are all updated constantly, often without notice and without a version an outsider can pin down. A measurement taken one week describes a system that is gone the next, so nobody can repeat it.
Benchmarks have the same problem. A benchmark may be measuring something other than what its name says. In the Foundation's own cross-linguistic work, swapping one tokenizer for another, with the corpus, the architecture, and the random seed held constant, moved a benchmark score by 7.70 percentage points. A placebo control showed that a separate apparent gain came from the shape of the prompt rather than from anything in its content. The Foundation reported both as artifacts of its own pipeline rather than as findings.
Two things follow. A system built this way cannot support a claim about why it behaves as it does, because the process that produced it was neither controlled nor disclosed. Nobody outside can check it either, because checking would take exactly the controlled, public, repeatable evidence that building it this way never produces.
The frontier labs have the money and the hardware to close this gap, but they are funded and judged on capability at scale. Measurement that is small, controlled, public, and open to refutation does not raise capability, and they cannot publish it without giving away proprietary method. The Foundation treats this as a gap in the public interest rather than as a market opening.
The limits of the Foundation's instruments
Every instrument the Foundation builds is published with a statement of what it cannot measure. They appear on this page because the Foundation applies the same test to its own instruments that it applies to everyone else's. The Standards page and the Language and thought page say what each one does measure.
The Slop Audit scores properties of a codebase and the operations around it. It does not say whether the software was worth building, and it does not judge style. A program written entirely from objects, with no pure functions, can pass it.
The Honest Framework says which categories of defect the structure of the code rules out. It does not say whether the program does what a user needed, and it says nothing about how the team that runs it works.
MÉTRON is one trained French checkpoint. The rest of it is a design, and it is not a published standard of the Foundation.
Three tiers
The Foundation proposes a claim, tests it under rules it stated in advance, and builds the next claim on whatever the evidence supported. Three tiers do different jobs in that cycle, and the Foundation says which tier any statement belongs to.
ONTOLOGICAL
The Foundation proposes its truth claims at this tier. Proposing one settles nothing, and the Foundation writes each claim so that the tier below can send it back.
EMPIRICAL
The Foundation tests those claims here. It runs controlled experiments, registers each one before it collects any data, and reports the result whichever way it comes out.
EPISTEMOLOGICAL
This tier sets the rules that govern the other two. It says what would count as support and where each instrument stops, and every experiment that maps one of those edges adds to it.
A failed probe or a named limit adds an edge to the governing tier, so neither one is a dead end. The Foundation cannot prove a universal claim, so it works by ruling things out.
The convergence-of-signals method
The method follows from the cone-of-light premise. When several independent instruments meet at the same boundary, their agreement supports an inference that no one of them supports alone. The instruments can be controlled experiments, philosophical inquiries, or scholarly and theological traditions. What makes the agreement informative is that the instruments share no assumptions, so a shared error is unlikely to explain it.
Formalizing the method into a peer-reviewable specification is one of the Foundation's stated contributions. A draft with worked examples is in preparation, and it will be released as an open-access deposit with a persistent identifier. A worked case is on the Language and thought page.
The discipline turns inward
An instrument-aware stance is worthless if it exempts its own instruments. The Foundation names the cone of each of its own standards rather than claiming a view from nowhere. It pre-registers its research so that a failed prediction counts against it. It reports its own false positives as readily as its findings, and the cross-linguistic work has published several apparent results as artifacts of its own pipeline. It stakes its central claims on public refutation rather than private assurance, from the Slop Audit's standing challenge to break its generated test suite to the open invitation to refute the Honest Framework's correctness claim.
No one has validated the Slop Audit against deployment outcomes. The studies that would settle whether its scores predict what happens after a system ships are registered and awaiting funding. Until they report, the Foundation's instrument is an open question on the same terms as everyone else's. An instrument that audits others while exempting itself is the failure this Foundation exists to correct.
Cross-traditional convenings
The Foundation organizes convenings that bring together scholars from philosophy, theology, computational linguistics, cognitive science, and adjacent fields around the convergence-of-signals method. The convenings produce peer-reviewable proceedings and method specifications, and the proceedings are published open access.
The Linguistic Telescope, written by the founder as part of his charitable scholarly service, is the focus they are built around. It applies the convergence-of-signals method to the cross-linguistic results and to the argument about language set out on the Language and thought page.
Lineage
The Foundation cites these works as the lineage of its stance. The works remain the property of their authors and publishers.
Robert Rosen on instruments, measurement, and the limits of formal systems. Anticipatory Systems, 1985, and Life Itself, 1991.
Thomas Kuhn on paradigm-bound observation and the dependence of evidence on prior commitments. The Structure of Scientific Revolutions, 1962.
Iain McGilchrist on hemispheric cognition and instrument-aware attention. The Master and His Emissary, 2009, and The Matter With Things, 2021.
Nancy Cartwright on locally applicable laws and the patchwork structure of physical knowledge. The Dappled World, 1999.
Adam Zachary Wasserman, The Robot in the Dark, a manuscript in preparation, where the cone-of-light formulation is set out as one discipline applicable across instruments. It is the founder's personal scholarly work, it predates the Foundation, and it is not a Foundation publication.
The applied form of this discipline, as it is used to build software, is on the Apophatic engineering page.