Part of our audit had been computed on the wrong series. It measured capital as realised only, leaving out the unrealised P&L of positions still open, which means it could not show the drawdown of a position that was still running. Every number in that section was calculated correctly on the wrong input.
It happened for a dull reason. That part was built before the rest of the work, by its own code, and was never checked against the same anchor as everything else.
Before recomputing, we said which way the answer would move, because saying it out loud is how you find out whether you understand the machinery.
The reasoning: the corrected series has the lower Sharpe ratio of the two. A lower Sharpe means a weaker signal-to-noise ratio. So significance should weaken.
It strengthened. Substantially.
The test statistic is computed on long-run variance, not on the naive standard deviation that goes into a Sharpe ratio. And the two series differ in exactly the way that matters for it.
A realised-only series sits flat for days and then jumps at the moment a position closes. Those rare steps inflate the long-run variance estimate badly. A series marked continuously moves a little every day instead, so the same average return sits on a much steadier base.
The corrected series was smoother, so the same mean was statistically firmer. The series that looked better by Sharpe was worse by significance.
We knew both facts. The behaviour of long-run variance was not news to us, and neither was the shape of the two series. We reasoned from the one that was easier to picture, because a Sharpe ratio is a single familiar number and a variance estimator is not.
The fix was not to be more careful next time, which is what people write down and then do not do. That grid now runs through the same anchor check as the rest of the work, and the check raises rather than warns.
The habit that came out of it is smaller and more useful: say the direction out loud before you recompute. It costs nothing, and being wrong about it in public is the fastest way to find the part of the mechanism you only thought you understood.