What AUC means — and why we put it next to every forecast
Every forecast in w4rn carries a small number next to it: its AUC. It's the single most honest thing we can show you, because it tells you how much to trust the forecast in the first place.
The plain-language version
AUC (the area under the ROC curve) answers one question: if you picked one genuinely risky day and one genuinely calm day at random, how often does the model score the risky one higher? Read it on the scale we use everywhere: 0.50 = coin flip and 1.00 = perfect. Real, honest models live in between: somewhere around 0.6 to 0.7 is a genuine, useful edge; anything near 0.5 is noise. Worth being straight about where ours land — the volatility-regime head scores about 0.6–0.7, inside that band, while the drawdown head is weaker at roughly 0.55–0.59: above the coin-flip floor and still useful as a risk read, but not the stronger of the two. We publish both rather than the flattering one.
Why out-of-sample matters
It's easy to score a high AUC on data the model has already seen — that's just memorising. The only number worth showing is the out-of-sample AUC: how well it scored on data it was never trained on, under walk-forward testing. That's the number we publish, and it's almost always lower than the in-sample one. We'd rather show you an honest 0.62 than a flattering 0.95 that falls apart live.
AUC is not profit
An important caveat we hold ourselves to: a good AUC means the model ranks risk well — it does not mean you can trade it for a profit. Our own walk-forward backtest found the forecast is a risk gauge, not a tradeable edge. That's why we frame w4rn as a tool for sizing and bracing, and pair AUC with calibration so the probabilities mean what they say.
See today's live read in the cockpit → Nothing here is financial advice.