What AUC means — and why we put it next to every forecast
Every forecast in w4rn carries a small number next to it: its AUC. It's the single most honest thing we can show you, because it tells you how much to trust the forecast in the first place.
The plain-language version
AUC (the area under the ROC curve) answers one question: if you picked one genuinely risky day and one genuinely calm day at random, how often does the model score the risky one higher? An AUC of 0.5 means it can't tell them apart — no better than a coin flip. An AUC of 1.0 would be a perfect oracle. Real, honest models live in between: somewhere around 0.6 to 0.7 is a genuine, useful edge; anything near 0.5 is noise.
Why out-of-sample matters
It's easy to score a high AUC on data the model has already seen — that's just memorising. The only number worth showing is the out-of-sample AUC: how well it scored on data it was never trained on, under walk-forward testing. That's the number we publish, and it's almost always lower than the in-sample one. We'd rather show you an honest 0.62 than a flattering 0.95 that falls apart live.
AUC is not profit
An important caveat we hold ourselves to: a good AUC means the model ranks risk well — it does not mean you can trade it for a profit. Our own walk-forward backtest found the forecast is a risk gauge, not a tradeable edge. That's why we frame w4rn as a tool for sizing and bracing, and pair AUC with calibration so the probabilities mean what they say.
Every forecast in the cockpit → shows its own AUC. We never hide where the model is weak.