NHL Prediction Accuracy | Model Performance Explained
This page explains how to interpret our model's prediction accuracy metrics and provides transparent performance tracking across the 2025–26 NHL season.
Current Season Performance
OOS Accuracy = true holdout result (train 2023–25, test 2025–26 season, 1,056 games) — the honest generalization estimate. In-sample figure includes training data and overstates performance. Brier = probabilistic score (lower is better, 0.25 = random). MAE Total = mean absolute error on predicted total goals (the model's point estimate is the median, which MAE — not RMSE — is the consistent error metric for).
What the Metrics Mean
Accuracy
The fraction of games where the model correctly predicted the winner (the team with win probability >50%). A naive 50/50 coin flip gives ~50%. Our model typically achieves 54–63%, depending on the game window. Hockey is highly random; research suggests ~58% may be near the practical ceiling for single-game NHL predictions.
Brier Score
The Brier score measures probabilistic accuracy: it is the mean squared difference between the predicted probability and the binary outcome (1 = home win, 0 = home loss). A random model predicting 50% every game scores 0.25. Lower Brier scores are better. Our model targets <0.235, indicating meaningful probability calibration beyond chance.
Calibration
Calibration measures whether predicted probabilities match observed frequencies. If the model says 65% in 100 games, those teams should win about 65 of them. Our isotonic calibration post-processing ensures predicted probabilities are honest, not just directionally correct. See the Performance page for calibration curves.
MAE (Totals)
Mean absolute error on predicted total goals (over/under). The model's point estimate is the median of its simulated total, and the median minimizes MAE (the mean minimizes RMSE), so MAE is the error metric consistent with how the point estimate is chosen. A perfect model would score 0. The cross-validation table below reports RMSE instead, because those folds score the mean (expected goals), where RMSE is the consistent metric.
Cross-Validation Results (3 folds)
Walk-forward cross-validation (no future data leakage). Avg Brier: 0.2480 | Avg Log-loss: 0.6894 | Avg RMSE (Totals): 2.406
| Fold | Brier | Log-loss | RMSE Total | Train N | Val N |
|---|---|---|---|---|---|
| 1 | 0.2510 | 0.6957 | 2.446 | 701 | 2,097 |
| 2 | 0.2490 | 0.6914 | 2.390 | 1,399 | 1,399 |
| 3 | 0.2441 | 0.6813 | 2.382 | 2,103 | 695 |
Monthly Accuracy Trend
Win/loss prediction accuracy by calendar month. Larger samples = more stable estimates.
Monthly Breakdown
| Month | Games | Accuracy | Brier Score |
|---|---|---|---|
| 2023-10 | 140 | 60.7% | 0.2206 |
| 2023-11 | 213 | 62.9% | 0.2291 |
| 2023-12 | 219 | 63.5% | 0.2241 |
| 2024-01 | 208 | 60.6% | 0.2221 |
| 2024-02 | 172 | 66.3% | 0.2195 |
| 2024-03 | 228 | 69.3% | 0.2089 |
| 2024-04 | 132 | 61.4% | 0.2253 |
| 2024-10 | 166 | 76.5% | 0.1943 |
| 2024-11 | 220 | 62.7% | 0.2209 |
| 2024-12 | 214 | 70.6% | 0.2077 |
| 2025-01 | 224 | 59.4% | 0.2306 |
| 2025-02 | 122 | 53.3% | 0.2496 |
| 2025-03 | 234 | 65.0% | 0.2275 |
| 2025-04 | 132 | 56.8% | 0.2463 |
| 2025-10 | 180 | 57.8% | 0.2395 |
| 2025-11 | 225 | 53.8% | 0.2397 |
| 2025-12 | 226 | 54.9% | 0.2501 |
| 2026-01 | 240 | 54.6% | 0.2486 |
| 2026-02 | 74 | 67.6% | 0.2221 |
| 2026-03 | 242 | 54.5% | 0.2453 |
| 2026-04 | 125 | 61.6% | 0.2237 |
True Out-of-Sample Accuracy
Cross-validation on historical data can still overestimate real-world performance. Our strictest test: train on 2023–24 and 2024–25 seasons, test on the 2025–26 season (entirely unseen data at training time).
This is the most honest estimate of how the model performs on genuinely new games. The gap between in-sample accuracy (63%) and holdout (54.6%) reflects variance and the inherent difficulty of hockey prediction.
The 2025–26 season accuracy shown above accumulates throughout the season as more games are played, so early-season figures may be noisier.
Why Is Accuracy Bounded?
NHL games have one of the highest upset rates in professional sports. Even the best teams win only ~60% of their games, meaning no model can exceed the "natural ceiling" set by game randomness. Key sources of unpredictability:
- Goalie performance variance (hot/cold streaks)
- Injuries and lineup uncertainty announced close to game time
- Puck luck (post hits, lucky bounces)
- Small sample sizes — 82-game seasons with nightly scheduling
Our goal is not to beat the ceiling — it is to provide well-calibrated probabilities that accurately reflect uncertainty, making them useful for decision-making even when no single prediction is guaranteed.
See also: Live Model Performance | Full Methodology | Today's Predictions