China AI Forecasting Visits: Ask for the Backtest
A forecasting chart can conceal hindsight. Ask for the forecast frozen at the original decision date, the comparison baseline and the cost of being wrong.
Direct answer. Request the prediction as it existed at the original planning date, using only information available then. Compare it with a relevant baseline on the same horizon and series, and explain the business consequence of each error.
Freeze the information boundary
During a China AI forecasting visit, choose one anonymized planning cycle. Record the decision date, forecast horizon and data cutoff. Ask whether later corrections, actual orders or revised product information entered the demonstration dataset. A chart reconstructed after the period ended may be useful for exploration but cannot by itself establish prospective forecasting performance.
Ask the host to distinguish a rolling update from the original forecast. Updating a prediction after new information arrives is reasonable; comparing that updated estimate against an earlier baseline without labeling the change is not a like-for-like evaluation.
Build a compact comparison
| Field | Required clarification |
|---|---|
| Origin | When was the forecast produced? |
| Inputs | Which data was available at that moment? |
| Baseline | What would the existing planning process predict? |
| Horizon | Which future periods were evaluated? |
| Consequence | What does overprediction or underprediction cost operationally? |
Segment the results where decisions differ, such as established products versus new launches. Do not assume one error metric adequately describes intermittent and high-volume demand alike. Ask the planning team which measure connects to its decision, and preserve the denominator behind every reported comparison.
NIST AI RMF Core supports documented experimental conditions and evaluation methods. It does not mandate this backtest worksheet or establish an acceptable forecast error. Those are business-specific choices.
Keep the conclusion prospective
The next step may be a shadow planning cycle: freeze both the current-process estimate and the proposed AI forecast before outcomes become known. That creates a more interpretable comparison than a selected historical chart. Define who reviews discrepancies and when the experiment ends before requesting purchasing authority.
Use the demo-versus-production guide to label the host's evidence. Brief your visit with finance and planning colleagues who understand the consequences of forecast errors, not just the appearance of the chart.