镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

docs: rerun time series forecasting notebook with saved results - #1592

Merged
Li Jiang (thinkall) merged 3 commits into
microsoft:mainfrom
thinkall:rerun/1584-time-series-forecast
Sep 8, 2026
Merged

Li Jiang (thinkall) merged 3 commits into
microsoft:mainfrom
thinkall:rerun/1584-time-series-forecast

Conversation

@thinkall

@thinkall Li Jiang (thinkall) commented Sep 7, 2026 •

Copy link
Copy Markdown
Collaborator

Why are these changes needed?

Rerun notebook/automl_time_series_forecast.ipynb from a local checkout of the latest GitHub main, commit d687a4d, and commit fresh tables, predictions, metrics, training logs, and plots. This includes the train-only discrete-label threshold from #1584 by Amir Fathi (@AmirF194), already merged upstream.

Restore the previous %pip install -q "flaml[automl,forecast]" ... cell and remove the Git URL pin and extra commit-provenance code. Every notebook cell source now exactly matches the earlier good run (fa9cf0d70).

The execution environment used an editable installation of the local main checkout. Imports were verified to resolve to that checkout, including after the notebook's installation cell. Current main has also been merged into this PR branch.

The end-to-end rerun also required updating the notebook's installation instructions for panel forecasting and hcrystalball, replacing the removed pandas fillna(method=...) calls, and using the actual volume target in the panel example. Keep the CO2 test data separate from the panel test data so the final comparison runs correctly, use explicit prediction/target arguments for MAPE, and label the classification validation metric as accuracy.

No library implementation changes. Original data splits, search spaces, and training budgets are retained. Consecutive notebook progress streams are merged using nbconvert, and local filesystem paths are redacted from stream output; predictions, metrics, and figures are unchanged.

Running results

All 47 code cells executed sequentially in a fresh Python 3.12.12 kernel on Windows, with zero cell errors. Full execution took approximately 33 minutes, including panel training and the ARIMA/SARIMA comparisons. The local checkout's main branch, HEAD, editable import path, and all 88 FLAML Python source files were verified.

Example Held-out result
Discrete sales Accuracy: 0.7333333333
CO2, FLAML MAPE: 0.0006941877
CO2, default Prophet MAPE: 0.0011382263
CO2, auto ARIMA MAPE: 0.0032017190
CO2, auto SARIMA MAPE: 0.0007927568
Energy consumption MAPE: 0.1096034038
Panel sales sMAPE: 64.46%

The CO2 search evaluated all eight model families and selected Prophet, with 152 Prophet trials. Its MAPE is back to the earlier good run's level (0.000708), rather than the previous run's 0.007631. The 240-second budget, seed, model settings, and data splits were not changed. These are single-run results from a time-budgeted search.

The notebook also retains the panel MAPE output (2.0512e15); percentage error is dominated by zero-volume targets in this dataset.

The discrete example retains 150 training rows and 30 test rows. Its threshold is the training-only mean, 16833.4, rather than the full-series mean, 17020.144444; both splits use the same training threshold.

Execution used FLAML 2.6.0 from the local main checkout at d687a4ddb3445e38e03ba4206a538721bedfe35e, not a Git-reference package installation. Other dependencies: NumPy 2.5.2, pandas 3.0.5, scikit-learn 1.9.0, statsmodels 0.15.0, Prophet 1.4.0, pmdarima 2.1.1, hcrystalball 0.1.12, PyTorch 2.13.0, and pytorch-forecasting 1.8.0.

Related issue number

Related to #1584 and #1175.

Checks

  • I've used pre-commit run --all-files to lint the changes in this PR (all hooks passed).
  • I've included the corresponding documentation changes.
  • I've rerun the entire notebook and checked sequential execution counts, absence of errors, the train-only label threshold, and saved figures.
  • I've made sure all auto checks have passed.

Li Jiang (thinkall) and others added 3 commits September 7, 2026 18:27
Include the training-only discrete-label fix from microsoft#1584 and refresh every notebook result. Fix notebook dependencies, the panel target, and CO2 comparison state for a complete clean-kernel execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 85d1d99b-3e7a-4103-8c27-5b311ad324db
Merge bc4bc02 to resolve the notebook conflict after microsoft#1584. Pin and verify that exact FLAML revision and replace all outputs with a fresh 47-cell execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 85d1d99b-3e7a-4103-8c27-5b311ad324db
Merge main at d687a4d. Restore the previous notebook install and import cells, remove the Git package pin, and replace all outputs with a fresh run using an editable local main checkout.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 85d1d99b-3e7a-4103-8c27-5b311ad324db
@thinkall
Li Jiang (thinkall) added this pull request to the merge queue Sep 8, 2026
Merged via the queue into microsoft:main with commit cf0672d Sep 8, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants