Repository navigation
docs: rerun time series forecasting notebook with saved results - #1592
Merged
Li Jiang (thinkall) merged 3 commits intoSep 8, 2026
Merged
Li Jiang (thinkall) merged 3 commits into
Li Jiang (thinkall) merged 3 commits into
Conversation
Include the training-only discrete-label fix from microsoft#1584 and refresh every notebook result. Fix notebook dependencies, the panel target, and CO2 comparison state for a complete clean-kernel execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85d1d99b-3e7a-4103-8c27-5b311ad324db
Merge bc4bc02 to resolve the notebook conflict after microsoft#1584. Pin and verify that exact FLAML revision and replace all outputs with a fresh 47-cell execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85d1d99b-3e7a-4103-8c27-5b311ad324db
Merge main at d687a4d. Restore the previous notebook install and import cells, remove the Git package pin, and replace all outputs with a fresh run using an editable local main checkout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85d1d99b-3e7a-4103-8c27-5b311ad324db
jianglibigdata
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Rerun
notebook/automl_time_series_forecast.ipynbfrom a local checkout of the latest GitHubmain, commitd687a4d, and commit fresh tables, predictions, metrics, training logs, and plots. This includes the train-only discrete-label threshold from #1584 by Amir Fathi (@AmirF194), already merged upstream.Restore the previous
%pip install -q "flaml[automl,forecast]" ...cell and remove the Git URL pin and extra commit-provenance code. Every notebook cell source now exactly matches the earlier good run (fa9cf0d70).The execution environment used an editable installation of the local
maincheckout. Imports were verified to resolve to that checkout, including after the notebook's installation cell. Currentmainhas also been merged into this PR branch.The end-to-end rerun also required updating the notebook's installation instructions for panel forecasting and
hcrystalball, replacing the removed pandasfillna(method=...)calls, and using the actualvolumetarget in the panel example. Keep the CO2 test data separate from the panel test data so the final comparison runs correctly, use explicit prediction/target arguments for MAPE, and label the classification validation metric as accuracy.No library implementation changes. Original data splits, search spaces, and training budgets are retained. Consecutive notebook progress streams are merged using nbconvert, and local filesystem paths are redacted from stream output; predictions, metrics, and figures are unchanged.
Running results
All 47 code cells executed sequentially in a fresh Python 3.12.12 kernel on Windows, with zero cell errors. Full execution took approximately 33 minutes, including panel training and the ARIMA/SARIMA comparisons. The local checkout's
mainbranch, HEAD, editable import path, and all 88 FLAML Python source files were verified.The CO2 search evaluated all eight model families and selected Prophet, with 152 Prophet trials. Its MAPE is back to the earlier good run's level (
0.000708), rather than the previous run's0.007631. The 240-second budget, seed, model settings, and data splits were not changed. These are single-run results from a time-budgeted search.The notebook also retains the panel MAPE output (
2.0512e15); percentage error is dominated by zero-volume targets in this dataset.The discrete example retains 150 training rows and 30 test rows. Its threshold is the training-only mean, 16833.4, rather than the full-series mean, 17020.144444; both splits use the same training threshold.
Execution used FLAML 2.6.0 from the local
maincheckout atd687a4ddb3445e38e03ba4206a538721bedfe35e, not a Git-reference package installation. Other dependencies: NumPy 2.5.2, pandas 3.0.5, scikit-learn 1.9.0, statsmodels 0.15.0, Prophet 1.4.0, pmdarima 2.1.1, hcrystalball 0.1.12, PyTorch 2.13.0, and pytorch-forecasting 1.8.0.Related issue number
Related to #1584 and #1175.
Checks
pre-commit run --all-filesto lint the changes in this PR (all hooks passed).