[林SO (Linxiushen)]

Why are these changes needed?

TimeSeriesEstimator.score() passes predictions as y_true to sklearn.metrics.r2_score() when no metric is specified. R² is not symmetric: this uses prediction variance as its denominator and reports an incorrect default score. For example, predictions [7, 9, 7, 9] against references [6, 10, 8, 12] score -2.0 instead of 0.4.

Pass the references first. Add native SeasonalNaive regressions for DataFrame and TimeSeriesDataset inputs, with constant-reference, perfect-forecast and explicit-metric controls. AutoML.score() delegates to the estimator's score; the training objective is outside this change.

Related issue number

None; found while inspecting the time-series scoring path.

Checks

• Ran pre-commit run --all-files and the configured hooks on both changed files; all passed.
• No documentation change needed; this corrects the existing default R² calculation.
• Added regression tests: unchanged base has 2 failures and 6 passing controls; the fix passes all 8. Regression and nearby time-series tests: 27 passed, 28 deselected. git diff --check and pip check passed.
• Upstream CI has not run for this branch.

Validation used Windows/Python 3.12, NumPy 1.26.4, pandas 2.2.3, SciPy 1.14.1 and scikit-learn 1.6.1. Tests use the local seasonal random-walk forecaster and synthetic data, with no downloads or external services. No real-model, GPU or performance claim.

AI assistance disclosure: Codex developed the fix and tests, and a separate agent reviewed the exact change and evidence.

[Copilot]

🟢 Approval recommended

No active review issues are reported for the default time-series R² scoring correction.

No open issues 🎉

What changed in this PR

Corrects the default R² calculation in time-series estimator scoring by passing reference values before predictions, so the score uses reference variance.

Changes:

• Fixes the argument order without changing explicit-metric loss behavior.
• Adds synthetic SeasonalNaive regression tests for DataFrame and TimeSeriesDataset inputs, including constant-reference, perfect-forecast, and explicit-metric controls.

File	Description
test/​automl/​test_time_series_score.py	Adds regression tests and controls for time-series scoring.
flaml/​automl/​time_series/​ts_model.py	Corrects the default r2_score argument order.

_{Review effort: Balanced}

[Li Jiang (thinkall)]

Review cycle 20261103-1941; exact source a55ffc314b5213c105785f4394c4238b69734f35.

Completed an overall review of the entire two-file R2 correction and regression tests with review-code (Low, actual gpt-6.1-sol) plus independent native validation. No actionable issue was found in the proposed change: default R2 now uses reference variance, while explicit R2/MAE loss conventions remain unchanged.

Local command on each native Windows interpreter:
python -m pytest -q --import-mode=importlib -p no:cacheprovider --color=no test\automl\test_time_series_score.py

Result: 8 passed on Python3.10.19/NumPy1.26.4 and 8 passed on Python3.13.5/NumPy2.3.3. The same tests against exact prior main produced 2 expected failures and 6 passing controls per stack: the old default returns -2.0 instead of 0.4 for both direct and dataset inputs. Actual public import roots were verified; no baseline source was changed.

Review evidence is committed locally as 821b28b5e6e45348d562a7ecd9d5aaa1605981a5; pre-commit run --all-files passed before that evidence-only commit. Contributor source was not edited or pushed.

Fresh current-head gates pass: all review threads resolved, Balanced Copilot approval recommended, all applicable check contexts successful, and actual current-head Build/docs/formatting workflow runs completed successfully. The PR release job is intentionally skipped, not a passed deployment.

This signoff covers the proposed R2 fix, not unrelated existing main-branch findings or an unexecuted full forecast/Spark/TFT/GPU/performance/minimum-version matrix.

Posted by thinkall-agent-auto-reviewer