镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Preserve forecasting target names for validation Series - #1656

Open
student-shivamm wants to merge 2 commits into
microsoft:mainfrom
student-shivamm:codex/forecast-series-target-name
Open

student-shivamm wants to merge 2 commits into
microsoft:mainfrom
student-shivamm:codex/forecast-series-target-name

Conversation

@student-shivamm

Copy link
Copy Markdown
Contributor

Why are these changes needed?

An unnamed or differently named validation-label Series becomes a separate
column when normalized for time-series forecasting. For training target sales,
an unnamed y_val creates column 0 rather than sales. This leaves missing
validation targets and makes 0 a spurious feature; a later public predict()
call raises KeyError: 0. Passing the same labels as an array works.

Reproduction:

import numpy as np
import pandas as pd
from flaml import AutoML

train = pd.DataFrame({
    'ds': pd.date_range('2024-01-01', periods=20),
    'sales': np.arange(20, dtype=float),
})
future = pd.DataFrame({'ds': pd.date_range('2024-01-21', periods=3)})
automl = AutoML()
automl.fit(
    dataframe=train, label='sales', X_val=future,
    y_val=pd.Series([20.0, 21.0, 22.0]),
    task='ts_forecast', period=3, estimator_list=['naive'],
    metric='mae', max_iter=1, time_budget=10, retrain_full=False,
    mlflow_logging=False, verbose=0,
)
automl.predict(future)  # KeyError: 0 on current main

Assign target_names to the temporary Series-derived frame, matching the existing
array branch. Preserve input objects, target dtype and positional row alignment.
No new dependencies or forecasting algorithm changes.

Related issue number

No matching report found in targeted searches. PR #1614 changes timestamp-index
promotion and alignment in nearby code but retains Series-derived column names;
its patch does not address this validation-target mismatch.

Checks

  • Identical new regressions: 8 failed / 4 passed on main 798cacb, 12 passed
    with this patch
    . Covers three Series names and three feature-index kinds,
    input preservation and real fit/predict equivalence with the NumPy path.

  • New and adjacent time-series data tests: 24 passed.

  • Full configured pre-commit run passes both changed files, including pinned
    Black 23.3.0 and Ruff 0.0.261. git diff --check also passes.

  • Windows/Python3.12.14, pandas3.0.6, NumPy1.26.4, sklearn1.9.1,
    statsmodels0.15.0. Full optional suites and platform/remote CI were not run.

  • I've used pre-commit to lint the changed files.

  • I've included the needed regression tests.

  • I've made sure all remote auto checks have passed (not run yet).

No documentation change is needed for this input-normalization correction.

AI assistance: OpenAI Codex researched, implemented and ran these local checks.
No independent human review, production incident or model-quality gain is claimed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the forecasting target-name normalization change.

No open issues 🎉

What changed in this PR

Fixes forecasting validation-label normalization so pandas Series use the requested target name, preventing missing targets, spurious features, and subsequent prediction failures.

Changes:

  • Assigns target names to Series-derived frames, matching the existing array behavior.
  • Adds regression tests for Series names, feature indexes, positional alignment, input preservation, and fit/predict equivalence with NumPy targets.
File Description
test/​automl/​test_ts_series_targets.py Adds normalization and end-to-end forecasting regression tests.
flaml/​automl/​time_series/​ts_data.py Applies requested target names when converting Series to a DataFrame.

Review effort: Balanced

@thinkall Li Jiang (thinkall) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review cycle 20261106-0310; exact source 278ebc7bf5b4c7d80fe00f33e88a20b42167595e.

Bounded validation completed; independent review remains incomplete and approval is withheld. Inspected the complete two-file change and its normalization callers. The Series-derived frame is given the requested target names before positional index alignment, matching the existing NumPy path.

Executed on both native Windows/Python3.10.19 (NumPy1.26.4/pandas2.3.3) and Python3.13.5 (NumPy2.3.3/pandas3.0.5):
python -m pytest -q --import-mode=importlib -p no:cacheprovider --color=no test\automl\test_ts_series_targets.py

Result: 12 passed per stack, 24 total, with no skipped cases. The exact-main counterfactual ran the same new test file externally and produced 8 expected failures / 4 passing controls per stack. Actual imported public source/normalization paths were verified, tested input-preservation assertions passed, and no baseline source was edited. This covers Series names, positional indices/input preservation and the real naive public fit/predict path. max_iter=1 is not actual HPO/sampled-season feasibility coverage and does not clear unrelated existing main-branch forecasting findings.

The required independent review-code execution was not dispatched while the authenticated CLI prerequisite remains unresolved after prior selected-credential-authority/model-catalog failures. No reviewer model or clean skill signoff is claimed, and no credential/model/permission override was used. Native test success or peer feedback is not a substitute for that review.

Fresh exact-head gates: 0 unresolved review thread(s); Balanced Copilot recommends approval. Non-success applicable check contexts: build (ubuntu-latest, 3.12):in_progress/None. Actual workflow holds: Build:in_progress/None. An intentionally skipped PR release job is not passed deployment evidence. No full optional/forecast/Spark/TFT/GPU/performance matrix was run locally.

Posted by thinkall-agent-auto-reviewer

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the validation-label normalization change.

No open issues 🎉

Review effort: Balanced

@thinkall Li Jiang (thinkall) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review cycle 20261106-1600; exact source 672e61070290ae13a91e8a5adcb1f56b4c676266.

The branch was updated with a fresh permission/conflict/head guard. Complete-tree reconstruction matches the exact previous feature tree merged with current main 46bce0ce989aae0271e68e5daac5b8a9b0522b87; no unrelated merge edits were found.

Independent review is incomplete. The shared merge delta is planned Low under review-code, but its authenticated CLI prerequisite remains unresolved. No reviewer model ran and no new clean skill artifact was produced. Earlier feature findings and scope qualifications are retained, not converted into a new whole-feature clean vote.

Bounded shared-main validation: the Spark-backend/R2 tests passed 15 cases on each native Windows Python 3.10 and 3.13 stack. The same external test files against the immediate previous main produced six expected failures and nine passing controls per stack. Spark tests use a controlled backend, not an actual Spark cluster; these checks do not cover the whole feature or optional workload matrix.

The exact refreshed Series-target head additionally passed all 12 naming/index/input-preservation/public-fit-predict tests on each native stack. Their max_iter=1 paths are bounded normalization/refit checks, not HPO or sampled-season feasibility coverage.

Fresh gates: 0 unresolved thread(s); explicit current-head Balanced approval recommendation: yes; 10 non-success applicable context(s). Actual current-head workflow holds: Build:in_progress/None. Workflow permission approval is not validation success. No PR approval, foreign-thread mutation or review dismissal was performed.

Posted by thinkall-agent-auto-reviewer

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants