镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Align pipelines show preview and add pipeline ID support to history - #6995

Merged
janniklasrose merged 8 commits into
mainfrom
chenyuem-db/pipelines-public-api
Oct 13, 2026
Merged

janniklasrose merged 8 commits into
mainfrom
chenyuem-db/pipelines-public-api

Conversation

@chenyuem-db

@chenyuem-db chenyuem-db commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Changes

PRD: SDP Turbo CLI — Requirements

Implementation files below are relative to cmd/pipelines/.

databricks tables get FULL_NAME already uses the UC GetTable API and returns the full TableInfo, including the schema. We don't need another pipelines schema command.

Command PRD requirement This PR Files
show P0: show the table name Not added because TABLE is already a command argument. —
show P0: preview the first 5 rows by default Keeps the existing default of 10; callers can use --limit 5. show.go (existing path)
show P0: allow more rows with --limit n Keeps the existing override. show.go (existing path)
show P0: support JSON output for agents Keeps the existing JSON format. show.go (existing path)
show P1: hint how to see more rows For the default text preview, shows a hint only when more than 10 rows exist. show.go
show P2: show immediate upstream/external sources Not included. —
history Accept <pipeline id> Accepts IDs via uuid.Parse; direct-ID mode skips bundle loading unless --target is set. history.go
history P0 output requirement: last 5 pipeline runs, if available Deferred until limit behavior with start/end-time filters and API ownership are decided. —
history P0 output requirement: metrics matching the UI Deferred pending public API support; existing state/time output is unchanged. —
history Support --output json Keeps the existing Key / Updates format. history.go (existing path)

Supporting changes:

  • Guard empty history pages when time filters are used.
  • Pin one pipeline ID across the history argument, output, and wire request; an invalid local bundle verifies direct-ID mode does not load it.
  • Cover three show cases: 11 rows shows the hint, exactly 10 does not, and explicit --limit 9 is honored without a hint.
  • Split the release notes for show and history.

Why

Keep this PR scoped to the show hint and pipeline-ID history support. Existing history key mode, pagination, time filters, dry-run inclusion, and output formats remain unchanged.

Tests

Pipelines unit tests, focused acceptance tests, diff checks, changelog validation, and GitHub CI passed.

Implementation assisted by Codex.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: db5a6d3

Run: 38371714814

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 276 20 5:19
✅​ aws-windows-latest-4core-16gb 278 18 3:25
✅​ azure linux-2core-8gb 275 20 5:13
✅​ azure-windows-latest-4core-16gb 277 18 3:22
✅​ gcp linux-2core-8gb 276 20 5:04
✅​ gcp-windows-latest-4core-16gb 278 18 3:24
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:56 azure linux-2core-8gb TestAccept
3:50 aws linux-2core-8gb TestAccept
3:48 gcp linux-2core-8gb TestAccept
3:23 aws-windows-latest-4core-16gb TestAccept
3:22 gcp-windows-latest-4core-16gb TestAccept
3:20 azure-windows-latest-4core-16gb TestAccept

@chenyuem-db
chenyuem-db force-pushed the chenyuem-db/pipelines-public-api branch from 8b617d6 to e10050e Compare October 8, 2026 21:39
Comment thread .nextchanges/cli/pipelines-public-api.md Outdated
Comment thread cmd/pipelines/show.go Outdated
Comment thread cmd/pipelines/show.go
Comment thread cmd/pipelines/show.go Outdated
Comment thread acceptance/pipelines/history/basic/script
Comment thread cmd/pipelines/history.go
Comment thread cmd/pipelines/history.go Outdated
Comment thread cmd/pipelines/history.go
Comment thread cmd/pipelines/show.go
Comment thread cmd/pipelines/show.go
Comment on lines +58 to +60
if checkForMoreRows {
queryLimit++
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if it is strictly better to increase limit by 1 and drop the last row just to have the hint show or not. your call, can also revert to the earlier behaviour and simply print a more generic hint ("there may be more rows, try with --limit <rows>" or similar)

Comment thread acceptance/pipelines/history/basic/databricks.yml
@janniklasrose
janniklasrose added this pull request to the merge queue Oct 13, 2026
Merged via the queue into main with commit d013cdc Oct 13, 2026
28 checks passed
@janniklasrose
janniklasrose deleted the chenyuem-db/pipelines-public-api branch October 13, 2026 09:22
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: d013cdc

Run: 38375958023

Env ❌​FAIL 🔄​flaky ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 8 1420 1140 122:45
❌​ aws-windows-latest-4core-16gb 7 8 1338 1165 149:09
❌​ azure linux-2core-8gb 8 4 1265 1192 111:27
❌​ azure-windows-latest-4core-16gb 7 8 1187 1217 131:57
❌​ gcp linux-2core-8gb 8 2 1256 1196 113:58
❌​ gcp-windows-latest-4core-16gb 7 1184 1221 129:31
30 interesting tests: 22 flaky, 8 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
❌​ TestAccept ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/invariant/no_drift ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
🔄​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=app.yml.tmpl/READPLAN= ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
🔄​ TestAccept/bundle/resources/apps/inline_config ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/apps/inline_config/DMS= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/apps/lifecycle-started-toggle ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/genie_spaces/change-serialized-space ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/genie_spaces/change-serialized-space/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/genie_spaces/change-serialized-space/DMS=true ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/genie_spaces/simple ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/genie_spaces/simple/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/genie_spaces/simple/DMS=true ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/resources/postgres_projects/remove_history_retention/DMS= ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/postgres_projects/remove_history_retention/DMS=true ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/quality_monitors/change_assets_dir ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/quality_monitors/change_assets_dir/DMS= ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/quality_monitors/change_output_schema_name/DMS= ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/quality_monitors/change_output_schema_name/DMS=true ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/quality_monitors/create ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/quality_monitors/create/DMS=true ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/secret_scopes/permissions-collapse ✅​p ✅​p ✅​p 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/secret_scopes/permissions-collapse/DMS= ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/secrets/basic ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/secrets/basic/DMS= ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/secrets/remove-expire-time/DMS= ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/secrets/remove-expire-time/DMS=true ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/secrets/update-expire-time/DMS= ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/secrets/update-expire-time/DMS=true ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
❌​ TestAccept/ssh/bulk-transfer ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
Top 50 slowest tests (at least 2 minutes):
duration env testname
13:48 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
11:20 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
10:58 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:50 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:23 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
10:17 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:01 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:53 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:22 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
8:08 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
7:52 gcp linux-2core-8gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
7:52 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
7:50 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:42 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:37 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:36 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:31 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:30 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:26 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:20 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:20 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:20 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:17 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:10 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:02 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
7:00 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:57 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
6:54 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:53 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
6:53 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:51 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:48 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
6:46 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
6:43 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
6:28 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:23 aws linux-2core-8gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:11 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:10 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:07 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:01 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:00 azure linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=
5:45 aws-windows-latest-4core-16gb TestAccept/bundle/deploy/spark-jar-task/DMS=
5:40 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:35 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:27 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:24 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:21 aws-windows-latest-4core-16gb TestAccept/bundle/deploy/spark-jar-task/DMS=true
5:20 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:08 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:08 aws linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants