镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Backport .fullsend/ directory to release-v0.7, release-v0.8, release-v0.9 to restore review-agent coverage #3625

Open

Description

@fullsend-ai-retro

What happened

On PR #3622 targeting release-v0.7, the review agent dispatch (run 38546064382, job dispatch / Review) failed with:

✗ Environment validation failed
Error: validating env: 1 unresolved host variable(s):
    FULLSEND_APP_SET is not set (referenced by env.runner[FULLSEND_APP_SET])

Tracing the workflow graph showed this is not isolated: 23 runs on 2026-10-13 and 2026-10-14 failed with the identical error, and every one of them targeted release-v0.7, release-v0.8, or release-v0.9. Main-branch PRs succeed in the same window (e.g. run 38546047332).

The divergence is the .fullsend/ directory:

  • main ships .fullsend/customized/harness/review.yaml whose base: pins fullsend-ai/agents@ce2eedd0.../harness/review.yaml. That ref has no FULLSEND_APP_SET reference, so dispatch passes host-env validation.
  • release-v0.7, release-v0.8, release-v0.9 have no .fullsend/ at all (GET repos/conforma/cli/contents/.fullsend?ref=release-v0.7 → 404). The dispatch therefore resolves harness/review.yaml from a newer agents-repo ref (38faabd0...), which added FULLSEND_APP_SET: "${FULLSEND_APP_SET}" under forge.github.env.runner at line ~98 without a :- default. The host dispatch workflow in fullsend-ai/fullsend does not export FULLSEND_APP_SET, so strict validation aborts.

First observed failure of this signature: run 38446667490 on 2026-10-13 18:25 UTC. The same renovate/release-v0.7-github-actions branch had succeeded 16 hours earlier (run 38337025984 at 02:28 UTC), indicating a tooling-side change pulled in a newer base ref between the two runs.

What could go better

Release-branch PRs — including security backports like this one (openssl, glibc, curl, pcre2) — are merging without any agent review because the review dispatch cannot even start. Humans are catching nothing concerning on these small bumps today, but the safety net is absent precisely on the branches most sensitive to regressions. Two secondary gaps compound it:

  1. Dispatch failures are invisible on the PR itself — the status_notifications.comment.completion: on_failure hook in .fullsend/config.yaml runs when the agent reports failure, not when the dispatch stage fails before the agent runs. The only signal is a red X on the GitHub Actions check list.
  2. The agents-repo harness treats a missing host env var as a hard failure rather than defaulting to empty, which makes any repo whose .fullsend/ pin drifts fragile in the same way.

Confidence: high on root cause and scope — the error message, the ref divergence, and the branch pattern are all directly observable. Lower confidence on "best fix location": backporting .fullsend/ solves it immediately for this repo, but the underlying harness-strictness and dispatch-silence problems are platform-level and would benefit other repos too. This proposal targets the repo-level fix because it is self-contained and unblocks release-branch review coverage today; the platform fixes can be filed separately once the repo-owner confirms the backport is desired.

Proposed change

On each of release-v0.7, release-v0.8, and release-v0.9, create a .fullsend/ directory mirroring main's layout:

  • .fullsend/config.yaml — copy from main, adjusting any branch-specific values if needed.
  • .fullsend/customized/harness/review.yaml — copy from main. The pinned base ref (currently fullsend-ai/agents@ce2eedd0...) is known to pass env validation.
  • .fullsend/customized/harness/retro.yaml and .fullsend/customized/skills/ — copy from main for parity.

Smallest viable change: just add .fullsend/customized/harness/review.yaml (with the known-good base: pin and the REVIEW_FINDING_SEVERITY_THRESHOLD: high override), since that alone will shadow the broken default resolution path. The full backport is preferred so release-branch behaviour matches main for every agent, not just review.

Consider adding a lightweight CI check (or a Renovate preset) that fails if .fullsend/customized/harness/review.yaml on a release branch drifts from main's structure, so future bumps don't silently re-break this.

Validation criteria

After the backport, open a trivial PR against each of release-v0.7, release-v0.8, release-v0.9 (e.g., a comment-only change) and confirm the dispatch / Review job in the resulting fullsend.yaml run completes success and that the review agent posts a review (approval, comment, or change request) on the PR within the usual ~15–20 min window. The next ~5 Renovate/RHSA bumps against release branches should also show a successful review dispatch rather than the FULLSEND_APP_SET is not set failure.


Generated by retro agent from #3622

Activity

  1. fullsend-ai-triage commented on Oct 14, 2026

    @fullsend-ai-triage

    Triage summary

    This is a well-scoped repo-level bug and the proposed backport is self-contained. Independently confirmed the key facts: .fullsend/ is present on main (config.yaml, customized/harness/{review,retro}.yaml) and returns 404 on each of release-v0.7, release-v0.8, and release-v0.9. No open PR currently addresses this.

    Scope note. The files to add live under .fullsend/customized/harness/ — these are harness configuration consumed by the dispatch workflow, not GitHub Actions workflow files, so the usual workflow-file permission constraint does not apply here.

    Severity: high. Release branches are precisely where the automated review safety net matters most (security backports land here), and today it is absent on all three.

    Suggested implementation order.

    1. Mirror main's .fullsend/ layout onto each release branch via one PR per branch (three PRs total). Keep the known-good base: pin in review.yaml and the REVIEW_FINDING_SEVERITY_THRESHOLD: high override.
    2. Review config.yaml per branch for anything that should legitimately differ (e.g., branch-scoped defaults); when in doubt copy verbatim and note the question in the PR for maintainer review.
    3. The drift-detection CI check / Renovate preset mentioned in the proposal is a reasonable follow-up but is out of scope for the backport itself — recommend tracking it as a separate issue.

    Proposed test case (what to verify after each backport PR merges):

    For each branch in {release-v0.7, release-v0.8, release-v0.9}:
      1. Open a trivial PR (comment-only).
      2. Wait for the fullsend.yaml workflow run.
      Expect: dispatch / Review completes success (no FULLSEND_APP_SET error)
              and the review agent posts a review within ~15–20 min.
      Regression guard: watch the next ~5 Renovate/RHSA bumps per branch
                        for continued success.
    

    Labels: Bug affecting agent dispatch tooling on three specific release branches; labels reflect the bug nature, agentic/tooling/ci subsystems, and the release branches impacted.


    Next steps:

    • /fs-code — agent creates a PR to implement this issue
    • /fs-code <your instruction> — agent implements with your specific guidance
  2. fullsend-ai-coder commented on Oct 14, 2026

    @fullsend-ai-coder
    Contributor

    ⚠️ Post-code script failed — Post-script failed (exit code 141)

    The code agent completed, but the post-code script failed before finishing.

    Workflow run: https://github.com/conforma/cli/actions/runs/38564900715

    Details:
    Post-code script failed before push or PR creation completed.
    Please check the workflow logs for full details and retry with /fs-code if appropriate.

  3. fullsend-ai-coder commented on Oct 14, 2026

    @fullsend-ai-coder
    Contributor

    🤖 Finished Code · ❌ Failure (post-script /home/runner/work/cli/cli/.fullsend/.fullsend-cache/resources/sha256/d81f7714542b53ba7a9d77e1ffccef3e0981d91a868dc0d072dbea6546b14807/scripts/post-code.sh failed: exit status 141) · Started 12:37 PM UTC · Completed 12:43 PM UTC

    Commit: f12db6d · View workflow run →

    Runtime: claude · Model: opus → claude-opus-4-7 · Effort: high · Cost: $3.23

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions