[cplegendre]

Summary

Adds OpenCode-only subagent configuration support for benchmark experiments.

Features

• Added optional OpenCode reviewer subagent configuration
• Supports independent reviewer:
  • model
  • endpoint
  • context/output limits
  • prompt and description
• Added CLI and API support
• Added example configuration
• Added resume support

Security hardening

• Reviewer endpoints are validated at submission and worker/resume time
• Reviewer hosts participate in restricted agent-phase network allowances
• Wildcard reviewer hostnames are rejected
• OpenRouter reviewer credential detection uses normalized configuration

Validation

• uv run pytest

  • 343 passed
  • 2 skipped

• Ruff on modified Python files: passed

• git diff --check: passed

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 23d370b0-7a41-4e12-a187-3d60250531c5

📥 Commits

Reviewing files that changed from the base of the PR and between 1389cc6 and 619ed32.

📒 Files selected for processing (10)

• README.md
• examples/opencode-subagent.json
• src/coding_agent_bench/agents/configs.py
• src/coding_agent_bench/agents/opencode.py
• src/coding_agent_bench/api.py
• src/coding_agent_bench/builder.py
• src/coding_agent_bench/cli.py
• src/coding_agent_bench/job.py
• src/coding_agent_bench/resume.py
• tests/test_opencode_subagent.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Summary

Summary by CodeRabbit

• New Features

  • OpenCode runs and queued jobs can now use an optional reviewer subagent with a configurable model, endpoint, and instructions.
  • Reviewer endpoints can use OpenRouter or a specified server. Inherited endpoints follow the primary model endpoint when a run is resumed.
  • Added an example configuration for an OpenCode reviewer.

• Bug Fixes

  • Reviewer endpoint checks help prevent invalid or disallowed network destinations from being used in queued jobs.

Walkthrough

The change adds an optional OpenCode reviewer subagent for CLI runs and queued jobs. It validates reviewer settings and endpoints, configures reviewer permissions and network access, and handles reviewer endpoints during job launch and resume.

Changes

OpenCode reviewer configuration

Layer / File(s)	Summary
Reviewer settings and agent setup src/coding_agent_bench/agents/opencode.py, src/coding_agent_bench/agents/configs.py, src/coding_agent_bench/cli.py, src/coding_agent_bench/builder.py, tests/test_opencode_subagent.py	Defines and parses reviewer settings. The CLI and builder pass them to OpenCode configuration, which sets reviewer model limits and permissions. Tests cover configuration, parsing, and local setup.
Queued job validation and launch src/coding_agent_bench/api.py, src/coding_agent_bench/job.py, tests/test_opencode_subagent.py	The API accepts reviewer settings for OpenCode, validates reviewer endpoints, and serializes the configuration into the command. Worker launch revalidates explicit URLs, and the job specification enables the OpenRouter key when the reviewer uses OpenRouter. Tests cover queue handling, host allowlisting, and worker validation.
Resume endpoint handling src/coding_agent_bench/resume.py, tests/test_opencode_subagent.py	Resume preparation validates saved explicit reviewer endpoints. Inherited reviewer endpoints and allowed hosts follow changes to the primary endpoint. Tests cover saved configurations and inherited endpoints.
Configuration example and documentation examples/opencode-subagent.json, README.md	Adds a sample job configuration and documents reviewer settings, endpoint checks, permissions, and resume behavior.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant QueueClient
  participant API
  participant HarborCommandBuilder
  participant Worker
  participant OpenshiftJob
  QueueClient->>API: Submit job with reviewer configuration
  API->>API: Validate reviewer endpoint and build command
  API->>HarborCommandBuilder: Pass reviewer configuration
  Worker->>Worker: Revalidate explicit reviewer URL before launch
  Worker->>OpenshiftJob: Create job specification
  OpenshiftJob->>OpenshiftJob: Enable OpenRouter key when reviewer uses OpenRouter

Loading

Suggested reviewers: rounakbende10

Merge Risk: ⚪ Minimal · up to 619ed

This change adds an optional OpenCode reviewer subagent. When the option is omitted, behavior is unchanged, and reviewer endpoints are validated before use. No concrete merge-blocking issue was identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.53% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 47 functions across 8 files. (2 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the main change: adding OpenCode subagent configuration support.
Description check	✅ Passed	The description directly summarizes the OpenCode-only reviewer configuration, CLI and API support, validation, resume support, and tests.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 25.53% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 47 functions across 8 files. (2 skipped: 2 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}

[cplegendre]

@taagarwa-rh
I’ve opened the PR for the OpenCode subagent configuration support. When you get a chance, could you take a look? Thanks!

[taagarwa-rh]

Hi @cplegendre, this is a great start! My immediate feedback would be to make this more general, so that we can configure subagents in multiple ways. To do this, consider the following:

1. Instead of using a unique schema for bringing in agents, use the existing OpenCode Schema for the agent. Ask the caller to provide a json that gets directly injected into the opencode.json like

{
  "<agent-name>": {
    "description": <description>,
    "prompt": <prompt>,
    "model": <model>,
    ...
  }
}

2. --opencode-subagent is not a valid arguement for harbor. Use --agent-kwarg opencode-config="<opencode-config>" instead, and specify the complete config. Other methods that use that config will merge their changes into the config you specify.
3. Please make it so we can specify multiple subagents by repeating the --opencode-subagent flag.
4. You could allow users to specify additional providers themselves, providing a --opencode-provider and a json that follows the OpenCode provider schema.
5. Please add these new options to the UI.