[CodeAlex52]

Fixes #2230

Problem

With a guardrails middleware attached to a streaming workflow, the caller receives Python
repr() dumps of the stream chunks instead of the reply text.

GuardrailsMiddleware.function_middleware_stream buffered the wrapped function's output and
reconstructed it with:

ctx.output = "".join(str(chunk) for chunk in buffered)

NAT's own agents stream typed chunks (ChatResponseChunk), so str(chunk) produces
ChatResponseChunk(...) reprs. The token-by-token path has the same problem: its upstream()
generator feeds str(chunk) into LLMRails.stream_async(), so the output rails evaluate reprs
rather than reply text.

Fix

Both paths now render each chunk with stream_chunk_to_text(), the shared helper the other
defense middlewares (defense_middleware_pii, ..._content_guard, ..._output_verifier) already
use. It routes non-str values through the global type converter — ChatResponseChunk is
registered there in nat/data_models/api_server.py as choices[0].delta.content — and falls back
to str() for values without a text representation, so arbitrary stream items keep working.

Tests

Two cases in packages/nvidia_nat_security/tests/middleware/guardrails/test_nemo_guardrails_middleware.py:

• test_buffered_stream_renders_typed_chunks_as_text — default (buffered) path: two
  ChatResponseChunks in, ["Hello world"] out. Fails before the fix with the repr form.
• test_stream_output_rails_evaluate_text_not_chunk_reprs — stream_output_rails=True: asserts
  the pieces handed to the generator consumed by LLMRails.stream_async() are the reply text.
  Fails before the fix.

Validation

• pytest packages/nvidia_nat_security/tests/middleware → 196 passed (the guardrails middleware
  file alone: 37 passed, +2 over the previous 35).
• yapf --style ./pyproject.toml and ruff check (v0.15.0, the versions pinned in
  .pre-commit-config.yaml) are clean on both files.
• tests/eval/** could not run in this environment (nvidia_nat_eval is not installed in my
  checkout), and I did not run the full-repo suite.

Summary by CodeRabbit

• Bug Fixes
  • Corrected how streaming responses are processed by output guardrails: buffered streams now produce readable combined text, and live streams evaluate each chunk’s text rather than its representation.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/NeMo-Agent-Toolkit/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 81bb1bb3-daee-4d21-bb65-3f68f6ed8cea

📥 Commits

Reviewing files that changed from the base of the PR and between be97c3a and 5217277.

📒 Files selected for processing (2)

• packages/nvidia_nat_security/src/nat/plugins/security/middleware/guardrails/nemo_guardrails_middleware.py
• packages/nvidia_nat_security/tests/middleware/guardrails/test_nemo_guardrails_middleware.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The guardrails middleware now converts stream chunks to text in buffered and live output-rail processing. Tests cover ChatResponseChunk values in both paths.

Changes

Guardrails stream text conversion

Layer / File(s)	Summary
Convert stream chunks to text packages/nvidia_nat_security/src/nat/plugins/security/middleware/guardrails/nemo_guardrails_middleware.py, packages/nvidia_nat_security/tests/middleware/guardrails/test_nemo_guardrails_middleware.py	The middleware uses stream_chunk_to_text instead of str() for buffered and live chunks. Tests assert that buffered chunks produce "Hello world" and that output rails receive "Hello" and " world" as text.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 52172

No actionable merge-blocking issue is established; the change is mergeable after normal checks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title is concise, descriptive, within the approximate 72-character limit, and uses imperative mood. It accurately describes the change to render guardrails stream chunks as text instead of represe…
Linked Issues check		Issue #2230 requires guarded streaming output to return reply text for typed chunks such as ChatResponseChunk. The middleware now uses stream_chunk_to_text() in both the buffered path and the stream_o…
Out of Scope Changes check		The changes stay within issue #2230. They update only guardrails stream chunk conversion and add focused tests for typed chunks in the two affected paths. No unrelated behavior or files are changed.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[willkill07]

/ok to test 5217277

[willkill07]

/merge