Guardrails: Implement response phase

[liavweiss]

Summary

• Implement response-side guardrail evaluation in the ai_guardrails
  filter, enabling NeMo to inspect upstream responses before they reach
  the client.
• Add on_response_body using block_in_place to bridge the sync
  Pingora constraint into the async GuardProvider::evaluate call
  (tracked in Investigate async on_response_body support #51 for a proper async solution).
• Unify record_verdict to handle both request and response phases:
  request-phase blocks return a clean 403; response-phase blocks replace
  the body with a JSON error payload padded to the committed
  Content-Length (Pingora commits headers before on_response_body
  runs, so status code cannot be changed).
• Add extract_response_messages to parse OpenAI Chat Completion
  response format (choices[].message), fail-closed on unrecognized
  bodies.
• Add fit_to_committed_length utility that pads (space) or truncates
  the replacement body to match the original Content-Length, preventing
  PrematureBodyEnd framing errors.
• Remove the phase.response: true rejection guard from from_config
  and the #[dead_code] suppression on GuardPhase::Response.
• Add 18 new tests covering response body access, skip conditions,
  validation (fail-closed), provider verdicts (pass/block/error via
  wiremock), and fit_to_committed_length edge cases (equal, padding,
  truncation, None body).
• Update nemo-guardrails.yaml example to document both request and
  response phase behavior.

Validation

• Unit tests: 66 guardrails tests pass (cargo test -p praxis-ai-filters --lib -- guardrails)
• Manual testing against real NeMo Guardrails instance (port 18000) and mock vLLM upstream
• Verified response-phase blocking produces valid JSON error with correct padding
• Verified request-phase behavior unchanged (403 on block, pass-through on success)
• make lint (no new warnings)

Checklist

• I reviewed every changed line and can explain the change.
• New capabilities include an example config and functional example test.
• User-facing behavior and generated documentation are updated.
• Performance-sensitive changes include appropriate benchmark or load-test evidence.
• Commits are signed and include a Signed-off-by trailer.

Breaking changes

None. The phase.response config field already existed with a default of
false. Existing configs that omit it or set it to false are unaffected.

Fix

#50

[praxis-bot]

PR Review

Summary: Implements response-side guardrail evaluation with block_in_place bridging and body padding/truncation to match committed Content-Length.

Overall: Implementation is mostly sound with good test coverage. Two medium-severity findings require fixes to prevent UTF-8 corruption and ensure fail-closed behavior.

Severity	Count
Critical	0
Large	0
Medium	2

Findings without inline placement

None - all findings are inline on changed code.

[praxis-bot]

[Medium] UTF-8 boundary corruption risk in truncation path

When the error JSON replacement is larger than the original body, Bytes::from(replacement).slice(0..original_len) truncates at an arbitrary byte offset. If the reason field contains multi-byte Unicode characters (e.g., non-ASCII rail names), truncation can split a multi-byte UTF-8 sequence, producing invalid UTF-8 in the response body.

Fix by truncating at a UTF-8 character boundary, then padding:

std::cmp::Ordering::Greater => {
    tracing::warn!(
        new_len = replacement.len(),
        original_len,
        "ai_guardrails: replacement body larger than committed Content-Length; truncating",
    );
    // Truncate at a character boundary to avoid invalid UTF-8.
    let safe = std::str::from_utf8(&replacement[..original_len])
        .map(|s| s.len())
        .unwrap_or_else(|e| e.valid_up_to());
    let mut result = replacement;
    result.truncate(safe);
    result.resize(original_len, b' ');
    Bytes::from(result)
},

This also avoids the extra allocation from Bytes::from(replacement).slice(...).

[praxis-bot]

[Medium] Silently skips choices without message field (fail-open behavior)

The filter_map silently discards any choice lacking a "message" field. If an upstream response has 3 choices but only 2 contain "message" keys, the filter validates only the 2 parseable messages and forwards the entire response (including the un-validated choice) to the client. In a fail-closed security filter, unparseable content should block the response.

Fix by validating all choices contain messages:

if let Some(choices) = json.get_mut("choices").and_then(|c| c.as_array_mut()) {
    let num_choices = choices.len();
    let messages: Vec<serde_json::Value> = choices
        .iter_mut()
        .filter_map(|c| c.get_mut("message").map(std::mem::take))
        .collect();
    if messages.is_empty() {
        return Err("ai_guardrails: response body does not contain recognizable choices".into());
    }
    if messages.len() != num_choices {
        return Err(
            format!(
                "ai_guardrails: {num_choices} choices but only {} contain a message field",
                messages.len(),
            )
            .into(),
        );
    }
    return Ok(messages);
}

Add a test case with mixed choices (one with message, one without) asserting error.

[leseb]

@liavweiss did you address the bot comments?

[liavweiss]

@leseb, I addressed them now.
I also ran all the unit and integration tests I ran previously to make sure everything is still working as expected.

[praxis-bot]

PR Re-review

Re-review: new commits since last review on 2026-08-28.

Summary: Both previous findings (UTF-8 truncation and fail-open mixed choices) are addressed. One new finding: streaming response incompatibility.

Severity	Count
Critical	0
Large	1
Medium	0

[praxis-bot]

[Large] Response-phase guardrails will break all streaming/SSE responses.

extract_response_messages expects a {"choices": [{"message": {...}}]} JSON body. Streaming responses (SSE with data: {...} events) are buffered by StreamBuffer into a raw byte blob that is not valid JSON — serde_json::from_slice will fail, producing a FilterError. Depending on failure_mode:

• closed (default for security filters): every streaming response returns 500
• open: streaming responses pass through without guardrail evaluation

Neither outcome is correct. Add a streaming-response check before extract_response_messages so SSE bodies are either skipped with a clear log or handled differently:

if is_streaming_response(ctx) {
    tracing::debug!("ai_guardrails: skipping response-phase evaluation for streaming response");
    return Ok(FilterAction::Continue);
}

You can detect streaming from the response Content-Type header (e.g. text/event-stream) or from Transfer-Encoding: chunked without Content-Length. Also add a test that sends an SSE-shaped body through on_response_body and verifies the filter does not error.

[praxis-bot]

PR Re-review

Re-review: new commits since last review on 2026-08-28.

Previous findings addressed:

• UTF-8 boundary truncation: fixed with from_utf8/valid_up_to + space padding.
• Fail-open choices: fixed with messages.len() != num_choices check.

Summary: Missing integration test and example config exercising the new response-phase guardrails path.

Severity	Count
Critical	0
Large	1
Medium	0

[praxis-bot]

[Large] Missing integration test for response-phase guardrails

This PR adds a significant new capability (response body evaluation and replacement) but only includes unit tests. Per the project's test requirements, new capabilities need:

1. Integration tests
2. Example config in examples/configs/ that exercises the feature
3. Functional integration test for the example config

The example config (nemo-guardrails.yaml) keeps response: false, so existing integration tests don't exercise the response path at all.

Add a dedicated example config (e.g., nemo-guardrails-response.yaml) with response: true and a corresponding integration test that verifies:

• Response-phase block verdict replaces the body with the error JSON and preserves Content-Length framing
• Response-phase pass verdict forwards the body unchanged
• Response-phase provider error fails closed