Validate and sanitize LLM responses before rendering

[nate-layman]

Problem

Currently, the app directly renders LLM responses from Gemini without validation or sanitization. This could lead to several issues:

1. Security: No protection against potential injection attacks or malicious content in responses
2. Formatting errors: Unexpected response formats could break the UI or cause rendering issues
3. Missing data: The app assumes responses will always contain expected patterns (e.g., principle extraction via regex)
4. Error handling: Limited validation when responses don't match expected structure

Current Behavior

In app/app.R (lines 655-677), responses are:

• Converted directly to HTML with simple regex replacements
• Parsed for principle names using a single regex pattern
• Displayed without schema validation

Proposed Solutions

1. Response Schema Validation

• Define expected response structure for each character
• Validate that responses contain required fields (rating, feedback, principle recommendation)
• Fall back gracefully when structure doesn't match

2. Content Sanitization

• Sanitize HTML/markdown before rendering
• Escape any potentially dangerous content
• Whitelist allowed HTML tags and attributes

3. Improved Error Handling

• Handle cases where principle extraction fails
• Provide user-friendly error messages
• Log validation failures for debugging

4. Response Testing

• Add unit tests for response parsing logic
• Test edge cases (malformed responses, missing fields, etc.)
• Validate against example responses from each character

Implementation Ideas

# Example validation function
validate_character_response <- function(response, character) {
  # Check for required patterns based on character type
  if (character == "promptulus") {
    # Expect rating (mice), feedback, principle recommendation
    has_rating <- grepl("\\d+/5 mice", response)
    has_principle <- grepl("consider using the \\*\\*[^*]+\\*\\*", response)
    return(has_rating && has_principle)
  }
  # Similar checks for other characters
}

Priority

Medium - The app currently works, but adding validation would improve robustness and security before wider deployment.

Related Files

• app/app.R (lines 655-689) - Response handling logic
• System prompt files - Define expected response structure