conflicts

[yiftachashkenazi]

CFP Analyzer Integration - System Conflicts Analysis

🚨 Critical System Conflicts (Non-Format Related)

Conflict 1: Model & Prompt Changes Breaking Template Generation Pipeline

Type: Logic/Behavior Conflict
Severity: 🔴 CRITICAL
Files Affected: 3 files

File	Line	Current Issue
services/rag/src/grant_template/cfp_section_analysis.py	1-300	Uses different model (Claude → Gemini) with different prompts
services/rag/src/grant_template/extract_sections.py	200-400	Template extraction logic expects current CFP analysis format
services/rag/src/grant_template/handlers.py	134-142	Hard-coded to use current handle_analyze_cfp function

System Impact:

• Pipeline Breaking: Different models produce different section structures
• Validation Failures: Enhanced JSON schema will reject current data
• Template Corruption: Existing templates may become incompatible

Is this Format-Only?: ❌ NO - This is core logic change that affects system behavior

---

Conflict 2: Enhanced Data Schema vs. Current Type Definitions

Type: Data Structure Conflict
Severity: 🟡 MODERATE
Files Affected: 4 files

File	Line	Current Issue
packages/shared_utils/src/dto.py	13-29	ExtractedSectionDTO missing enhanced fields
packages/db/src/json_objects.py	115-130	CFPAnalysisResult doesn't include comprehensive constraint data
services/rag/src/grant_template/dto.py	16-24	Stage DTOs don't handle enhanced analysis metadata
services/rag/src/grant_template/generate_metadata.py	149-384	Validation functions expect current schema

System Impact:

• Data Loss: Enhanced CFP data won't be captured/stored
• Type Errors: TypeScript/Python type mismatches
• Validation Errors: Enhanced data will fail current validation

Is this Format-Only?: ❌ NO - Missing data fields can break dependent logic

---

Conflict 3: Database Transaction & Job Management

Type: System Integration Conflict
Severity: 🟡 MODERATE
Files Affected: 2 files

File	Line	Current Issue
services/rag/src/grant_template/handlers.py	256-315	handle_save_grant_template expects current CFP analysis structure
packages/db/src/tables.py	48-55	GrantTemplate.cfp_analysis field typed to current CFPAnalysisResult

System Impact:

• Save Failures: Enhanced CFP data may not serialize properly to database
• Job State Corruption: RAG job checkpoints may become invalid
• Transaction Rollbacks: Database saves may fail silently

Is this Format-Only?: ❌ NO - Database integrity and job management are core system functions

---

✅ Format-Only Conflicts (Low Risk)

Conflict 4: API Response Structure Changes

Type: Format/Serialization
Severity: 🟢 LOW
Files Affected: 2 files

File	Line	Current Issue
services/rag/src/grant_template/handlers.py	223-253	GrantLongFormSection creation with current metadata format
services/backend/src/api/routes/grant_templates.py	N/A	API responses may include additional fields

System Impact:

• Frontend Compatibility: Additional fields in API responses
• Serialization: New fields may not serialize correctly

Is this Format-Only?: ✅ YES - Only affects output format, not system logic

---

Conflict 5: Test Data & Assertions

Type: Format/Expectations
Severity: 🟢 LOW
Files Affected: 5+ files

File Pattern	Current Issue
services/rag/tests/grant_template/*_test.py	Test assertions expect current CFP analysis output format
services/rag/tests/integration/grant_application/*_test.py	Integration tests expect current template structure

System Impact:

• Test Failures: All CFP-related tests will fail with new output format
• CI/CD Breaks: Test pipeline will fail until tests are updated

Is this Format-Only?: ✅ YES - Tests just need updated expectations

---

📊 Summary

Critical System Conflicts (Non-Format)

• 3 Conflicts affecting 9 files total
• High Risk: Pipeline breaking changes, data schema conflicts, database integration

Format-Only Conflicts

• 2 Conflicts affecting 7+ files total
• Low Risk: Output format changes, test expectations

Total Impact: 16+ files need changes

Risk Assessment:

• 🔴 3 Critical conflicts that can break core system functionality
• 🟢 2 Low-risk conflicts that only affect output formats

Recommendation:

Use Feature Flag approach to avoid breaking existing system while integrating enhanced CFP analyzer. Focus on the 9 critical files first before addressing format-only changes.

---

🛠️ Detailed File Changes Required

Priority 1: Critical System Files (Must Fix)

1. services/rag/src/grant_template/cfp_section_analysis.py

   • Add feature flag for enhanced vs. current analyzer
   • Create enhanced_handle_analyze_cfp() function
   • Maintain backward compatibility

2. services/rag/src/grant_template/handlers.py

   • Line 134-142: Add conditional logic for analyzer selection
   • Line 256-315: Update save handler for enhanced data
   • Line 223-253: Handle enhanced section metadata

3. services/rag/src/grant_template/extract_sections.py

   • Update section extraction to handle enhanced CFP analysis
   • Add validation for new constraint data
   • Maintain current section format compatibility

4. packages/shared_utils/src/dto.py

   • Extend ExtractedSectionDTO with NotRequired enhanced fields
   • Ensure backward compatibility with existing code

5. packages/db/src/json_objects.py

   • Extend CFPAnalysisResult with enhanced constraint data
   • Add new TypedDict for comprehensive analysis

6. services/rag/src/grant_template/dto.py

   • Update stage DTOs to handle enhanced analysis
   • Add new enhanced stage types

7. services/rag/src/grant_template/generate_metadata.py

   • Update validation functions for enhanced schema
   • Add enhanced template generation logic

8. packages/db/src/tables.py

   • Verify GrantTemplate.cfp_analysis JSON field compatibility
   • No schema changes needed (JSON is flexible)

9. services/backend/src/api/routes/grant_templates.py

   • Handle enhanced data in API responses
   • Ensure serialization compatibility

Priority 2: Format-Only Files (Can Wait)

10. Test files (services/rag/tests/grant_template/*_test.py)

    • Update test assertions for new output format
    • Add new test cases for enhanced features

11. Integration test files (services/rag/tests/integration/*)

    • Update expected template structures
    • Add enhanced CFP analysis test scenarios

---

🚀 Safe Integration Strategy

Phase 1: Add Enhanced System (No Breaking Changes)

• Add enhanced functions alongside current ones
• Use feature flags to control which system is used
• Ensure all current tests still pass

Phase 2: Gradual Migration

• Test enhanced system on specific CFP types
• Compare outputs between current and enhanced systems
• Fix any compatibility issues

Phase 3: Full Rollout

• Switch default to enhanced system
• Update tests to expect enhanced output
• Remove old system after validation

This approach ensures zero downtime and no system breaks during integration.

[yiftachashkenazi]

@Goldziher the integration of new system have some conflicts, this is what (and Claude) found. what do you think? I just to be on the save side and don't break piplines.

[Goldziher]

i dont want feature flags, and i dont want "enhanced", you can update or replace files, but i dont want two implementation.s just one implementation. ok?

[Goldziher]

explain to your friend no backwards compatible code and no legacy code - clean update, change or replace files in place where possible. same for tests.

[yiftachashkenazi]

The truth is, he was more into making a mess, and I was afraid to start :)

[Goldziher]

indeed.