use nlp to improve template creation

[yiftachashkenazi]

NLP Enhancement Architecture & Benchmark Results

Benchmark Results Summary

Benchmark	Baseline Items	Enhanced Items	Change	Quality Improvement	NLP Time	Latency Overhead
ERC Grant	5 items	5 items	+0	+16.9%	536ms	+1.6s (+9.2%)
ICS Medical	6 items	9 items	+3	+120.3%	652ms	+7.6s (+85.8%)
Melanoma Alliance	10 items	6 items	-4	+44.2%	1.8s	+4.7s (+25.7%)
AVERAGE	7 items	6.7 items	-0.3	+60.5%	982ms	+40.2%

Current Architecture (Before NLP Enhancement)

graph TD
    A[CFP Document Upload] --> B[Content Extraction]
    B --> C[Text Sanitization]
    C --> D[RAG Source Creation]
    
    D --> E[Chunk Generation]
    E --> F[LLM Extraction]
    F --> G[Structured CFP Data]
    
    subgraph Current Files
        H[extract_cfp_data.py]
        I[RagSourceData TypedDict]
        J[LLM Prompt Template]
    end
    
    D --> H
    H --> I
    I --> J
    J --> F
    
    style A fill:#e1f5fe
    style G fill:#c8e6c9

Loading

Enhanced Architecture (With NLP Integration)

graph TD
    A[CFP Document Upload] --> B[Content Extraction]
    B --> C[Text Sanitization]
    C --> D{NLP Enhancement Enabled?}
    
    D -->|Yes| E[NLP Analysis Pipeline]
    D -->|No| F[Standard RAG Processing]
    
    E --> G[Pre-loaded spaCy Model]
    G --> H[Sentence Categorization]
    H --> I[NLP Breakdown Formatting]
    I --> J[Enhanced RAG Source Creation]
    
    F --> K[Standard RAG Source Creation]
    
    J --> L[Enhanced LLM Extraction]
    K --> M[Standard LLM Extraction]
    
    L --> N[Structured CFP Data]
    M --> N
    
    subgraph NLP Categories
        O[Money/Budget Requirements\nAverage: 8 items found]
        P[Date/Time Requirements\nAverage: 7 items found]
        Q[Mandatory vs Optional\nAverage: 29 mandatory items]
        R[Format Requirements\nAverage: 4 items found]
        S[Evaluation Criteria\nAverage: 5 items found]
        T[Prohibitions\nAverage: 9 items found]
    end
    
    H --> O
    H --> P
    H --> Q
    H --> R
    H --> S
    H --> T
    
    subgraph Performance Impact
        U[NLP Processing Time\n982ms average]
        V[Total Latency Overhead\n+40.2 percent average]
        W[Quality Improvement\n+60.5 percent average]
    end
    
    E --> U
    U --> V
    V --> W
    
    style E fill:#fff3e0
    style G fill:#f3e5f5
    style W fill:#c8e6c9
    style V fill:#ffecb3

Loading

File Changes Required

graph LR
    subgraph New Files
        A[nlp_breakdown.py\nNEW\nCore NLP Analysis]
        B[nlp_breakdown_test.py\nNEW\nUnit Tests with Benchmarks]
        C[async_nlp.py\nNEW\nAsync Processing]
        D[nlp_metrics.py\nNEW\nPerformance Monitoring]
    end
    
    subgraph Modified Files
        E[extract_cfp_data.py\nMODIFY\nAdd NLP Integration]
        F[main.py\nMODIFY\nPre-load spaCy Model]
        G[RagSourceData TypedDict\nMODIFY\nAdd nlp_breakdown Field]
        H[Prompt Template\nMODIFY\nNLP-Enhanced Prompts]
    end
    
    subgraph Test Files
        I[integration_nlp_test.py\nNEW\nPipeline Integration Tests]
        J[performance_test.py\nNEW\nLatency Benchmarks]
    end
    
    A --> E
    E --> F
    F --> G
    G --> H
    
    A --> B
    E --> I
    C --> J
    D --> J
    
    style A fill:#e8f5e8
    style B fill:#e8f5e8
    style C fill:#e8f5e8
    style D fill:#e8f5e8
    style E fill:#fff3e0
    style F fill:#fff3e0
    style G fill:#fff3e0
    style H fill:#fff3e0
    style I fill:#e3f2fd
    style J fill:#e3f2fd

Loading

NLP Processing Pipeline Detail

graph TD
    A[CFP Text Input] --> B[spaCy Model Processing<br/>Pre-loaded at Startup]
    B --> C[Sentence Segmentation]
    C --> D[Token Analysis]
    D --> E[Named Entity Recognition]
    E --> F[Pattern Matching]

    F --> G[Money / Budget Detection<br/>Keywords: budget, $, cost, funding<br/>Symbols: $, €, £, USD]
    F --> H[Date / Time Detection<br/>Entities: DATE, TIME<br/>Keywords: deadline, due, submit]
    F --> I[Mandatory Analysis<br/>Keywords: must, required, shall<br/>Imperative mood detection]
    F --> J[Format Requirements<br/>Keywords: page, word, font, margin<br/>Number-plus-format pattern]
    F --> K[Evaluation Criteria<br/>Keywords: evaluate, assess, criteria<br/>Pattern: will be evaluated]
    F --> L[Prohibitions<br/>Pattern: not, cannot, shall not<br/>Keywords: prohibited, forbidden]

    G --> M[Structured NLP Breakdown]
    H --> M
    I --> M
    J --> M
    K --> M
    L --> M

    M --> N[Format for LLM Prompt<br/>Categorised guidance]

    subgraph Performance Metrics
        O[ERC: 536 ms<br/>97.1 % confidence]
        P[ICS: 652 ms<br/>79.0 % confidence]
        Q[Melanoma: 1.8 s<br/>100 % confidence]
    end

    B --> O
    B --> P
    B --> Q

    style A fill:#e1f5fe
    style M fill:#f3e5f5
    style N fill:#c8e6c9

Loading

graph LR
    A[Current NLP Implementation<br/>Latency = 100 %] --> B[Optimisation Steps<br/>Efficient batching<br/>Caching]
    B --> C[Optimised NLP<br/>Latency ↓ 50 %]
    C --> D[spaCy Model Pre-loaded<br/>Zero extra load time]

    style A fill:#ffcdd2
    style C fill:#c8e6c9

Loading

[yiftachashkenazi]

@Goldziher please take a look

[Goldziher]

ok, interesting.

So - lets assume we always enable this - we do not need to gate this functionlality. Lets try to keep it as simple as possible.

My question is on the actual latency. You say +40%? +60%? i am a bit confused. Whats the actual performance impact of this. I see the quality increase, which is great, but what is the latency added?

[yiftachashkenazi]

flowchart TD
classDef header fill:#f0f0f0,stroke:#333,stroke-width:1px,font-weight:bold;

subgraph ERC["ERC Grant Benchmark"]
    E1["Metric | Baseline | Enhanced | Improvement"]:::header
    E2["Mandatory Requirements | 9-10 | 10-11 | +10%"]
    E3["Evaluation Criteria | 2 | 5 | +150%"]
    E4["Total Processing Time | 17.1s | 19.4-29.4s | +9-71%"]
    E5["NLP Confidence | N/A | 97% | New capability"]
end

subgraph NIH["NIH Grant Benchmark"]
    N1["Metric | Baseline | Enhanced | Improvement"]:::header
    N2["Content Items | 5 | 6 | +20%"]
    N3["Budget References | 6 | 7 | +17%"]
    N4["Deadline References | 18 | 14-16 | Variable"]
    N5["Total Processing Time | 14.0s | 27.0s | +93%"]
end

subgraph MA["Melanoma Alliance Benchmark"]
    M1["Metric | Baseline | Enhanced | Improvement"]:::header
    M2["Mandatory Requirements | 16 | 19 | +19%"]
    M3["Budget References | 0 | 12 | +∞%"]
    M4["Format Requirements | 3 | 9 | +200%"]
    M5["Evaluation Criteria | 2 | 9 | +350%"]
    M6["Total Processing Time | 18.1s | 22.8s | +26%"]
end

[yiftachashkenazi]

@Goldziher These are slightly more accurate figures, but in any case, when it comes to time, it depends on two factors: whether the spaCy model is loaded and available, and how complex the guide lines text is (more text = more complex = more analysis = more time).
I believe I can cut the duration of this part in half, so its cost would be around 20% more time :roughly 3 to 6 seconds (it needs to be built smartly, using an asynchronous approach).
I’m sure you’ll also be able to help me optimize this part further.

[Goldziher]

ok, sounds good- lets proceed. Create an issue in linear for this please.

[yiftachashkenazi]

thanks! on it :)