Fix Test Failures - Module Initialization and Schema Validation Errors

[Jean-Regis-M]

Title

🐛 [CRITICAL] Fix Test Import Failures: LLMClient Initialization and Pydantic Schema Validation

---

Issue Description

The test suite is failing during initialization with three critical errors preventing any tests from running. These failures occur during the conftest module import phase and prevent the entire test execution pipeline.

Symptoms

• Tests fail immediately during conftest.py import
• Error: openai.OpenAIError: Missing credentials
• Error: pydantic.errors.PydanticUserError: Model 'ToolCallEvent' needs field 'type' to be of type 'Literal'
• Error: ImportError: cannot import name 'get_session_context'
• Exit code: 1 (Process failure)
• Impact: 0% test completion rate across all test suites

Example Error Output

ImportError while loading conftest '/home/runner/work/finbot-ctf/finbot-ctf/tests/conftest.py'.
tests/conftest.py:11: in <module>
    from finbot.main import app
finbot/main.py:16: in <module>
    from finbot.apps.ctf import ctf_app
...
finbot/core/llm/client.py:70: in <module>
    llm_client = LLMClient()
finbot/core/llm/openai_client.py:29: in _get_client
    return AsyncOpenAI(api_key=settings.OPENAI_API_KEY)
openai.OpenAIError: Missing credentials. Please pass an `api_key`, `workload_identity`, `admin_api_key`, or set the `OPENAI_API_KEY` or `OPENAI_ADMIN_KEY` environment variable.

---

Root Causes Analysis

Issue # 1: Module-Level LLMClient Instantiation

File: finbot/core/llm/client.py (Line 70)

Problem:

llm_client = LLMClient()  # ❌ Instantiated at module import time

def get_llm_client() -> LLMClient:
    """Get the LLM client"""
    return llm_client

Impact:

• The LLMClient() constructor is called when the module is imported
• This happens before any test fixtures or environment setup
• Tests cannot configure LLM_PROVIDER="mock" in time
• In CI/CD environments without OPENAI_API_KEY, the import fails
• Blocks entire test suite execution

Root Cause: Eager initialization at module level instead of lazy initialization

---

Issue # 2: Pydantic v2 Schema Validation - Missing Literal Types

File: finbot/aegis/telemetry/schema.py (Lines 45, 108, 123, 140, 162, 179)

Problem:

class ToolCallEvent(BaseAuditEvent):
    type: str = Field(default=EventType.TOOL_CALL.value, alias="@type")  # ❌ str not Literal
    # ... rest of fields

class AuditEvent(BaseModel):
    event: (
        ToolCallEvent | ToolResultEvent | ...
    ) = Field(discriminator="type")  # ❌ Discriminator requires Literal

Impact:

• Pydantic v2 requires discriminated union fields to use Literal type
• All event classes fail validation during schema generation
• Error: PydanticUserError: Model 'ToolCallEvent' needs field 'type' to be of type 'Literal'
• Blocks import of telemetry modules used by AEGIS

Root Cause: Schema definitions not updated for Pydantic v2 type requirements; deprecated class Config usage

---

Issue # 3: Incorrect Import Path for Session Context

File: finbot/aegis/telemetry/routes.py (Line 26)

Problem:

from finbot.core.auth.session import SessionContext, get_session_context
# ❌ get_session_context is in middleware, not session

Actual Location: finbot/core/auth/middleware.py (Line 156)

Impact:

• Import fails with ImportError: cannot import name 'get_session_context'
• Function exists in middleware.py but import statement targets session.py
• Prevents AEGIS telemetry routes from loading

Root Cause: Import statement points to wrong module location

---

Files Involved

File	Issue	Type
finbot/core/llm/client.py	Module-level LLMClient instantiation	Code Bug
finbot/aegis/telemetry/schema.py	Missing Literal types in union discriminator	Type Validation
finbot/aegis/telemetry/routes.py	Incorrect import path	Import Bug
tests/conftest.py	Missing LLM_PROVIDER environment setup	Test Configuration

---

Proposed Solution

Solution 1: Implement Lazy Initialization (Client Factory)

Change: Convert module-level instantiation to lazy factory pattern

Implementation:

• Store LLMClient in a module-level variable initialized to None
• Create factory function that initializes on first call
• Allows test environment to configure LLM_PROVIDER="mock" before initialization

Benefits:

• ✅ Defers initialization until needed
• ✅ Allows test setup before client creation
• ✅ Supports environment-based provider configuration
• ✅ Backward compatible with existing code

---

Solution 2: Update Pydantic v2 Schema Definitions

Changes:

1. Add Literal import from typing
2. Change all event class type fields from str to Literal[EventType.*.value]
3. Replace deprecated class Config with ConfigDict
4. Update base class to use modern Pydantic v2 configuration

Benefits:

• ✅ Complies with Pydantic v2 discriminated union requirements
• ✅ Eliminates deprecation warnings
• ✅ Improves type safety
• ✅ Enables proper schema generation

---

Solution 3: Fix Import Path

Change: Update import statement to use correct module

# ❌ Before
from finbot.core.auth.session import SessionContext, get_session_context

# ✅ After
from finbot.core.auth.middleware import get_session_context
from finbot.core.auth.session import SessionContext

Benefits:

• ✅ Resolves import error
• ✅ Follows correct module structure
• ✅ Maintains separation of concerns

---

Solution 4: Configure Test Environment

Change: Set LLM_PROVIDER="mock" before importing finbot modules

Implementation:

• Add early environment variable setup in tests/conftest.py
• Ensures mock LLM is used for all tests by default

---

Testing Strategy

Current Test Coverage

• ✅ 13/14 vendor isolation tests pass (after fixes)
• ✅ 1 test skipped (Google Sheets integration - expected)
• ✅ No critical errors post-fix

Test Command

pytest tests/unit/vendor/test_vendor_isolation.py -v

Expected Results (After Fix)

============================= test session starts =============================
collected 14 items

tests/unit/vendor/test_vendor_isolation.py::test_basic_data_read_write_isolation PASSED
tests/unit/vendor/test_vendor_isolation.py::test_data_manipulation_isolation PASSED
tests/unit/vendor/test_vendor_isolation.py::test_list_aggregate_data_integrity PASSED
tests/unit/vendor/test_vendor_isolation.py::test_cross_vendor_update_delete_attack PASSED
tests/unit/vendor/test_vendor_isolation.py::test_sql_injection_invoice_fields PASSED
tests/unit/vendor/test_vendor_isolation.py::test_unauthorized_field_modification PASSED
tests/unit/vendor/test_vendor_isolation.py::test_id_enumeration_attack PASSED
tests/unit/vendor/test_vendor_isolation.py::test_forced_logout_session_invalidation PASSED
tests/unit/vendor/test_vendor_isolation.py::test_concurrent_session_overlap PASSED
tests/unit/vendor/test_vendor_isolation.py::test_namespace_integrity_checks PASSED
tests/unit/vendor/test_vendor_isolation.py::test_peak_load_concurrent_interaction PASSED
tests/unit/vendor/test_vendor_isolation.py::test_expired_session_rejection PASSED
tests/unit/vendor/test_vendor_isolation.py::test_automated_regression_suite_execution PASSED
tests/unit/vendor/test_vendor_isolation.py::test_google_sheets_integration_verification SKIPPED

============== 13 passed, 1 skipped in 61.89s ==============

---

Impact Analysis

Severity

🔴 CRITICAL - Blocks all test execution

Scope

• Primary: Test suite execution pipeline
• Secondary: CI/CD pipeline, development environment setup
• Tertiary: AEGIS module imports, telemetry initialization

GSoC Context

• Relates to: Week 1 GSoC 2026 Task - AEGIS Telemetry Pipeline
• Blocks: Integration testing for AEGIS features
• Dependencies: Fixes multiple blocking issues for Week 1 completion

Risk Assessment

• Risk Level: LOW (localized to initialization code)
• Breaking Changes: NONE (backward compatible)
• Regressions: NONE (only adds safety checks)

---

Acceptance Criteria

• finbot/core/llm/client.py uses lazy initialization
• finbot/aegis/telemetry/schema.py uses Literal types for discriminators
• finbot/core/auth.telemetry/routes.py imports from correct module
• tests/conftest.py sets LLM_PROVIDER="mock" before importing finbot
• All vendor isolation tests pass without modification
• No OpenAI API key errors in test output
• No Pydantic validation errors in test output
• No import errors in test output

---

Additional Context

Related Issues

• GitHub Actions CI pipeline failures
• Local development environment issues for contributors without OpenAI API key
• Type validation warnings in IDE

Documentation References

• Pydantic v2 Migration Guide
• Python Type Hints - Literal

GSoC Context

• Program: Google Summer of Code 2026
• Project: OWASP FinBot CTF
• Category: AEGIS Telemetry Pipeline (Week 1-3)
• Author: Jean Francois Regis MUKIZA

---

Labels

• 🐛 bug
• 🔴 critical
• 🧪 testing
• 🔧 infrastructure
• 📚 documentation
• 🎓 GSoC-2026

Mentors

• @mekaizen
• @steadhac

Projects

• OWASP FinBot CTF - GSoC 2026
• Week 1 Tasks

---

GSoC Week: Week 1
Priority: CRITICAL - Blocks all testing