CAPL Automation Framework

AI-assisted CLI-based automotive networking validation framework for generating disciplined, protocol-aware validation scenarios, validator-facing CANoe-compatible CAPL assets, execution guidance, traceability, coverage, and portfolio-grade reports as local artifacts.

The current version now includes a real pluggable AI provider layer, a controlled multi-pass agentic refinement loop, deterministic offline fallback, and CAPL-centered validator assistance as the main automation output.

Quick View

One-Line Description

This project is a local command-line engineering tool that converts automotive communication requirements into realistic validation scenarios, AI-assisted CAPL drafts, CANoe execution guidance, and report artifacts written to disk.

What This Project Is

• A CLI-based engineering tool, not a web application
• A local artifact generator that writes outputs to the outputs/ folder
• An AI-assisted CAPL and CANoe workflow accelerator for automotive validators
• A portfolio project focused on realistic automotive networking validation scope

Overview

This project is intentionally not a generic test generator. It stays inside realistic automotive networking validation scope such as CAN, Ethernet, DoIP, diagnostics-related communication behavior, timeout handling, wake-up and sleep behavior, error handling, and recovery. The framework is designed to generate validator-usable CAPL and CANoe guidance without drifting into UI, backend, cloud, manufacturing, or speculative test content.

The core pipeline is:

1. Parse structured requirement input into a normalized requirement model.
2. Build baseline protocol and intent understanding.
3. Run a guarded AI refinement loop to add scenario emphasis, CAPL focus, ambiguity flags, and validator risks.
4. Generate only in-scope automotive validation scenarios plus CAPL and CANoe execution guidance.
5. Optionally analyze CANoe compile or runtime evidence to propose CAPL refinements and validator next steps.
6. Publish JSON, CSV, HTML, CAPL, guide, traceability, coverage, and runtime-review artifacts.

Project Structure

CAPL_Automation_Framework/
|-- .env.example
|-- .gitattributes
|-- .gitignore
|-- LICENSE
|-- app/
|   `-- main.py
|-- engines/
|   |-- agentic_refiner.py
|   |-- ai_orchestrator.py
|   |-- ai_provider.py
|   |-- capl_generator.py
|   |-- canoe_execution_guide.py
|   |-- coverage_tracker.py
|   |-- html_report.py
|   |-- protocol_classifier.py
|   |-- report_writer.py
|   |-- requirement_parser.py
|   |-- runtime_feedback_analyzer.py
|   |-- runtime_feedback_loader.py
|   |-- scenario_generator.py
|   `-- traceability_matrix.py
|-- models/
|   |-- requirement_model.py
|   `-- testcase_model.py
|-- outputs/
|   |-- capl/
|   |-- guides/
|   |-- runtime_reviews/
|   `-- report artifacts generated at runtime
|-- sample_data/
|   |-- requirement_can_sleep_wakeup.json
|   |-- requirement_can_timeout.json
|   |-- requirement_doip_diagnostics.json
|   |-- requirement_ethernet_recovery.json
|   |-- runtime_feedback_portfolio.json
|   `-- requirements_portfolio.json
|-- tests/
|   `-- test_requirement_understanding.py
|-- README.md
`-- requirements.txt

Core Features

Requirement Understanding

The parser and classifier extract or normalize:

• requirement ID, title, summary, protocol, message ID, signal name, service ID
• timing and timeout constraints
• preconditions
• expected and negative behavior
• validation type, priority, risk domain, automation feasibility
• intent classification and validation checkpoints
• AI mode, provider, confidence, feasibility score, ambiguity flags, manual-review questions
• CAPL focus points, CAPL improvement suggestions, validator risks, and accepted versus rejected agentic actions

Scenario Generation Discipline

The scenario engine only generates scenario types supported by the requirement context:

• positive
• negative
• boundary
• timeout
• recovery
• disturbance
• state transition
• protocol compliance

Each scenario is explicitly marked as in-scope automotive networking validation and carries automation feasibility guidance.

The requirement-understanding layer now includes stronger guardrails:

• missing or unknown protocol values do not automatically pass scope control
• service identifiers are not misinterpreted as CAN message IDs
• generic timing phrases such as within 150 ms after wake-up are not silently converted into timeout semantics
• summary-only diagnostic no-response requirements can still trigger the correct negative scenario path

CAPL Strategy

The framework uses realistic CAPL feasibility decisions:

• CAN message monitoring with periodicity and timeout detection can generate executable CAPL monitoring logic.
• Generated CAN CAPL checks both slower-than-expected and faster-than-expected cycle-time deviations using validator-visible Write Window verdict messages.
• CAN sleep or wake-up validation generates monitoring logic plus TODO markers where project-specific triggers are still required.
• Ethernet, DoIP, and diagnostics requirements generate template-only CAPL skeletons instead of fake full automation, and those templates now label placeholder summaries explicitly until project-specific hooks are bound.

The CAPL layer is the highlight of the project:

• AI-assisted requirement understanding drives which timers, event handlers, and observation points are drafted into the script.
• Generated CAPL uses validator-oriented Write Window verdict messages so execution evidence is easier to follow inside CANoe.
• The CANoe guide treats the AI-generated CAPL node as the validator's primary observation and verdict aid.
• AI-generated CAPL improvement suggestions and validator-risk notes stay attached to the script and execution guide so the validator knows what still needs project binding or tolerance confirmation.
• External compile and measurement feedback can now trigger AI runtime diagnosis, CAPL refinement suggestions, and validator next-step guidance after the initial script is generated.
• Template-only scripts still provide practical observation structure for diagnostics and DoIP instead of pretending to be fully executable.

Agentic AI Layer

The framework now contains a controlled agentic loop:

• Pass 1 builds the baseline rule-based interpretation.
• Pass 2 requests structured AI advice for scenario emphasis, CAPL focus points, ambiguity flags, validator risks, and feasibility scoring.
• Pass 3 critiques the initial AI proposal and repairs safe missing guidance such as CAPL observation hints or scenario emphasis.
• Pass 4 escalates unresolved ambiguity into manual-review questions and CAPL improvement notes instead of guessing project-specific details.
• After the four-pass agentic loop, an optional runtime-review stage can consume CANoe compile or measurement evidence and turn it into runtime-specific CAPL refinement suggestions and validator actions.
• AI CAPL focus points and execution hints are preserved only if they stay within protocol-aware validation scope.
• Accepted, repaired, and rejected AI actions are recorded in the JSON and HTML reports.

Reporting

Generated outputs include:

• JSON report
• scenario CSV
• traceability CSV
• HTML report
• CAPL files
• CANoe execution guide markdown files
• runtime-review markdown files
• AI runtime metadata inside the portfolio report

Running The Project

Python 3.11+ is recommended.

python app/main.py

You can control the AI runtime explicitly:

python app/main.py --ai-mode deterministic
python app/main.py --ai-mode openai --ai-model gpt-5
python app/main.py --runtime-feedback sample_data/runtime_feedback_portfolio.json

You can also point to a single requirement file:

python app/main.py --input sample_data/requirement_can_timeout.json --output outputs

If --ai-mode openai is used, set OPENAI_API_KEY first. If credentials are missing or the request fails, the framework falls back safely to deterministic mode and records that in the generated report.

If --runtime-feedback is provided, the framework ingests external CANoe compile or measurement evidence and generates:

• runtime issue summaries
• CAPL refinement suggestions
• validator next steps
• per-requirement runtime review artifacts under outputs/runtime_reviews/

Run the regression tests with:

python -B -m unittest discover -s tests -v

Sample Requirements Included

• REQ-CAN-001: CAN periodic transmit with timeout supervision
• REQ-CAN-002: CAN sleep and wake-up communication behavior
• REQ-ETH-003: Ethernet establishment, logging, and recovery
• REQ-DOIP-004: DoIP diagnostic response timing and invalid payload handling

AI Design

The current AI runtime already supports:

• pluggable provider architecture with deterministic fallback and optional OpenAI structured-output integration
• a guarded multi-pass agentic loop for proposal, critique, repair, and ambiguity escalation
• requirement-level AI outputs such as scenario emphasis, CAPL focus points, feasibility score, validator risks, and manual-review questions
• runtime-feedback analysis that converts CANoe compile or measurement evidence into CAPL refinement suggestions and validator next steps
• explicit reporting of configured provider versus effective provider, effective pass count, confidence, ambiguity counts, and runtime failure counts

The design is also ready for future expansion in these areas:

• richer free-form requirement interpretation
• smarter protocol-specific scenario ranking
• CAPL refinement from real compile logs or trace anomalies
• automated execution-debug recommendations for failed CANoe runs

Design Guardrails

This repository deliberately avoids:

• UI or infotainment UX test generation
• backend or cloud test generation
• manufacturing or mechanical validation content
• speculative scenarios not accepted by real automotive networking validation teams

The goal is to present realistic validation engineering judgment, not artificial feature inflation.