deepeval-metrics

Pluggable DeepEval scaffold for RAG, agents, and LLM apps across Anthropic, Bedrock, Azure OpenAI, and Vertex. Ships traceability, test synthesis, safety/PII gating, multi-turn conversation eval, agentic tool-use scoring, JSON validation, judge benchmarks, hyperparameter sweeps, and pytest CI — one Makefile target per feature.

📚 Stateful, self-correcting RAG research assistant built with LangGraph & Streamlit. Features hybrid search (dense + BM25), Cohere reranking, Tavily web/arXiv claim verification, session-isolated Qdrant storage, Guardrails AI, DeepEval testing, LangSmith tracing, /btw off-topic channel and AWS Fargate deployment configured with Terraform & CI/CD.

Drop-in TruLens evaluation harness for tool-calling LangGraph agents. Swap LLM providers (OpenAI, Anthropic via LiteLLM, Bedrock, Cortex, Gemini, Ollama) with a single env var. Ships with the RAG Triad plus Plan Quality, Plan Adherence, Execution Efficiency, and Logical Consistency metrics.

READ-MAS generates software requirements specifications (SRS) and system designs from natural language queries using multi-agent AI orchestration built on Google ADK.

Improves a production agent system by repairing the evals that judge it. Failures are mined into new eval cases, and gates on both sides control what enters: false negatives the suite would never catch, false positives that refuse a correct change. The system improves by having to pass.

A production-ready LLM evaluation framework built with DeepEval and a custom Groq-powered judge (LLaMA 3.3 70B). Evaluates LLM outputs across 5 key metrics — Answer Relevancy, Faithfulness, Hallucination, Toxicity & G-Eval — without requiring an OpenAI API key. Fast, free-tier compatible.

This is a portable lifecycle benchmark harness for comparing AI coding agents on real QA engineering tasks. It rigorously evaluates agent competencies across test authoring, test repair, root-cause debugging, token efficiency, framework adaptation, and complex browser or mobile automation workflows.

LLM-based RAG evaluation pipeline using DeepEval + Ollama to measure retrieval quality with Contextual Precision, Contextual Recall, and automated evaluation.

RAG pipeline with LLM evaluation using DeepEval, Ollama/Groq judge models, and Langfuse observability.

Successfully developed a multi-domain RAG pipeline that retrieves and answers questions grounded in research papers on lung cancer detection, NSCLC microbiome classification, and Bhutan land cover, with a full safety evaluation suite covering toxicity, information leakage, and scope adherence using DeepEval.

RAG Evaluation Suite on Finance Documents

DeepEval LLM evaluation framework using Claude API — tests GEval, Hallucination, and Prompt Alignment metrics on an AI system that rewrites customer issue reports.

DeepEval LLM evaluation framework using Claude API — tests GEval, Hallucination, Answer Relevancy, Faithfulness, Contextual Precision/Recall, Bias, Toxicity, PII Leakage, and Tool Correctness on a mock AI customer support assistant.