PrompTalk

PrompTalk is an open-source Prompt Engineering, Diagnostics, & Lifecycle Management Platform. It serves as a unified system-of-record and development environment for managing LLM instructions across modern software workflows.

Rather than copy-pasting raw strings or relying on ad-hoc LLM playtesting, PrompTalk provides developers with a structured compiler, real-time regex AST diagnostics, Git-like version control, localized regression benchmarking, and multi-language client library bindings (TypeScript, Python, Go) alongside IDE-native tooling (VS Code, Cursor, MCP).

📖 Directory & Quick Links

Getting Started: Node Installation | CLI Tool | Docker Container
API & Integrations: Versioned REST API v1 | TypeScript SDK | Python SDK | Go SDK
IDE & Agent Extensions: Model Context Protocol (MCP) Server | VS Code Extension
Platform Manuals: Architecture Blueprint | Technical Specifications | Industry Benchmarks | Model Registry Specs
Community: Roadmap & Milestone Tracker | Contribution Guidelines | Security Policy

💡 Why PrompTalk?

As AI-driven applications scale, developers face severe prompts management hurdles:

The Playground Dilemma: Browser-based playgrounds (OpenAI Playground, Google AI Studio) are built for single users, isolating templates from source code and version control.
Instruction Fragility: LLM output changes with minor wording adjustments (LLM Drift). Without automatic, localized linter pipelines, prompts are prone to regression.
Variable Injection Spaghetti: Concatenating input placeholders manually via template strings ("Write a summary of " + text) is prone to formatting escapes and prompt injection vulnerabilities.
Platform Lock-in: Different model providers (Gemini, Claude, GPT, DeepSeek) parse instructions differently. A prompt tuned for one model family fails or runs inefficiently on another.

PrompTalk solves these challenges by treating prompt engineering as an engineering discipline. It introduces a central, offline-capable prompt optimization compiler, deterministic scoring heuristics, and automated test-suite execution directly inside local developer workflows.

🛠️ Core Capabilities

1. Unified Prompt Compiler (`promptService`)

Converts raw, informal user inputs (e.g., "help me write SQL counts") into highly organized, production-grade system instruction sheets.

Model-Specific Adapters: The compiler automatically formats system instructions based on the targeted LLM:
- Anthropic Claude: Wraps directives, role contexts, and input variables inside semantic XML tags (<instructions>, <rules>) to maximize attention steering.
- OpenAI GPT / DeepSeek-R1: Uses bold markdown subheaders and rigorous verification boundaries.
- Google Gemini: Uses robust system-instruction formatting with clean structural context and input-to-output schemas.
Dynamic Variable Slotting: Automatically identifies variables (e.g., {{topic}}, {{audience}}) and exposes schema structures so prompts remain reusable.

2. Regex-Based AST Diagnostics & Offline Linter

Detects subtle prompt flaws, formatting bugs, and safety leaks 100% offline without requiring internet connection or remote API consumption.

Catches common syntax errors, excessive punctuation patterns, empty constraints, missing input placeholders, and insecure instruction structures.
Outputs diagnostics matching standard compiler warning models (Severity: Error, Warning, Info), complete with actionable fixSuggestion fields.

3. Prompt Score Engine

Provides quantitative metric evaluations, generating an overall PrompTalk quality certificate (Grade A to F). The scoring system assesses prompts across four distinct axes:

Clarity: Unambiguous directives and objective goals.
Constraints: Negative rules and boundaries (e.g., "Do NOT explain code").
Specificity: Persona mapping, detailed parameters, and clear domain framing.
Variables: Placeholders and customizable injection slots.

4. Git-like Version Control & Branch History

PrompTalk integrates a local in-memory Git-style prompt repository:

Create prompt commits, save prompt revisions, compare differences (diff counts), and rollback prompts to stable versions instantly.

5. Multi-Language SDK Bindings & REST API

Native Client SDKs for TypeScript/Node.js, Python, and Go.
Versioned Gateway endpoints (/api/v1/*) supporting request ID correlation and structured JSON console logs for production telemetry auditing.

6. IDE-Native & Agentic Protocols

MCP Server: Exposes compiler, linter, and recommender tools directly to AI clients like Cursor and Claude Desktop.
VS Code Extension: Direct workspace editor integration providing real-time inline text markers, hover diagnostics, and Command Palette shortcuts.

📊 Feature Comparison Matrix

Feature Module	PrompTalk	Manual String Templates	Generic Cloud Playgrounds	Legacy Prompt Libraries
Offline Diagnostic Linter	Yes (AST-based)	No	No	No
Quantitative Score Engine	Yes (0-100 & Grade)	No	No	No
Model-Specific Formatting	Yes (Auto Adapters)	No	No	No
Git-like Prompt Versioning	Yes (Committed Hist)	No	Limited (Cloud only)	No
Standardized SDK Bindings	Yes (TS, Python, Go)	No	No	No
Local MCP Server	Yes (Cursor & Claude)	No	No	No
Inline VS Code Diagnostics	Yes (Real-time Line)	No	No	No
Zero-Mock Local Benchmarks	Yes (8-Suite Cases)	No	No	No

📐 System Architecture

PrompTalk is built with strict modularity, ensuring that the same underlying compiler and linter drive the Web UI, REST endpoints, CLI, multi-language SDKs, MCP, and VS Code layers.

                      +------------------------------------------+
                      |             PROMPTALK CLIENTS            |
                      +----+--------------+--------------+-------+
                           |              |              |
                           | (HTTP)       | (JSON-RPC)   | (Library)
                           v              v              v
                      +------------+ +------------+ +------------+
                      | VS Code &  | | MCP Agent  | | Multi-SDKs |
                      | CLI Tool   | | Clients    | | (JS/PY/Go) |
                      +----+-------+ +-----+------+ +----+-------+
                           |               |             |
                           v               v             v
  +--------------------------------------------------------------+
  |                   OBSERVABLE GATEWAY LAYER                   |
  |      X-Request-ID Header Injection | Structured Logging      |
  |      Backward-Compatible V1 Router Proxying                  |
  +--------------------------------------------------------------+
  |              CENTRAL BUSINESS SERVICES ENGINE                |
  |                                                              |
  |    +-------------------------+     +--------------------+    |
  |    |     promptService       | <-> |    promptLinter    |    |
  |    | (State, VC, Recommender)|     |  (Offline AST/Reg) |    |
  |    +------------+------------+     +--------------------+    |
  +-----------------|--------------------------------------------+
                    v
  +-------------------------------------+
  |          Google Gemini API          |
  |        (gemini-3.5-flash)           |
  +-------------------------------------+

Observable Gateway Layer: Automatically correlates logs and response headers with tracking IDs (x-request-id). Outputs cycle audits to standard console in clean, single-line JSON format.
Central Business Services Engine: Standardized core libraries encapsulating all core business algorithms. Excluded from dependency bloating to maintain instant start speeds.
Intelligence Pipeline: Interacts with the latest Google Gemini models to perform dynamic meta-prompt optimizations.

📥 Installation

1. Node and Server Setup

To run the full-stack PrompTalk server locally (which hosts the API gateway, developer playground, and MCP tunnel):

# Clone the repository
git clone https://github.com/yourusername/PrompTalk.git
cd PrompTalk

# Install standardized packages
npm install

# Copy environment example and configure your secrets
cp .env.example .env
# Edit .env and paste your GEMINI_API_KEY=your_key

# Start the unified developer container server
npm run dev

The application will launch on port 3000. You can navigate to http://localhost:3000 to interact with the Visual IDE Playground.

2. CLI Tool Setup

The CLI tool allows you to perform prompt audits directly from your local terminal. Ensure the tool is executable, then run commands:

# Verify system CLI behavior
npx tsx cli/promptalk.ts --help

3. Client SDKs installation

TypeScript SDK: Directly imported from sdk/promptalk-ts
Python SDK: Directly imported from sdk/promptalk-py
Go SDK: Directly imported from sdk/promptalk-go

Docker Deployment

Build and package the stateless gateway node as a secure local container:

docker build -t promptalk:v1.0.0 .
docker run -d -p 3000:3000 --env GEMINI_API_KEY="your-key" promptalk:v1.0.0

⚡ Quick Start & Examples

A. CLI Code Audits

Run quick quality checks and diagnostic audits inside automated scripts:

# 1. Auditing a prompt file for syntax and format issues
npx tsx cli/promptalk.ts lint prompts/marketing.md

# 2. Score a local prompt file and output machine-readable JSON
npx tsx cli/promptalk.ts score prompts/coding.md --json

# 3. Compile/Optimize a raw prompt blueprint against a profile
npx tsx cli/promptalk.ts compile prompts/raw.md --profile coding

B. REST API v1 (HTTP Client)

Interact with the stateless prompt engine using standard HTTP packages:

# Trigger standard prompt optimization compiler
curl -X POST http://localhost:3000/api/v1/compile \
  -H "Content-Type: application/json" \
  -H "x-request-id: cli-test-01" \
  -d '{
    "prompt": "write a node server script",
    "modelId": "gemini-1.5-pro",
    "profileId": "coding"
  }'

C. TypeScript SDK Usage

import { PrompTalkClient } from './sdk/promptalk-ts';

const client = new PrompTalkClient({
  baseUrl: 'http://localhost:3000'
});

async function run() {
  // Offline Linter Diagnostics
  const lintReport = await client.lint("Write a code blocks. Do not say hello.");
  console.log(`Lint Score: ${lintReport.score}/100`);

  // LLM Model Recommendation
  const rec = await client.recommend("Create a complex SQL statement for transaction ledger tracking.");
  console.log(`Recommended Target Model: ${rec.recommendedModelId} (Reason: ${rec.reason})`);

  // Prompt Optimization Compiler
  const compilation = await client.compile("help me explain physics", {
    profileId: "creative"
  });
  console.log(`Optimized Instruction:\n${compilation.refinedPrompt}`);
}
run();

D. Python SDK Usage

from sdk.promptalk_py.promptalk import PrompTalkClient

# Connect to the stateless API server
client = PrompTalkClient(base_url="http://localhost:3000")

# Calculate objective quality scoring certificates
score_card = client.score("Translate this text to french: {{text}}")
print(f"Overall Quality Rating: {score_card['overall']}/100 (Grade: {score_card['grade']})")

# Run high-fidelity latency and adherence benchmark regressions
bench_result = client.benchmark("Refactor this python script", category="Coding")
print(f"Expected Outcome: {bench_result['expectedOutcome']}")

E. Go SDK Usage

package main

import (
	"fmt"
	"log"
	"sdk/promptalk-go"
)

func main() {
	client := promptalk.NewClient("http://localhost:3000", "")

	// Direct endpoint comparisons
	diff, err := client.Compare("Write a script.", "You are an expert systems engineer. Write a highly performant shell script.")
	if err != nil {
		log.Fatalf("Error comparing inputs: %v", err)
	}

	fmt.Printf("Visual Score Gain: +%f points\n", diff["scoreImprovement"])
}

F. MCP Integration (Claude & Cursor)

Connect your AI code assistants natively to PrompTalk.

Claude Desktop Configuration (claude_desktop_config.json):

{
  "mcpServers": {
    "promptalk": {
      "command": "npx",
      "args": ["tsx", "src/lib/mcp/server.ts"],
      "env": {
        "GEMINI_API_KEY": "your-key-here"
      }
    }
  }
}

Cursor Setup:

Open Cursor Editor Settings -> Models -> MCP.
Click + Add New MCP Server.
Select Type command, enter name promptalk, and paste command npx tsx src/lib/mcp/server.ts.

G. VS Code Integration

To install and start developing prompts with real-time inline highlighting:

Open the /vscode directory in your VS Code workspace.
Compile and package the extension:
```
cd vscode
npm install
npm run compile
```
Exposes instant workspace command utilities:
- promptalk.score: Rates the currently highlighted prompt block.
- promptalk.compile: Optimizes your draft document against a specific LLM profile.

🗺️ Product Roadmap

  v1.0.0-Stable (CURRENT)                v1.1.0-Release (NEXT)                    v2.0.0-LTS (FUTURE)
 ┌─────────────────────────┐            ┌─────────────────────────┐             ┌──────────────────────────┐
 │ • Stateless v1 Core API │            │ • SQLite Revision DB    │             │ • Federated LLM Tuning   │
 │ • Offline Diagnostics   │            │ • Multi-tenant Orgs     │             │ • Headless WASM Sandbox  │
 │ • SDKs & MCP Servers    │   ────►    │ • Multi-Model Providers │    ────►    │ • Git-Remote Auto Sync   │
 │ • VS Code Extension     │            │ • Schema Export Formats │             │ • Real-time Cost Audits  │
 └─────────────────────────┘            └─────────────────────────┘             └──────────────────────────┘

🟩 Completed in v1.0.0-Stable

Universal Compile Engine: Symmetrical prompt optimizations with Gemini integration.
AST Linter Pipeline: Offline warning highlighting and safety leak protections.
Multi-SDK Bindings: Native TypeScript, Python, and Go client integrations.
MCP Service: Standard tools matching Claude Desktop and Cursor spec.
VS Code Extension: Inline diagnostic markers and Hover Card fix suggestions.
Platform Monitoring: Request ID tracking, health routes, and structured JSON logs.

🟨 Targeted for v1.1.0-Release

Local SQLite Database Store: Persist workspace blueprints across server reboots.
Multi-Model Provider Benchmarking: Compare outputs of optimized prompts side-by-side between Gemini, Claude, and GPT.
JSON Schema Exporter: Export generated dynamic variables into official JSON/YAML formats.

🟦 Scheduled for v2.0.0-LTS

Git-Remote Synchronization: Auto-sync local markdown files with GitHub repository histories.
Headless WASM sandboxed environments: Safely execute compiled codes locally to evaluate correctness.

🎯 Project Scope

PrompTalk focuses on improving developer workflows during prompt design and continuous integration.

What PrompTalk Is

A Prompt Engineering IDE Extension & Workspace Tool: Accelerating prompt writing through linting, diagnostics, and structured formatting compilers.
A Symmetrical API and SDK Engine: Ensuring compilation and scoring heuristics produce identical results regardless of whether executed in JS, Python, Go, or the VS Code editor.
A Prompt Regression Benchmark Harness: Standardizing prompt optimization testing against curated test-cases.

What PrompTalk Is NOT

An LLM Hosting or Serving Platform: It does not host or execute direct inference endpoints for weights.
An Agent Orchestrator: It is not a runtime for multi-agent loops or memory vectors (like LangGraph or AutoGen).
An Authentication Identity Gateway: It relies on user-provided developer API keys for compiling drafts rather than wrapping provider costs.

🚦 Stability Status

We clearly document the stability classification of every module within the repository:

Module / Component	Stability Level	Production Scope Guidance
Core Service (`promptService`)	Stable	Standardized core linter, score generator, and template formatter.
*REST API (`/api/v1/`)**	Stable	Backward-compatible JSON routes with request ID correlation.
CLI Tool (`promptalk`)	Stable	Complete exit codes mapping for automated CI/CD gating.
TypeScript SDK	Stable	Local fallback capabilities and ready for local project import.
Python SDK	Stable	Standard `urllib` HTTP client wrapper for prompt querying.
Go SDK	Stable	Standard HTTP JSON request-response bindings.
MCP Server	Beta	Fully functional with Claude Desktop and Cursor clients.
VS Code Extension	Preview	Live editor diagnostics. Requires manual loading of VSIX artifacts.

🛡️ Support & Compatibility Policy

Node.js Compatibility: Verified on Node.js versions >=18.0.0 (LTS and current).
Browser Compatibility: Active Web UI targets modern evergreen browsers (Chrome, Safari, Firefox, Edge).
Semantic Versioning (SemVer): PrompTalk adheres strictly to SemVer 2.0.0 rules. Major version increments are triggered only if breaking structural changes are introduced to versioned APIs (/api/v1/*).
Dependency Audits: Dependencies are monitored automatically via the project build pipeline.

⚠️ Current Limitations & Caveats

In-Memory Version History: Prompt revision tracking and commits reside in memory. Restarting the Express server clears prior commits. Durable persistence is a primary target of the v1.1.0 roadmap.
Gemini API Requirement: Dynamic prompt compilation requires a valid GEMINI_API_KEY loaded on the server. Offline AST-based linting and scoring run instantly with zero key requirements.
SDK Distribution: Official client libraries are currently bundled directly in this repository's tree under the /sdk folder. They are not yet published to npm, PyPI, or Go proxy registries. To use them, developers can import them locally.
VS Code Packaging: The extension in /vscode must be compiled and loaded manually in developer environments using VSIX files; it is not yet listed on the Visual Studio Marketplace.
Heuristic Scoring: Benchmarking and score numbers are qualitative structural indicators, not absolute models of factual accuracy or alignment.

🤝 Community & Contributions

PrompTalk is fully open-source and welcoming to community developers.

How to contribute: Read CONTRIBUTING.md to set up your local linter, run platform regression tests (npm run test), and submit structured Pull Requests.
Code of Conduct: We hold our community to a positive, welcoming standard. See CODE_OF_CONDUCT.md.
Issues: Report bugs or suggest enhancements via our GitHub Issue Templates.
Security Concerns: Please report sensitive vulnerability issues securely following the rules in SECURITY.md.

📄 License

PrompTalk is licensed under the MIT License. See LICENSE for details.

Name		Name	Last commit message	Last commit date
Latest commit History 17 Commits
.github		.github
cli		cli
docs		docs
sdk		sdk
src		src
tests		tests
vscode		vscode
.env.example		.env.example
.gitignore		.gitignore
AGENTS.md		AGENTS.md
ARCHITECTURE.md		ARCHITECTURE.md
CHANGELOG.md		CHANGELOG.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
PLATFORM.md		PLATFORM.md
README.md		README.md
ROADMAP.md		ROADMAP.md
SECURITY.md		SECURITY.md
index.html		index.html
metadata.json		metadata.json
netlify.toml		netlify.toml
package-lock.json		package-lock.json
package.json		package.json
server.ts		server.ts
tsconfig.json		tsconfig.json
vite.config.ts		vite.config.ts

Folders and files

Latest commit

History

Repository files navigation