Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,11 @@
dist
node_modules

.eliza
.elizadb
.env
.elizadb*
.env
.cursor
coverage
cypress/screenshots
docs
231 changes: 231 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,25 @@ The plugin provides these actions that your agent can use:

- "Process the document at /path/to/file.pdf"
- "Remember this: The sky is blue"
- Send files as attachments in your message - they'll be automatically processed!

2. **SEARCH_KNOWLEDGE** - Search the knowledge base
- "Search your knowledge for quantum computing"

## 📎 Attachment Processing

The knowledge plugin now intelligently handles attachments:

- **Direct file attachments** - Just attach files to your message
- **URL attachments** - Share links to documents and they'll be downloaded and processed
- **Multiple attachments** - Process many files at once
- **Smart detection** - The agent automatically detects when you want to save attachments

Examples:
- "Save these documents" + [attach PDFs]
- "Add this to your knowledge" + [attach text file]
- "Learn from this website" + [URL attachment]

## 🌐 Web Interface

The plugin includes a web interface for managing documents! Access it at:
Expand All @@ -78,6 +93,14 @@ The plugin includes a web interface for managing documents! Access it at:
http://localhost:3000/api/agents/[your-agent-id]/plugins/knowledge/display
```

Features:
- 📋 List all documents with metadata
- 🔍 Search through your knowledge base
- 📊 Visual graph of document relationships
- ⬆️ Upload new documents
- 🗑️ Delete existing documents
- 🔄 Update document metadata

---

## ⚠️ Advanced Configuration (Developers Only)
Expand Down Expand Up @@ -157,6 +180,8 @@ MAX_OUTPUT_TOKENS=4096
- `GET /api/agents/{agentId}/plugins/knowledge/documents` - List documents
- `GET /api/agents/{agentId}/plugins/knowledge/documents/{id}` - Get specific document
- `DELETE /api/agents/{agentId}/plugins/knowledge/documents/{id}` - Delete document
- `PUT /api/agents/{agentId}/plugins/knowledge/documents/{id}` - Update document metadata
- `POST /api/agents/{agentId}/plugins/knowledge/search` - Search knowledge base
- `GET /api/agents/{agentId}/plugins/knowledge/display` - Web interface

### Programmatic Usage
Expand All @@ -174,10 +199,216 @@ const result = await knowledgeService.addKnowledge({
roomId: 'room-id',
entityId: 'entity-id',
});

// Update document metadata
await runtime.updateMemory({
id: documentId,
metadata: {
type: MemoryType.DOCUMENT,
tags: ['updated', 'important'],
source: 'manual-update',
},
});

// Delete a document
await knowledgeService.deleteMemory(documentId);
```

</details>

## 📝 License

See the ElizaOS license for details.

### Advanced Features

The knowledge plugin now includes several advanced features for enterprise-grade knowledge management:

#### 🔍 Advanced Search

Search with filters, sorting, and pagination:

```typescript
const results = await knowledgeService.advancedSearch({
query: 'machine learning',
filters: {
contentType: ['application/pdf', 'text/markdown'],
tags: ['ai', 'research'],
dateRange: {
start: new Date('2024-01-01'),
end: new Date('2024-12-31'),
},
minSimilarity: 0.7,
},
sort: {
field: 'similarity', // or 'createdAt', 'updatedAt', 'title'
order: 'desc',
},
limit: 20,
offset: 0,
includeMetadata: true,
});
```

Natural language search examples:
- "Search for pdf documents about AI from last week"
- "Find recent markdown files sorted by relevant"
- "Look for documents with blockchain tags from today"

#### 📦 Batch Operations

Process multiple documents efficiently:

```typescript
const result = await knowledgeService.batchOperation({
operation: 'add', // or 'update', 'delete'
items: [
{ data: { content: 'Doc 1', contentType: 'text/plain', ... } },
{ data: { content: 'Doc 2', contentType: 'text/plain', ... } },
// ... more items
],
});

console.log(`Processed: ${result.successful} successful, ${result.failed} failed`);
```

#### 📊 Analytics & Insights

Get comprehensive analytics about your knowledge base:

```typescript
const analytics = await knowledgeService.getAnalytics();

// Returns:
// {
// totalDocuments: 150,
// totalFragments: 450,
// storageSize: 5242880, // bytes
// contentTypes: {
// 'application/pdf': 80,
// 'text/plain': 50,
// 'text/markdown': 20,
// },
// queryStats: {
// totalQueries: 1000,
// averageResponseTime: 250, // ms
// topQueries: [
// { query: 'AI research', count: 50 },
// { query: 'blockchain', count: 30 },
// ],
// },
// usageByDate: [...],
// }
```

#### 📤 Export & Import

Export your knowledge base in multiple formats:

```typescript
// Export to JSON
const jsonExport = await knowledgeService.exportKnowledge({
format: 'json',
includeMetadata: true,
documentIds: ['id1', 'id2'], // optional filter
dateRange: { start: new Date('2024-01-01') }, // optional filter
});

// Export to CSV
const csvExport = await knowledgeService.exportKnowledge({
format: 'csv',
});

// Export to Markdown
const markdownExport = await knowledgeService.exportKnowledge({
format: 'markdown',
includeMetadata: false,
});
```

Import knowledge from various formats:

```typescript
// Import from JSON
const importResult = await knowledgeService.importKnowledge(jsonData, {
format: 'json',
validateBeforeImport: true,
overwriteExisting: false,
});

// Import from CSV
const csvResult = await knowledgeService.importKnowledge(csvData, {
format: 'csv',
batchSize: 100,
});
```

#### 🎯 Action Chaining

The SEARCH_KNOWLEDGE action now returns structured data that can be used by other actions:

```typescript
// SEARCH_KNOWLEDGE returns:
{
data: {
query: 'machine learning',
results: [...], // KnowledgeItem[]
count: 5,
},
text: 'Found 5 results...',
}

// This data can be consumed by other actions like:
// - ANALYZE_KNOWLEDGE: Analyze search results
// - SUMMARIZE_KNOWLEDGE: Create summaries
// - FILTER_KNOWLEDGE: Further filter results
```

#### ⚙️ Configuration Options

New configuration options for advanced features:

```env
# Search Configuration
SEARCH_MATCH_THRESHOLD=0.7 # Minimum similarity score (0-1)
SEARCH_RESULT_COUNT=20 # Default number of results

# Feature Flags
ENABLE_VERSIONING=true # Track document versions
ENABLE_ANALYTICS=true # Enable analytics tracking

# Performance
BATCH_PROCESSING_SIZE=5 # Items processed in parallel
```

### Available Actions

The plugin provides these enhanced actions:

1. **PROCESS_KNOWLEDGE** - Add documents, text, URLs, or attachments
2. **SEARCH_KNOWLEDGE** - Basic knowledge search with action chaining support
3. **ADVANCED_KNOWLEDGE_SEARCH** - Advanced search with filters and sorting
4. **KNOWLEDGE_ANALYTICS** - Get analytics and insights
5. **EXPORT_KNOWLEDGE** - Export knowledge base to various formats

</details>

## 🧪 Testing

The plugin includes comprehensive test coverage:

```bash
# Run all tests
npm test

# Run unit tests
npm run test:unit

# Run E2E tests (including advanced features)
npm test

# Run Cypress UI tests
npm run test:cypress:open
```

## 🤝 Contributing
Loading