🎥 HikAssist

Arabic-Egyptian RAG Assistant for Hikvision Technical Support

A Retrieval-Augmented Generation application designed to help users diagnose common Hikvision camera, NVR, DVR, Hik-Connect, storage, network, playback, and image-quality issues using trusted support documents.

🚀 Live Demo

👉 Click here to try the application

---

🚀 Live Demo

Try the deployed application directly on Streamlit Community Cloud:

Launch HikAssist

The application may take a few seconds to start after a period of inactivity because it loads the multilingual embedding model during startup.

---

📌 Project Overview

HikAssist is an Arabic conversational Retrieval-Augmented Generation system created to support users facing common Hikvision device problems.

The assistant accepts technical questions written in Arabic or Egyptian Arabic, retrieves the most relevant troubleshooting documents, and generates a grounded response based on the retrieved sources.

Example questions:

• الكاميرا أوفلاين ومش ظاهرة على الـ NVR
• الـ NVR مش بيسجل على الهارد
• الكاميرات مش ظاهرة على Hik-Connect
• الصورة بالليل مش واضحة
• نسيت كلمة مرور الجهاز

---

✨ Key Features

• Arabic and Egyptian Arabic query support
• Hybrid document retrieval
• BM25 keyword search
• Multilingual semantic embeddings
• Source-grounded answer generation
• Diagnostic follow-up questions
• Hikvision support-document citations
• Streamlit web interface
• OpenRouter LLM integration
• Offline retrieval fallback
• Public FAQ collection and translation pipeline
• Separate retrieval evaluation dataset

---

🧠 RAG Architecture

flowchart TD
    A[User Question] --> B[Arabic Text Preprocessing]
    B --> C[Query Representation]

    C --> D[BM25 Keyword Retrieval]
    C --> E[Multilingual Semantic Retrieval]

    D --> F[Hybrid Scoring]
    E --> F

    F --> G[Top Relevant Chunks]
    G --> H[Context Construction]
    H --> I[OpenRouter LLM]
    I --> J[Grounded Arabic Answer]
    J --> K[Streamlit Interface]

Loading

The hybrid retrieval score combines sparse keyword matching and dense semantic similarity:

Hybrid Score = 0.35 × BM25 + 0.65 × Semantic Similarity

---

🔄 Application Workflow

1. The user enters a technical question.
2. Arabic text is cleaned and normalized.
3. Egyptian Arabic technical terms are mapped to standardized forms.
4. BM25 retrieves documents based on keyword overlap.
5. Multilingual embeddings retrieve semantically similar documents.
6. The two retrieval scores are combined.
7. The highest-ranking documents are added to the prompt context.
8. The language model generates an answer using only the retrieved sources.
9. The response and relevant sources are displayed in Streamlit.

---

🛠️ Technology Stack

Component	Technology
Programming Language	Python
Web Interface	Streamlit
Sparse Retrieval	BM25
Dense Retrieval	Sentence Transformers
Embedding Model	paraphrase-multilingual-MiniLM-L12-v2
Vector Database	ChromaDB
LLM Provider	OpenRouter
Generation API	OpenAI-compatible API
Deployment	Streamlit Community Cloud
Data Format	Python and JSON

---

📁 Project Structure

simple-rag-lab/
│
├── data/
│   ├── raw/
│   └── processed/
│
├── 01_documents.py
├── 02_preprocessing.py
├── 03_chunking.py
├── 04_vector_representation.py
├── 05_create_chroma_store.py
├── 06_retrieve_context.py
├── 07_prompting.py
├── 08_scrape_hikvision.py
├── 09_translate_scraped_data.py
├── 10_import_scraped_documents.py
│
├── evaluation_questions.json
├── streamlit_app.py
├── requirements.txt
├── .env.example
├── .gitignore
└── README.md

Main Modules

File	Responsibility
01_documents.py	Loads and organizes the knowledge documents
02_preprocessing.py	Normalizes Arabic and Egyptian Arabic text
03_chunking.py	Divides documents into searchable chunks
04_vector_representation.py	Creates embeddings and hybrid retrieval scores
05_create_chroma_store.py	Creates the persistent Chroma vector store
06_retrieve_context.py	Retrieves and formats relevant context
07_prompting.py	Builds prompts and calls the language model
08_scrape_hikvision.py	Collects public Hikvision support content
09_translate_scraped_data.py	Translates and restructures collected content
10_import_scraped_documents.py	Imports processed records into the knowledge base
streamlit_app.py	Runs the Streamlit user interface

---

⚙️ Local Installation

1. Clone the repository

git clone https://github.com/abdelrhmanmouse/simple-rag-lab.git
cd simple-rag-lab

2. Create a virtual environment

Windows PowerShell

python -m venv .venv
.\.venv\Scripts\Activate.ps1

Linux or macOS

python3 -m venv .venv
source .venv/bin/activate

3. Install dependencies

python -m pip install --upgrade pip
pip install -r requirements.txt

4. Configure environment variables

Copy the environment example file:

Windows

Copy-Item .env.example .env

Linux or macOS

cp .env.example .env

Open .env and add your OpenRouter configuration:

OPENROUTER_API_KEY=your_openrouter_api_key
OPENROUTER_MODEL=openai/gpt-4o-mini
OPENROUTER_TRANSLATION_MODEL=openrouter/free

Never upload a real API key to GitHub.

5. Create the Chroma vector store

python 05_create_chroma_store.py

6. Run the application

streamlit run streamlit_app.py

Open the local address shown in the terminal:

http://localhost:8501

---

☁️ Streamlit Deployment

The application is deployed using Streamlit Community Cloud.

Deployment configuration:

Repository: abdelrhmanmouse/simple-rag-lab
Branch: main
Main file: streamlit_app.py
Python version: 3.11

Required Streamlit secrets:

OPENROUTER_API_KEY = "your_openrouter_api_key"
OPENROUTER_MODEL = "openai/gpt-4o-mini"
OPENROUTER_TRANSLATION_MODEL = "openrouter/free"

Do not store API keys directly inside the repository.

---

📚 Knowledge Base

The knowledge base contains sourced troubleshooting information covering areas such as:

• Cameras
• NVR and DVR devices
• Hik-Connect
• Network connectivity
• PoE problems
• Playback and recording
• Storage and hard-drive issues
• Password and device activation
• Image quality
• Night vision
• Remote access

Each document may contain:

• Document ID
• Title
• Category
• Product type
• Troubleshooting content
• Diagnostic questions
• Verification information
• Official source URL

---

📊 Evaluation

The repository includes:

evaluation_questions.json

This file contains Egyptian Arabic evaluation questions and the expected source document for each question.

It can be used to calculate retrieval metrics such as:

• Hit Rate at K
• Recall at K
• Mean Reciprocal Rank
• Top-1 Accuracy
• Top-3 Accuracy

---

🔐 Security Notes

• Never commit .env to GitHub.
• Never expose the OpenRouter API key inside Python files.
• Store deployment keys in Streamlit Secrets.
• Use an API key with a spending limit for public applications.
• Do not request passwords, verification codes, or private device credentials from users.
• Sensitive operations such as reset, formatting, and firmware updates should include clear warnings.

---

⚠️ Limitations

• The application is intended for educational and technical-assistance purposes.
• Retrieval quality depends on the available knowledge documents.
• The generated response should not replace official Hikvision support.
• The application may require a short startup period after being inactive.
• Complex hardware or account-related problems may require escalation to an authorized technician.

---

🔮 Future Improvements

• Add a cross-encoder reranker
• Add full conversational memory
• Add automatic retrieval evaluation
• Add response-groundedness evaluation
• Improve metadata filtering
• Expand the knowledge base
• Add multilingual support
• Improve document chunking
• Add feedback collection
• Add administrator analytics
• Cache embeddings for faster startup
• Connect retrieval directly to the persistent vector database

---

🤝 Contributing

Contributions are welcome.

To contribute:

1. Fork the repository.
2. Create a new branch.
3. Make your changes.
4. Commit your changes.
5. Push the branch.
6. Open a Pull Request.

---

👨‍💻 Author

Developed as an educational Retrieval-Augmented Generation project for Arabic technical support.

GitHub: abdelrhmanmouse

---

📄 Disclaimer

HikAssist is an independent educational project and is not an official Hikvision product.

Hikvision and related product names are trademarks of their respective owners.

---

⭐ Support the Project

If you find this project useful, consider giving the repository a star.

Open the Live Application