🚀 FlashDB

An Advanced, Highly Concurrent, In-Memory Key-Value Store in C++17

FlashDB is a high-performance, multithreaded Key-Value database engineered from the ground up in modern C++. By implementing the Redis Serialization Protocol (RESP), it serves as a drop-in replacement for basic Redis workloads.

Unlike naive implementations that rely on a single global mutex, FlashDB is architected to solve massive concurrent throughput challenges. It utilizes a Custom Thread Pool, Lock-Striping (Sharding), and Strict Resource Hierarchies to eliminate CPU bottlenecking and prevent deadlocks at the system level.

---

Repository Structure

├── include/                # Public headers
│   ├── RedisCommandHandler.h
│   ├── RedisDatabase.h
│   └── RedisServer.h
├── src/                    # Implementation files
│   ├── RedisCommandHandler.cpp
│   ├── RedisDatabase.cpp
│   ├── RedisServer.cpp
│   └── main.cpp            # Entry point
├── CMakeLists.txt
├── Makefile                # Build rules
├── README.md               # This documentation
└── benchmark   
└── tests            

---

🧠 Architectural Overview

FlashDB is decoupled into three distinct layers to maximize maintainability and scalability, adhering strictly to SOLID Principles.

graph TD;
    Client1[Client / redis-cli] -->|TCP / RESP| Server[Network Layer: RedisServer];
    Client2[Client / redis-cli] -->|TCP / RESP| Server;
    Client3[Client / redis-cli] -->|TCP / RESP| Server;

    subgraph FlashDB Process
        Server -->|Pushes Socket| TaskQueue[(Thread-Safe Task Queue)];
        TaskQueue -->|Wakes Worker| ThreadPool[Thread Pool 8-16 Workers];
        ThreadPool -->|Raw Bytes| CmdHandler[Protocol Layer: RedisCommandHandler];
        CmdHandler -->|Parsed Command| DB[Storage Layer: RedisDatabase];

        subgraph Lock-Striped Storage
            DB --> Shard0[Shard 0: shared_mutex];
            DB --> Shard1[Shard 1: shared_mutex];
            DB --> ShardN[Shard N: shared_mutex];
        end

        Persistence[Background Persistence Thread] -.->|lockAllShared| DB;
        Persistence -->|Binary Dump| Disk[(dump.rdb)];
    end

Loading

1. Network Layer (Thread Pool Pattern)

• Pre-allocates worker threads using std::thread::hardware_concurrency().
• Workers sleep efficiently on a std::condition_variable.
• Incoming TCP sockets are pushed to a synchronized std::queue.
• Eliminates thread-creation latency and improves scalability under C10k workloads.

2. Protocol Layer (Command Dispatcher Pattern)

The RedisCommandHandler isolates networking from storage. It parses RESP byte streams (e.g. *3\r\n$3\r\nSET\r\n...) and dispatches them to the database layer.

3. Storage Layer (Concurrency Engine)

• In-memory Key-Value store
• Supports Strings, Lists, and Hashes
• Singleton Pattern ensures a consistent shared database instance

---

⚡ Advanced Concurrency Mechanisms

Lock-Striping (Sharding)

• Uses 16 independent shards
• getShardIndex(key) uses std::hash<std::string>
• Threads only contend when accessing the same shard

Readers-Writer Locking

• std::shared_lock → concurrent readers (GET)
• std::unique_lock → exclusive writers (SET, DEL)

Deadlock Avoidance

Global operations acquire locks in ascending shard order using a strict Resource Hierarchy, preventing circular wait conditions.

---

💾 Persistence

• Background snapshot thread
• Saves .rdb file every 5 minutes
• Uses shared locks (lockAllShared()) to avoid stop-the-world pauses while allowing concurrent reads

---

🛠️ Build & Installation

Prerequisites

• CMake ≥ 3.10
• C++17 Compiler (GCC / Clang / MSVC)
• POSIX OS (Linux/macOS)

Compilation

git clone https://github.com/Shubham-Kankotiya/FlashDB.git
cd FlashDB

mkdir build
cd build

cmake ..
make

./flashdb_server 6379

---

💻 Usage

$ redis-cli -p 6379

127.0.0.1:6379> PING
+PONG

127.0.0.1:6379> SET framework "FlashDB"
+OK

127.0.0.1:6379> GET framework
"FlashDB"

127.0.0.1:6379> LPUSH task_queue "write_docs" "compile_code"
:2

127.0.0.1:6379> HSET user:100 name "Shubham"
:1

---

Supported Commands

Common

• PING: PING → PONG
• ECHO: ECHO <msg> → <msg>
• FLUSHALL: FLUSHALL → clear all data

Key/Value

• SET: SET <key> <value> → store string
• GET: GET <key> → retrieve string or nil
• KEYS: KEYS * → list all keys
• TYPE: TYPE <key> → string/list/hash/none
• DEL/UNLINK: DEL <key> → delete key
• EXPIRE: EXPIRE <key> <seconds> → set TTL
• RENAME: RENAME <old> <new> → rename key

Lists

• LGET: LGET <key> → all elements
• LLEN: LLEN <key> → length
• LPUSH/RPUSH: LPUSH <key> <v1> [v2 ...] / RPUSH → push multiple
• LPOP/RPOP: LPOP <key> / RPOP <key> → pop one
• LREM: LREM <key> <count> <value> → remove occurrences
• LINDEX: LINDEX <key> <index> → get element
• LSET: LSET <key> <index> <value> → set element

Hashes

• HSET: HSET <key> <field> <value>
• HGET: HGET <key> <field>
• HEXISTS: HEXISTS <key> <field>
• HDEL: HDEL <key> <field>
• HLEN: HLEN <key> → field count
• HKEYS: HKEYS <key> → all fields
• HVALS: HVALS <key> → all values
• HGETALL: HGETALL <key> → field/value pairs
• HMSET: HMSET <key> <f1> <v1> [f2 v2 ...]

---

🧪 Testing & Benchmarking

./build/flashdb_tests

./build/flashdb_benchmark

Performance scales with available CPU cores.

---

🔮 Future Roadmap

• Asynchronous I/O (epoll / kqueue)
• Write-Ahead Logging (WAL)
• Lock-Free Data Structures (Hazard Pointers + CAS)