Blockchain Databases - Interoperability Framework

This project represents the practical implementation of the Master's thesis titled "Blockchain databases".

Its primary goal is to address the interoperability challenge between various distributed blockchain-based databases and traditional Data Lake architectures. The system enables secure, verifiable, and cost-effective data exchange within ELT (Extract-Load-Transform) processes.

🎯 Core Concepts

The project implements a hybrid architecture that ensures:

• Single Source of Truth: Maintenance of cryptographic hashes and metadata on a main blockchain (Hub Chain).
• Off-chain Storage: Storage of full, high-volume data in external repositories (Data Lake) to avoid prohibitive on-chain storage costs.
• Interoperability: Utilization of the LayerZero protocol for communication between disparate blockchain networks and databases.

🏗 Architecture

The system connects four heterogeneous data sources to a central Data Lake using EVM-compatible networks.

Supported Databases (Sources):

The project integrates two types of blockchain databases:

1. Relational (SQL-based):
   • Tableland: SQL tables managed by EVM smart contracts.
   • Space and Time: SQL data warehouse with cryptographic proofs (Proof-of-SQL).
2. Document-oriented:
   • Arweave: Permanent, decentralized storage (permaweb).
   • BigChainDB: Decentralized database with blockchain characteristics.

Synchronization Models:

Two data flow mechanisms have been implemented:

• PUSH Model: The source actively sends encrypted data packets to the Hub immediately after recording.
• PULL Model: The source only "anchors" a data reference on-chain. The Hub (Data Lake) retrieves the data on-demand by querying the source via LayerZero.

🛠 Technology Stack

• Programming Languages:
  • Solidity: Smart contracts (LayerZero Adapters, Hub, Write Logic).
  • TypeScript: Off-chain scripts, WebSocket services, ELT process handling, encryption.
• Communication Protocol: LayerZero (v2) – for cross-chain messaging.
• Security: AES-GCM (Galois/Counter Mode) encryption ensuring confidentiality and integrity of transmitted data.
• Frameworks: Hardhat (contract compilation and deployment).

📂 Repository Structure

• /contracts: Smart contract source code (Solidity).
  • MyOApp.sol, Reader.sol: LayerZero application logic.
  • interfaces/: Contract interfaces.
• /deploy: Deployment scripts (Hardhat Deploy).
• /src: Off-chain logic (TypeScript).
  • encrypt.ts / decrypt.ts: Cryptographic modules.
  • relayer.ts: Message transmission handling.
• /tasks: Hardhat helper tasks (e.g., sending test messages).
• /test: Unit and integration tests.