diff --git a/README.md b/README.md new file mode 100644 index 0000000..68986bc --- /dev/null +++ b/README.md @@ -0,0 +1,62 @@ +# go-smart-record +go-smart-record is the go implementation of Smart Records. Smart Records (SRs) provide +a *public blackboard for protocols*. + +- Detailed docs are available [here]() and throughout the code. + +## Overview +We currently don't have a standardized, shared, public +medium for: +- writing and reading +- by multiple participants +- talking multiple protocols +- scattered in multiple locations. + +Traditionally, DHTs have been used for this purpose. With Smart Record we generalize DHT's +key/value put/get as a separate protocol that can be leveraged by any other protocol +(including DHT protocols). +- UPDATE/GET interface of SRs are used to interact with records stored in a peer, +delegating the FIND operations of records to the DHT (or other available protocols), +decoupling the storage of records from transport protocols. +- Records become portable *data state machines*. They can be sent and updated using pubsub, +aggregated with other versions of the records even if a peer doesn't fully understand it, etc. + +> The layman description of smart records: *"they are DHT values which become publicly updatable JSON/IPLD documents by any peer*. + +SRs are a mixture between a CRDT and a smart contract. A record (for a key) is a replicated state machine holding generic data. +It supports reading, writing, merging and "smart services" (through smart tags included in the SR data model). + +## Model +- Each peer (identified by its public key) writes to a peer-specific documents. +- Peers can overwrite their own documents. +- Every document node has a TTL specified (and eventually paid for) by the writing peer. +- Users of SR can get the full record and process the information stored in the different user-spaces. + +## Architecture +The SR system has the following architecture: +- Syntactic representation [(xr)](./xr): Data model used by protocol and application developers +to interact with smart records. In their current implementation smart-records can +be transformed into the IPLD data model, and serialized/desearialized seamlessly for +transmission or any other purpose. +- Semantic representation [(ir)](./ir): Intemediate representation used by the SR VM. +Syntactic nodes are assembled into semantic nodes. In the assembly process, tags are parsed +and certain nodes may be transformed into smart nodes and trigger additional (i.e. "smart") +operations in the VM. + - The `BaseGrammar` currently supported by default for the semantic representation of SR can be found [here](./ir/base/base.go). +- VM [(vm)](./vm): The VM is responsible for storing and updating the SR stored in a peer. It exposes the SR interface to the "outside world" and triggers smart-tag operations when appropiate. The "outside world" use syntctic nodes to intercat with the VM interface that the VM assembles and stores in its datastore in its semantic form. +- Libp2p SR request/response protocol [(protocol)](./protocol) to interact with other peers SR. It includes a server implementation that instantiates a SR VM and exposes the SR interface to other peers in the network; and a client implementation that can be leveraged by applications and protocols to make requests to SR servers. + +![](./doc/sr-architecture.png) + +## Use cases + +Some examples of things you can do with SR: +- Deploy new applications without upgrading the whole network. +- Design protocols that can interact with other protocols. +- Facilitate cryptographic protocols that require a "trusted" party + - Fair exchange +- Unlock application development on the DHT to the public + - Private group chat, custom routing, decentralized limit-order marketbook, etc. + - New app types: Interaction between trustless parties, using a public jury. + +If you want to see SR in action, several examples of SR are provided in [the example directory](./examples) diff --git a/doc.go b/doc.go new file mode 100644 index 0000000..4ab9f92 --- /dev/null +++ b/doc.go @@ -0,0 +1,71 @@ +/* +Package go-smart-record includes the go implementation of Smart Records. Smart Records (SRs) provide +a public blackboard for protocols. + +Overview + +We currently don't have a standardized, shared, public +medium for: + - writing and reading + - by multiple participants + - talking multiple protocols + - scattered in multiple locations. + +Traditionally, DHTs have been used for this purpose. With Smart Record we generalize DHT's +key/value put/get as a separate protocol that can be leveraged by any other protocol +(including DHT protocols). + - UPDATE/GET interface of SRs are used to interact with records stored in a peer, + delegating the FIND operations of records to the DHT (or other available protocols), + decoupling the storage of records from transport protocols. + - Records become portable *data state machines*. They can be sent and updated using pubsub, + aggregated with other versions of the records even if a peer doesn't fully understand it, etc. + + +SRs are a mixture between a CRDT and a smart contract. A record (for a key) is a replicated state machine holding generic data. +It supports reading, writing, merging and "smart services" (through smart tags included in the SR data model). + - The layman description of smart records: *"they are DHT values which become publicly updatable JSON/IPLD documents by any peer*. + +Model + +SRs work as follows: + - Each peer (identified by its public key) writes to a peer-specific documents. + - Peers can overwrite their own documents. + - Every document node has a TTL specified (and eventually paid for) by the writing peer. + - Users of SR can get the full record and process the information stored in the different user-spaces. + +Architecture + +The SR system has the following architecture: + - Syntactic representation (xr): Data model used by protocol and application developers + to interact with smart records. In their current implementation smart-records can + be transformed into the IPLD data model, and serialized/desearialized seamlessly for + transmission or any other purpose. + + - Semantic representation (ir): Intemediate representation used by the SR VM. + Syntactic nodes are assembled into semantic nodes. In the assembly process, tags are parsed + and certain nodes may be transformed into smart nodes and trigger additional (i.e. "smart") + operations in the VM. + + - VM (vm): The VM is responsible for storing and updating the SR stored in a peer. It exposes the + SR interface to the "outside world" and triggers smart-tag operations when appropiate. + The "outside world" use syntctic nodes to intercat with the VM interface that the VM assembles + and stores in its datastore in its semantic form. + + - Libp2p SR request/response protocol (protocol) to interact with other peers SR. + It includes a server implementation that instantiates a SR VM and exposes the SR interface + through the network to other peers, and a client implementation that can be leveraged by + applications and protocols to make requests to SR servers. + + +Use cases + +Some examples of things you can do with SR: + - Deploy new applications without upgrading the whole network. + - Design protocols that can interact with other protocols. + - Facilitate cryptographic protocols that require a "trusted" party + - Fair exchange + - Unlock application development on the DHT to the public + - Private group chat, custom routing, decentralized limit-order marketbook, etc. + - New app types: Interaction between trustless parties, using a public jury. +*/ +package main diff --git a/doc/sr-architecture.png b/doc/sr-architecture.png new file mode 100644 index 0000000..048d94c Binary files /dev/null and b/doc/sr-architecture.png differ diff --git a/ir/doc.go b/ir/doc.go index 6a5ed85..b0e2d82 100644 --- a/ir/doc.go +++ b/ir/doc.go @@ -1,4 +1,4 @@ -// Package ir defines the Intermediate Representation (informally, in-memory representation) of smart records. +// Package ir defines the Semantic Representation of smart records. package ir /* @@ -18,6 +18,7 @@ The vocabulary consists of two types of nodes: syntactic and smart. Smart nodes, by their Go type: Cid Multiaddress + Reachable Peer Record Sign, Signed @@ -25,13 +26,13 @@ The vocabulary consists of two types of nodes: syntactic and smart. We use the following nomenclature: - "Syntactic IR", or "syntactic documents", refers to documents comprising only syntactic nodes. - "Semantic IR", or "semantic documents", refers to documents comprising syntactic and smart nodes. + "Syntactic IR (XR)", or "syntactic documents", refers to documents comprising only syntactic nodes. + "Semantic IR (IR)", or "semantic documents", refers to documents comprising syntactic and smart nodes. Users generally manipulate semantic documents (or just "documents", for short), consisting of both syntactic and smart nodes. Syntactic nodes represent generic structured data types (and support generic merge logic). -Smart nodes represent higher concepts (with custom merge logics) that have a syntactic representation. +Smart nodes represent higher concepts (with custom update logics and smart behavior) that have a syntactic representation. SERIALIZATION @@ -54,10 +55,10 @@ We use JSON as a running example. Serialization to JSON is also provided by this library out-of-the-box. Serialization: - JSON <--(marshal)-- Syntactic IR <--(disassemble)-- Semantic IR + JSON <--(marshal)-- IPLD <-- Syntactic IR <--(disassemble)-- Semantic IR Deserialization: - JSON --(unmarshal)--> Syntactic IR --(assemble)--> Semantic IR + JSON --(unmarshal)--> IPLD --> Syntactic IR --(assemble)--> Semantic IR */ diff --git a/protocol/doc.go b/protocol/doc.go new file mode 100644 index 0000000..e4ff429 --- /dev/null +++ b/protocol/doc.go @@ -0,0 +1,6 @@ +// Package protocol implements a libp2p request-response protocol to interact with other peers SR. +// It includes a server implementation that instantiates a SR VM and exposes the SR interface +// to other peers in the network; +// and a client implementation that can be leveraged by applications +// and protocols to make requests to SR servers. +package protocol diff --git a/vm/doc.go b/vm/doc.go new file mode 100644 index 0000000..4e5dab7 --- /dev/null +++ b/vm/doc.go @@ -0,0 +1,7 @@ +/* +Package vm implements a SR VM responsible for storing and updating the SR stored in a peer. +The VM exposes the SR interface to the "outside world" and triggers smart-tag operations when appropiate. +The "outside world" use syntctic nodes to intercat with the VM interface that the VM assembles and stores in its datastore in its semantic form. +- +*/ +package vm diff --git a/xr/doc.go b/xr/doc.go new file mode 100644 index 0000000..8a3230e --- /dev/null +++ b/xr/doc.go @@ -0,0 +1,8 @@ +/* +Package xr implements the Syntactic Representation of smart records. +This is the data model used by protocol and application developers +to interact with smart records. In their current implementation smart-records can +be transformed into the IPLD data model, and be serialized/desearialized seamlessly for +transmission or any other purpose. +*/ +package xr