hubble

Hubble

Hubble is a Typescript implementation of a Farcaster Hub. Hubs store messages uploaded by users and replicate them to other Hubs according to the protocol specification.

Getting Started

Starting Hubble

You'll need to install dependencies and have the RPC URL of an Ethereum Goerli node, which you can get Alchemy or Infura. Then:

1. Navigate to this folder (/apps/hubble) from the root
2. Run yarn identity create to generate an identity key pair.

Connecting to Devnet

Hubble can be started in devnet mode which is used for local testing. Identity state will be read from on-chain contracts but it will not connect to peers. To connect to devnet:

1. Run yarn start -e <eth-rpc-url> -n 3 to boot up the Hub, where eth-rpc-url points to the Goerli node's RPC

Connecting to Testnet

Hubble can be started in testnet mode where it peers with other hubs. The Farcaster team runs a set of hub that publish messages every 10 seconds to make testing easy. To connect to testnet:

1. Run yarn start -e <eth-rpc-url> -b /dns/testnet1.farcaster.xyz/tcp/2282 -n 2
2. Pass --reset-db flag on first run, if you've previously connected to devnet or mainnet

Connecting to Mainnet

Hubble mainnet is not yet available.

Deploying Hubble in the cloud

Hubble can be run on cloud servers with 2 vCPUs, 8 GiB RAM and 15 GiB of SSD storage. Follow these guides to get Hubble running on your preferred provider:

• AWS EC2
• GCP (planned)
• Azure (planned)
• Digital Ocean

Interacting with Hubble

Hubble exposes gRPC API's and uses Protobufs for data serialization. SDK's are available to interact with Hubble API's:

• Javascript / Typescript
• Golang (planned)

Architecture

A Hub is a single-process daemon that receives data from clients, other hubs and farcaster contracts. It has three main components:

• P2P Engine - establishes a gossipsub network to exchange messages with hubs.
• Sync Engine - handles edge cases when gossip fails to deliver messages.
• Storage Engine - checks message validity, persists them to disk and emits events.

flowchart LR
subgraph Hubble

    subgraph Networking
        P2PEngine(P2PEngine)
        SyncEngine(SyncEngine)
    end

    subgraph Storage
        StorageEngine(Storage Engine)
    end

end
Node[ETH Node] -.-> |RPC| Hubble
Clients[FC Clients] & Clients & Clients -.-> |RPC| Hubble
Hubble <-.-> |RPC + Gossip|Hubs[Hubs] & Hubs & Hubs
Storage <--> Networking

Loading

Storage Engine

Messages received by Hubble are forwarded to the Storage engine which forwards them to the appropriate CRDT Set. Once validated by the CRDT Set, messages are persisted to RocksDB and events are emitted to listeners.

flowchart LR
subgraph Hubble
    subgraph Storage
        StorageEngine(Storage Engine) --> Sets
        Sets(CRDT Sets) <--> DB[(Rocks<br/>DB)]
    end
end

Loading

CRDT sets are implemented to meet the specification in the Farcaster protocol. The engine also tracks state of the Farcaster contracts, which are necessary for validating the Signer CRDT Set.

P2P Engine

Hubble connects to other peers over a GossipSub network established using LibP2P. Messages merged into the Storage Engine are immediately gossiped to all of is peers.

flowchart LR
subgraph Hubble

    subgraph Networking
        P2PEngine(P2PEngine) --> LibP2P(LibP2P)
    end
end

Loading

Hubble will only peer with trusted peers and employs a simple network topology during beta. It peers only with known instances which must be configured at startup. In later releases, the network topology will be modified to operate closer to a trustless mesh.

Sync Engine

Hubble periodically performs a diff sync with other peers to discover messages that may have been dropped during gossip. This is performed using gRPC APIs exposed by each Hub instance.

flowchart LR
subgraph Hubble
    subgraph Networking
        SyncEngine(SyncEngine) --> RPC(RPC Client)
    end
end

Loading