Home

Awesome

<div style="text-align:center" align="center">

SSB CONN

SSB plugin for establishing and managing peer connections.

</div>

CONN (Connections Over Numerous Networks) plugin replaces the old gossip plugin, covering all its use cases. CONN has these responsibilities:

Installation

Prerequisites:

Recommended:

Not required, but:

npm install --save ssb-conn

Add this plugin to ssb-server like this:

 var createSsbServer = require('ssb-server')
     .use(require('ssb-onion'))
     .use(require('ssb-unix-socket'))
     .use(require('ssb-no-auth'))
     .use(require('ssb-master'))
     .use(require('ssb-db2'))
+    .use(require('ssb-conn'))
     .use(require('ssb-replicate'))
     .use(require('ssb-friends'))
     // ...

Now you should be able to access the muxrpc APIs under ssb.conn and ssb.gossip, see next section.

Basic API

You can call any of these APIs in your local peer.

This uses multiserver addresses.

APITypeDescription
ssb.conn.connect(addr, data?, cb)asyncConnects to a peer known by its multiserver address addr, and stores additional optional data (as an object) during its connection lifespan.
ssb.conn.disconnect(addr, cb)asyncDisconnects a peer known by its multiserver address addr.
ssb.conn.peers()sourceA pull-stream that emits an array of all ConnHub entries whenever any connection updates (i.e. changes it state: connecting, disconnecting, connected, etc).
ssb.conn.remember(addr, data?)syncStores (in cold storage) connection information about a new peer, known by its multiserver address addr and additional optional data (as an object).
ssb.conn.forget(addr)syncRemoves (from cold storage) connection information about a peer known by its multiserver address addr.
ssb.conn.dbPeers()syncReturns an array of ConnDB entries known at the moment. Does not reactively update once the database is written to.
ssb.conn.stage(addr, data?)syncRegisters a suggested connection to a new peer, known by its multiserver address addr and additional optional data (as an object).
ssb.conn.unstage(addr)syncUnregisters a suggested connection the peer known by its multiserver address addr.
ssb.conn.stagedPeers()sourceA pull-stream that emits an array of all ConnStaging entries whenever any staging status updates (upon stage() or unstage()).
ssb.conn.start()syncTriggers the start of the connections scheduler in CONN.
ssb.conn.stop()syncStops the scheduler if it is currently active.

An "entry" is a (tuple) array of form:

[addr, data]

where:

<details> <summary>Full details on fields present in <code>data</code> (click here)</summary>

Fields marked ๐Ÿ”ท are important and often used, fields marked ๐Ÿ”น come from CONN, fields marked ๐Ÿ”ธ are ad-hoc and added by various other modules, and fields suffixed with ? are not always present:

๐Ÿ”ท key: string: the peer's public key / feedId

๐Ÿ”ท state?: 'connecting' | 'connected' | 'disconnecting': (only from peers()) the peer's current connection status

๐Ÿ”ท type?: string: what type of peer this is; it can be any string, but often is either 'lan', 'bt', 'pub', 'room', 'room-endpoint', 'dht'

๐Ÿ”น inferredType?: 'bt' | 'lan' | 'dht' | 'internet' | 'tunnel': (only from peers()) when there is no type field, e.g. when a new and unknown peer initiates a client connection with us (as a server), then ConnHub makes a guess what type it is

๐Ÿ”น birth?: number: Unix timestamp for when this peer was added to ConnDB

๐Ÿ”น stateChange?: number: Unix timestamp for the last time the field state was changed; this is stored in ConnDB

๐Ÿ”น hubBirth?: number: Unix timestamp for when this peer was added to ConnHub

๐Ÿ”น hubUpdated?: number: Unix timestamp for when this data object was last updated in ConnHub, which means the last time it was connected or attempted

๐Ÿ”น stagingBirth?: number: Unix timestamp for when this peer was added to ConnStaging

๐Ÿ”น stagingUpdated?: number: Unix timestamp for when this data object was last updated in ConnStaging

๐Ÿ”น autoconnect?: boolean: indicates whether this peer should be considered for automatic connection in the scheduler. By the default this field is considered true whenever it's undefined, and if you want opt-out of automatic connections for this peer (thus delegating it to a manual choice by the user), then set it to false.

๐Ÿ”น failure?: number: typically stored in ConnDB, this is the number of connection errors since the last successful connection

๐Ÿ”น duration?: object: typically stored in ConnDB, this is a statistics object to measure the duration of connection with this peer

๐Ÿ”น ping?: object: typically stored in ConnDB, this is statistics object of various ping health measurements

๐Ÿ”น pool?: 'db' | 'hub' | 'staging': this only appears in ConnQuery APIs, and indicates from which pool (ConnDB or ConnHub or ConnStaging) was this peer picked

๐Ÿ”ธ name?: string: a nickname for this peer, when there isn't an ssb-about name

๐Ÿ”ธ room?: string: (only if type = 'room-attendant') the public key of the room server where this peer is in

๐Ÿ”ธ onlineCount?: number: (only if type = 'room') the number of room endpoints currently connected to this room

</details>

Advanced API

CONN also provides more detailed APIs by giving you access to the internals, ConnDB, ConnHub, ConnStaging, ConnQuery. These are APIs that we discourage using, simply because in the vast majority of the cases, the basic API is enough (you might just need a few pull-stream operators on the basic APIs), but if you know what you're doing, don't feel afraid to use the advanced APIs!

APITypeDescription
ssb.conn.ping()duplexA duplex pull-stream for periodically pinging with peers, fully compatible with ssb.gossip.ping.
sbb.conn.db()syncReturns the instance of ConnDB currently in use. Read their docs to get access to more APIs.
ssb.conn.hub()syncReturns the instance of ConnHub currently in use. Read their docs to get access to more APIs.
ssb.conn.staging()syncReturns the instance of ConnStaging currently in use. Read their docs to get access to more APIs.
ssb.conn.query()syncReturns the instance of ConnQuery currently in use. Read their docs to get access to more APIs.

(Deprecated) Gossip API

The following gossip plugin APIs are available once you install CONN:

APIType
ssb.gossip.ping()duplex

If you want to use other legacy ssb.gossip.* APIs and preserve the same gossip behavior as before, use ssb-legacy-conn which uses parts of CONN and tries to mirrors the old gossip plugin as closely as possible, even its log messages.

Config

Some parameters in CONN can be configured by the user or by application code through the conventional ssb-config. The possible options are listed below:

{
  "conn": {
    /**
     * Whether the CONN scheduler should start automatically as soon as the
     * SSB app is initialized. Default is `true`.
     */
    "autostart": boolean,

    /**
     * Whether the CONN scheduler should look into the SSB database looking for
     * messages of type 'pub' and add them to CONN. Default is `true`.
     */
    "populatePubs": boolean,
  }
}

Recipes

<details> <summary>How can I get a pull stream of all currently connected peers? (click here)</summary> <p>

You can use ssb.conn.peers() to get a stream of "all peers currently being processed" and then use Array filter to pick only peers that are strictly connected, ignoring those that are connecting or disconnecting:

var connectedPeersStream = pull(
  ssb.conn.peers(),
  pull.map(entries =>
    entries.filter(([addr, data]) => data.state === 'connected')
  )
)

Then you can drain the stream to get an array of connected peers:

pull(
  connectedPeersStream,
  pull.drain(connectedPeers => {
    console.log(connectedPeers)
    // [
    //   ['net:192.168.1...', {key: '@Ql...', ...}],
    //   ['net:192.168.2...', {key: '@ye...', ...}]
    // ]
  })
)
<ul></ul> </p> </details> <details> <summary>How can I <em>immediately</em> get all currently connected peers? (click here)</summary> <p>

ssb-conn-query has APIs for that and others, e.g.

var arr = ssb.conn.query().peersConnected()

console.log(arr)
// [
//   ['net:192.168.1...', {key: '@Ql...', ...}],
//   ['net:192.168.2...', {key: '@ye...', ...}]
// ]

If the above doesn't work (for instance, conn.query() is not available in the CLI and other similar cases), you can use ssb.conn.peers() plus some pull-stream operators:

function getConnectedPeersNow(cb) {
  pull(
    ssb.conn.peers(),
    pull.map(entries =>
      entries.filter(([addr, data]) => data.state === 'connected')
    )
    pull.take(1), // This is important
    pull.drain(connectedPeers => cb(null, connectedPeers))
  )
}

getConnectedPeersNow(arr => console.log(arr))
<ul></ul> </p> </details>

Learn more

<details> <summary>How CONN works (click here)</summary> <p>

diagram.png

Under the hood, CONN is based on three "pools" of peers:

ConnDB contains metadata on stable servers and peers that have been successfully connectable. ConnHub is the central API that allows us to issue new connections and disconnections, as well as to track the currently active connections. ConnStaging is an in-memory ephemeral storage of new possible connections that the user might want to approve or disapprove.

Then, ConnQuery has access to those three pools, and provides utilities to query, filter, and sort connections across all those pools.

ConnScheduler is an opinionated (โš ๏ธ) plugin that utilizes ConnQuery to select peers to connect to, then schedules connections to happen via ConnHub, as well as schedules disconnections if necessary. Being opinionated, CONN provides an easy way of replacing the default scheduler with your own scheduler, see instructions below.

There is also a Gossip Compatibility plugin, implementing all the legacy APIs, so that other SSB plugins that call these APIs will continue to function as normal.

When you install the ssb-plugin, it will actually setup three plugins:

[conn, connScheduler, gossip]
</p> </details> <details> <summary>Opinions built into the default scheduler (click here)</summary> <p>

The default scheduler is roughly the same as the legacy ssb-gossip plugin, with some opinions removed and others added. The scheduler has two parts: discovery setup on startup, and periodic connections/disconnections.

Discovery setup:

Periodic connections/disconnections:

In none of the cases above shall we connect to a peer that we block. In addition to the above, the following actions happen automatically every (approximately) 1 second:

Database cleanups:

Upon starting the scheduler:

Other events:

<ul></ul> </p> </details> <details> <summary>How to build your own ConnScheduler (click here)</summary> <p>

To experiment with your own opinions for establishing connections, you can make your own ConnScheduler, which is just a typical SSB plugin. You can write in the traditional style (like other SSB plugins), or with OOP decorators. The example below uses OOP decorators.

Here is the basic shape of the scheduler:

module.exports = {
  name: 'connScheduler',
  version: '1.0.0',
  manifest: {
    start: 'sync',
    stop: 'stop',
  },
  init(ssb, config) {
    return {
      start() {
        // this is called when the scheduler should begin making connections

        // You have access to CONN core here:
        ssb.conn.stage('some multiserver address');
        ssb.conn.disconnect('another multiserver address');
        // ...
      },

      stop() {
        // this is called when the scheduler should cancel its jobs, if any
      }
    }
  }
}

Note that the name of the plugin must be exactly connScheduler (or connScheduler) and it must have the methods start() and stop(), because the CONN core will try to use your scheduler under those names. The rest of the contents of the ConnScheduler class are up to you, you can use private methods, etc.

When you're done building your scheduler, you can export it together with CONN core and the gossip compatibility plugin like this:

var CONN = require('ssb-conn/core')
var Gossip = require('ssb-conn/compat')
var ConnScheduler = require('./my-scheduler')

module.exports = [CONN, ConnScheduler, Gossip]

That array is a valid secret-stack plugin which you can .use() in ssb-server.

<ul></ul> </p> </details> <details> <summary>Why was the gossip plugin refactored? (click here)</summary> <p>

The legacy gossip plugin is one of the oldest parts of the SSB stack in Node.js, and it contained several old opinions. It wasn't designed with multiserver in mind, so it made a lot of assumptions that peers have host/port fields. Nowadays with Bluetooth and other unusual modes of connectivity, that assumption breaks down often.

The gossip plugin also did not have the concept of "staging", which is useful for ephemeral connections (LAN or Bluetooth) in spaces that may have many strangers. So the gossip plugin tended to connect as soon as possible to any peer discovered.

Also, since the gossip plugin was a monolith, it had all these concerns (cold persistence, in-memory tracking of current connections, ephemeral peers, scheduling, old and new style addresses) squashed into one file, making it hard and brittle to change the code.

The objectives with CONN were to:

<ul></ul> </p> </details>

License

MIT