Willow Sideloading Protocol

Status: Proposal

The WGPS presents a way for two peers with an established connection to efficiently exchange data. But running the necessary infrastructure to establish such connections (e.g. a STUN server, distributed hash table, or relay servers) is a non-trivial task, limiting who can operate them.

The Willow Sideloading Protocol presents a way for peers to transmit data to each other asynchronously and via completely user-improvised channels.

Instead of statefully coordinating which data to exchange, the Sideloading Protocol uses drops, arbitrary sets of Entries and Payloads compiled into a single bytestring. A drop can then be treated as a kind of static store which can be joined with any other store as per the Willow Data Model.

Drops are then shared via the informal ad-hoc infrastructure we refer to as the Sidenet:In contrast with sneakernets which only use physically transported storage devices, the sidenet also includes the internet and other established networks.

• USB keys
• SD cards
• CDs
• Email attachments
• Instant messages
• Torrents
• Or whatever means users have at hand.

Drops will often undoubtedly contain stale data that users are already aware of or no longer need. But what the sideloading protocol gives up in efficiency it more than makes up for in simplicity and flexibility. It is simple to implement, and works with the infrastructure users already have.

Finally, given that this protocol cannot interactively authorise users (e.g. via private area intersection), drops are always fully encrypted.

Parameters

In order to use the sideloading protocol, one must first specify a full suite of instantiations of the parameters of the core Willow data model. In addition to this, the sideloading protocol requires the following:

• A total order on SubspaceId with least element default_subspace_id, in which for every SubspaceId subspace there exists a successor t such that s is less than t and no other SubspaceId is greater than s and less than t.
• An encoding function for NamespaceId
• An encoding function for SubspaceId
• An encoding function for PayloadDigest
• An encoding function encode_authorisation_token for AuthorisationToken
• A NamespaceId default_namespace_id
• A SubspaceId default_subspace_id
• A PayloadDigest default_payload_digest
• And a function which encrypts the final compiled encoding, encrypt.

Protocol

Let entries be an arbitrarily selected set of PossiblyAuthorisedEntries from a namespace namespace, transformed into a sequence by ordering according to the following criteria:

This ordering takes maximum advantage of our relative encodings.A PossiblyAuthorisedEntry p1 is prior to an PossiblyAuthorisedEntry p2 if

• using the total order on SubspaceIdp1.0.subspace_id < p2.0.subspace_id, or
• using lexicographic orderingp1.0.path < p2.0.path, or
• the newer relation on entries.

Let initial_entry be the result of default_entry(default_namespace_id, default_subspace_id, default_payload_digest).

Let contents be the concatenation of

• The size of entries, encoded as an unsigned, big-endian 64-bit integer, followed by
• Each PossiblyAuthorisedEntry of entries mapped to the concatenation of (encoded_entry, encoded_token, payload) where possibly_authorised_entry is the PossiblyAuthorisedEntry being mapped, and
  • And if possibly_authorised_entry.0 is the first entry, prior_entry is initial_entry.encoded_entry is the result of encode_entry_relative_entry(possibly_authorised_entry.0, prior_entry) where prior_entry is the Entry prior to possibly_authorised_entry in entries,
  • where encoded_token is the result of encode_authorisation_token(possibly_authorised_entry.1), and
  • Partial payloads are not permitted as they cannot be verified by the Willow Data Model.payload is the complete Payload of possibly_authorised_entry.

Finally, let drop be the result ofencrypt(contents).

Transport