News, Improvements, and Necessary Changes
Here, we maintain a changelog of breaking changes to our specs, and one of general site updates. Each section has their own RSS feed, and we also provide a single feed with everything here.
News and Updates
Here we’ll share bits of news relevant to the Willow protocol, as well as improvements to the site. For example, the completion of an implementation, or the addition of new explanatory text and drawings to the site. Updates will be occasional and meaningful.
A lot of everything
In the past year we built up a tremendous reservoir of unpublished work which we are finally making public. From here on out our updates will be shorter and more frequent. If you would like to keep up with our work in a more informal format, we’ve launched worm-blossom.org, where we discuss the development of Willow and much more besides.
We’ve done extensive work to the Confidential Sync (formally W.G.P.S) spec, and made it man-in-the-middle attack resistant through the notion of Read Access and Confidentiality and Handshake and Transport Encryption. Confidential Sync now supports streaming verification of payloads, and its multiplexing system has been promoted to a totally independent specification dubbed LCMUX.
The Drop Format (formally Sideloading protocol) now supports the inclusion of entries without any available payload. We have also created a more efficient encoding which will make drops smaller.
We’ve added Willow’25, a recommended set of parameters for the Willow Data model, Meadowcap, Confidential Sync, and Drop Format.
These are major changes that have brought about some spec status changes. Because the changes to Confidential Sync have been so broad, we’ve demoted its status to Proposal until we’ve finished building our Rust implementation of it. But on the upside, Drop format has been promoted to Candidate, and Meadowcap is now Final!
There’s a lot more we’ve added besides. Dozens of news illustrations and diagrams have been added to the site. We’ve added ActivityPub to the Comparison to Other Protocols page. We’ve updated and redesigned nearly every page, including the landing page.
We hope you enjoy the changes!
Rust implementation of Meadowcap released
Our own Rust implementation of Meadowcap now conforms to the specification as of August 10th, 2024. The source, can be found at the willow-rs repository and complete documentation can be found at docs.rs.
Willow Confidential Sync specification status promoted to ‘Candidate’
With a working implementation of the Confidential Sync in willow-js — and several other implementations now in progress — we feel comfortable in promoting the specification's status to Candidate.
Drop Format in willow-js
Last week we published the drop_format, and this week we have a working implementation available via willow-js 0.5.0.
For more information on this implementation, please see the documentation for
createDropandingestDrop.Willow Drop Format
A few months ago we published the Confidential Sync protocol, a synchronisation specification for securely and efficiently synchronising data over a network connection. However, there are many contexts where establishing such a connection is difficult, undesirable, or outright impossible.
The drop_format is a new encoding for securely delivering Willow data by whatever means possible. We build upon the tradition of sneakernets make use of the sporadically online and ad-hoc infrastructure users already have.
This new protocol can be used in place of (or as a complement to) Confidential Sync, is relatively simple to implement, and has a mercifully short specification.
willow-js released
willow-js, our own TypeScript implementation of the Data Model and Confidential Sync (formerly the WGPS), now conforms to these specifications as of May 6th, 2024. This module can be used to create, query, and sync Entries and their corresponding Payloads in stores in the browser and Deno runtime.
The source, documentation, and usage instructions can be found at the willow-js repository.
meadowcap-js 0.2.0 released
Our own TypeScript implementation of Meadowcap now conforms to the specification as of January 23rd, 2024. The source, documentation, and usage instructions can be found at the meadowcap-js repository.
Welcome to willowprotocol.org!
The site for the Willow protocol has been officially released to the public.
Necessary changes
Although we consider Willow’s specifications stable, unforeseen outcomes may force us to make amendments. Rather than making this a source of exciting surprises for implementors, we prefer to list those (hopefully few) changes here.
Changes to Confidential Sync and Drop Format
We have made sweeping (and breaking) changes to Willow Confidential Sync (formally W.G.P.S.).
We have made breaking changes to Willow Drop Format (formally Sideloading protocol), allowing for entries with no available payload and a more efficient encoding.
Sync protocol tweaks
We are progressing with implementing the Confidential Sync protocol (previously the Willow General Purpose Sync protocol), and have discovered some tweaks that make the protocol more powerful.
First, we have generalised the parameters for fingerprint computation in range-based set reconciliation: the old type of
Fingerprintsis now calledPreFingerprint. After computing aPreFingerprintbut before sending it to the other peer for comparison, it is transformed into an actualFingerprintwith an arbitrary function. This allows using large fingerprints with nice algebraic properties for the computations, but compressing them with a conventional hash function before transmission.The second generalisation concerns payload transmission. Instead of transmitting payloads verbatim, peers may now transform the payloads into arbitrary other bytestrings, and exchange those instead. This opens up features such as streaming verification of partial payloads, or on-the-fly compression.
Willow specification published
No changes yet!