ByteNoise

DistriWiki

A Proposal For a Distributed Wiki

The problem

Current wiki software is centralised. This causes several problems:

• One rogue administrator can permanently delete information from their own wiki, causing it to be lost forever. Any mirrors will slavishly forget anything that has been purged from the historical revisions.
• The main server can temporarily go offline, with its mirrors allowing read-only access in the meantime.
• Bandwidth costs to run a popular server are significant, requiring donations from end users.

The solution

As with USENET, the World-Wide Web, Freenet and BitTorrent, the hacker's choice of Internet protocol is a decentralised one that eschews authority, heirarchy and privilege. While there may be benefit from a small number of levels of users — eg a distinction between end users and moderators — it is generally desirable for each node on the network, each server itself, to be equal in standing to all others.

The proposal

Each wiki network consists of one or more wiki servers. Each server has one or more moderators, and can optionally allow members of the general public — end users — to edit its pages.

End users should be able to retrieve a page from any wiki, view the history of that page's changes, retrieve all the previous versions, and submit their own edits. The edits are then stored on the local wiki only, where they await approval from one of the local wiki's moderators.

Each wiki server should have a list of neighbouring wiki servers. These are any other wiki servers — the less overwhelmed the better — which have only moderators that the administrator of the wiki server in question trusts.

It should either be possible for an end user to request a given wiki server synchronise one of its pages, or a list could be maintained on the wiki server of all pages accessed since its last synchronisation, all of which would be synchronised at a routine time (for example, once a day).

In either case, a wiki server has a list of one or more pages that it wishes to synchronise. It then requests to each of its neighbouring wiki servers that they send it all more recent approved revisions of those particular pages.

Each wiki server has stored on it every version (both historical and current) of every webpage anyone has tried to access. The following information is stored for every version of every page:

• A unique identifier This can either be a UUID, an MD5 checksum, or an SHA1 checksum. (One of these should be agreed for the protocol as a whole.) This enables trees to be generated that keep track of the edits of each page, noting which previous edit any given other edit is in turn basd on. Checksums would see reverts as being identical to the original version being reverted to, whereas UUIDs would not, so the choice should be made carefully.
• The parent version's unique identifier So that people can see which version of the page the edit is in turn based upon.
• The page title With an agreed maximum length and character set. (I would suggest 72 UTF-8 characters.) Any pages with the same title are seen as revisions of the same page.
• The date and time The UTC date and time of the particular revision of the page, so that the different versions can be ordered chronologically and the most recent version determined and displayed more prominently.
• The editor's name The optional name of the end user who submitted the particular revision.
• The editor's e-mail address The optional e-mail address of the end user who submitted the revision.
• A comment An optional comment describing, explaining or justifying the revision.
• The moderator's name The compulsory name of the wiki server's moderator who approved the revision. Note that it is possible for a wiki server to store local copies of revisions that have not yet been approved by a moderator, but it should never propogate them to other wiki servers.
• The moderator's e-mail address The compulsory e-mail address of the wiki server's moderator who approved the revision.
• The wiki's name The compulsory name of the wiki server whom the moderator worked for. This is where the revision in question originated from, and where the end user directly submitted the revision.
• The wiki's URL The URL of the originating wiki server. The page name should not be included. This should take the form http://example.com/ or http://hostname.example.com/ . Adding a URL encoded (percent encoded) version of the page title to the end of this URL should retrieve the latest revision of the page in question.
• The page body The actual body of the page itself.

A wiki server should be able to directly request from another wiki server a list of all revisions made to a page made after a certain date. This will ensure that new revisions propogate across the network easily.

A wiki server should also be able to directly request from another wiki server a list of all revisions made to a page, regardless of the date. This allows a new node (wiki server) in the network to catch up with the rest.

If any wiki server has moderators who are not very vigilant, its neighbouring nodes will simply remove it from their lists. They may also choose to purge their histories of that server's submitted reivisions, although they are not required to.

In this fashion, just as IRC has several popular yet distinct and separate networks, so too could this decentralised wiki protocol allow several distinct networks of wiki servers to emerge. It is possible, and even desirable, that each network could cater to a different specific purpose, such as covering science, or popular culture, or being completist or emphasising quality over quantity, with each network fostering its own unique community.

I would appreciate any suggestions or feedback from anyone. I can be reached at zoe@bytenoise.co.uk and my own implementation of a DistriWiki is currently under development at https://sourceforge.net/projects/distriwiki/