DRAFT - getHash(id) - NSYNC - an eventually consistent system to share and update/verify content. Decentralised, NO Blockchain, CDN, P2P, TCP/ID

I will update this entry and turn it into a consistent documentation. But right now I need to braindump and share. Please comment!

TL;DR please

Work has started to define the protocol and implement a working prototype at the NSYNC repo.

The idea

Here is the simplest form. To everything you make available on the internet, you add a UUID, a Universally Unique ID. And this goes down to every single paragraph you write on a blog, every image you put on Mastodon or twitter, every line in your source code. OK. This isn’t easy, but just imagine it exists.

The result

Now. Every time you read a web page, you download a git reopsitory, you get the content and those UUIDs not only for the whole thing, but down to every block element, be it a paragraph, an image or a line of code. Nice. So over time you have loads and loads of UUIds. And now?

The Hash

If you want - and that is your decision - you can calculate a tolerant hash over that snippet of content and keep it with you. So now you have four things.

• the content
• the UUID
• the hash
• the timestanmp of when you generated the hash

With me thus far? OK. Now comes the kicker.

NSYNC

Whenever you read a web page, download source code or pictures, you may get an NSYNC UUID. Or not. But if you get it, you can (again, if you want to) calculate the hash.

And whenever you access that snippet, you can ask if anyone else has the same UUID and which hash they have.

Or in tech lingo:

getHash(id)

And the possible replies from your peers (or the entity claiming to be the original source) can reply with:

NACK - I've never seen that UUID, sorry.
HASH - I've seen that one! This is my hash!
validFrom (optional) - And as I've seen it, here
                       is the timestamp I consider
                       that content to be valid from.

That’s it. That’s all. Working title NSYNC, because it’s a funny name. But it becomes oh so powerful when you think about …

The consequences

You decide. You can just keep the UUIDs. Add the hashes. Or not. Add the content that created that hash. Or not. The timestamp. Or, again, not. But should you decide to keep it all - everything changes.

If someone quotes a paragraph or image or line of code somewhere elese - you don’t need to download it, if you have at least the UUID and the hash. So all of a sudden you have a globally distributed CDN (Content Delivery Network) a la CloudFare.

You ask for an updated hash with getHash(id). You will find out if that quoted content has changed, who claims it has and when (the validFrom timestamp). Depending on how much you trust that source, you can decide to add a visual marker (content might have changed) or create a diff (this seems to have changed) or just take the updated content (and new hash) wholesale.

So what?

What we have just created is very lightweight, eventually consistent way of sharing content of any kind based on a P2P (peer to peer) trust system without ANY central authority.

I will stop here - NOT

As I’ve said at the beginning, this a raw idea. A draft. A braindump. Let’s discuss. I will try my best to implement a raw prototype using a static web site generator (Jekyll) which creates this very blog. So soon you will be able to see block level UUIds here.

I am not good at JavaScript, so I will try my best to create something that can calculate the hashes and share them somehow. Almost no idea on how to do that. But you all are here too. I hope. To help. Let’s build this!

Sources

This is not at all a new idea. Project Xanadu had this fundamental concept of two-way links. Ray Ozzie pioneered this fundamental idea of distributed, eventual consistency in Lotus Notes. We had many approaches like pusubhubbub but most of them were server- not user centric.

And no. Blockchain isn’t even part of my thinking. We can do better.

Closing

As I’ve said at the beginning - this is a braindump. An idea. Iterating on many other ideas by far smarter people. I want to throw it out and share in the hope we all might find eventual consistency and a new way to share knowledge in a trusted and secure way. Is all. A big dream :)