1. 22
  1.  

  2. 8

    Distributed hash tables are great, but they solve the easy problem - distributing static content. We need to figure out how to decentralise dynamic sites and their databases over unsecure links and to potentially malicious nodes.

    Oh, and we want to update them too, but not have to run a single point of failure server in order to do so.

    1. 6

      Distributed hash tables are great, but they solve the easy problem - distributing static content. We need to figure out how to decentralise dynamic sites and their databases …

      There’s an easy trick though: Your DHT works because (in pseudo-erlang):

      on_dht_store({K, V}) ->
        if K == crypto:hash(sha256, V) -> …
      

      however if you also implement:

      on_dht_store({K, {S, V}}) ->
        if crypto:verify(ecdsa, sha256, V, S, K) -> …
      

      then you have a DHT that can store dynamic content which is addressed using a public/private keypair: Simply sign all of your updates.

      … over unsecure links and to potentially malicious nodes.

      Actually we don’t. The DHT “verifies” updates even if it accepts them from anywhere. There’s no need for something insecure like SSL/TLS in such a system because the name of objects is verified by its content. Malicious nodes can waste bandwidth (but can anyway), but they can’t corrupt the DHT.

      1. 2

        I think what’s meant by dynamic here is sites like, say Facebook, where what you see is rendered by a server specifically for your request. If we’re doing away with central servers, we should probably find a way to distribute their processing (and databases etc.) as well.

        1. 1

          one idea is to do computing client-side, where the use of a social network would include running some software that integrates static content signed by content producers. is there anything that would require a more complex approach?

          1. 2

            A shopping cart, and shopping in general comes to mind.

            1. 1

              Client simply publishes an order file on the DHT that is signed and encrypted to the store.

              We have been ordering things with PGP and email forever.

              1. 1

                that’s a good point, shopping is a special case of secure synchronous communication, which i agree is not well handled by distributed static content. the shopping cart could be kept track of client-side, but to place an order you’d need to establish a secure communication channel with the shop, just as you would if chatting with a friend. then your client would send a message to the store with the necessary order/payment information. but that seems to me like a simpler approach than thinking of it as distributing dynamic web pages.

                do we need any more primitives beyond (a) distributed signed static content and (b) secure connections for synchronous communication?

                1. 1

                  There’s nothing about shopping that requires synchronous communication. People have been buying via mail for centuries.

              2. 1

                In general, how do you handle private data? Where does the client store my private messages? Even worse, who handles private messages arriving at me when my client isn’t online?

                1. 4

                  Private messages are not an issue. In the simplest scheme, the message is encrypted so only you can read it. Then it is distributed to the network CAS/DHT/whatever just like any other data, where your client will find it once online.

                  If you care about the details of a more elaborate scheme, check out freemail: https://github.com/freenet/plugin-Freemail/blob/master/docs/spec/spec.tex

                  1. 2

                    i guess for email you’d still need some way to check for messages that are sent to you; you’d have to know where to look to get the message.

                    1. 3

                      That’s exactly what the signed, key-addressed storage that geocar proposed above solves.

                      The rest is implementation details. I.e. how exactly do people agree on the keys that will be inserted into by the people who want to send someone a message. In Freenet there’s a web of trust of identities that publish information and “puzzles” that are used to derive keys. Ultimately the recipient must poll for updates to these keys.

                      It’s not an unsolved problem; freemail, fms, and freetalk have been around for years.

                      1. 1

                        interesting. how do you think ipfs fits in with freenet? do they seek to solve the same problem? if ipfs is trying to solve a subset of freenet’s goals, why not just use what freenet already has?

            2. 2

              then you have a DHT that can store dynamic content which is addressed using a public/private keypair: Simply sign all of your updates.

              Now implement a Wikipedia clone using that. The distributed storage and the client side javascript are not enough to implement proper dynamic sites. We need distributed databases with schema versioning, access control, consistency guarantees, etc. It’s not something you can fake with a DHT.

              1. 1

                Now implement a Wikipedia clone using that.

                It’s been done.

                magnet:?xt=urn:btih:1379652cf48c825d71dd4a4d9c539f0268e82778

                1. 1

                  Now implement a Wikipedia clone using that.

                  It’s been done.

                  magnet:?xt=urn:btih:1379652cf48c825d71dd4a4d9c539f0268e82778

                  Is this supposed to be a joke? That’s just an XML dump of the most recent version of the actual English Wikipedia articles: https://torrentz2.eu/1379652cf48c825d71dd4a4d9c539f0268e82778

                  1. 1

                    Whose index (I’ve demonstrated) can be stored on a DHT.

                    There’s nothing being faked here.

                    1. 0

                      Whose index (I’ve demonstrated) can be stored on a DHT.

                      Do you not understand the difference between a static dump and a dynamic site that allows collaborative editing?

                      1. 1

                        Do you?

                        Imagine Wikipedia in the year 10k spanning star systems: TCP is no longer an option. How does my distant descendant make a change?

                        The most obvious way is that they write a change proposal, perhaps as a structured JSON object, digitally signs it (e.g. with PGP), and then publishes the proposal. This is easy because everyone has the private keys for Wikipedia10k.

                        Any Wikipedia “reader” thus polls the DHT for items marked as proposal, and while there’s a lot of guff out there, any eventual consistency algorithm they like can be used to “locally” reconcile what their potential Wikipedia can look like.

                        Perhaps the missing trick is that in 2017, people still remember what it was like to have limited storage and so there’s an impulse to reconcile early to save those precious spinning platters of rust.

                        1. 1

                          Imagine Wikipedia in the year 10k spanning star systems

                          Imagine wanting to replace the Internet with a system that only supports static assets in a distributed key-value storage.

                          Any Wikipedia “reader” thus polls the DHT for items marked as proposal, and while there’s a lot of guff out there, any eventual consistency algorithm they like can be used to “locally” reconcile what their potential Wikipedia can look like.

                          Yeah, good luck serving millions of diffs per page and letting the end user figure out which ones should be assembled based on cryptographic signing and timestamps.

                          1. 1

                            Imagine wanting to replace the Internet with a system that only supports static assets in a distributed key-value storage.

                            There’s only static assets; HTTP over TCP only sends static assets: You put a static request someplace and you get a static response something else. A DHT is no different in this respect. Client publishes their request, server publishes their response.

                            The cool thing about a DHT is that it’s much more durable than TCP.

                            Yeah, good luck serving millions of diffs per page and letting the end user figure out which ones should be assembled based on cryptographic signing and timestamps.

                            My laptop is fast enough to do it for a “small” site like Wikipedia which deals with only about ten updates per second – a quick check shows I can cpu-hash about 2GB per minute cold so that means each “page” could be around 3MB. If I’m only interested in part of Wikipedia (for example, the English-language pages) then it’s even less work.

                            In the future computing power will be cheap enough to make any current limitations that actually exist, moot.

                            1. 0

                              In the future computing power will be cheap enough to make any current limitations that actually exist, moot.

                              This reminds me of http://wiki.c2.com/?SufficientlySmartCompiler

                              Design for the technology at hand, not for your dreams of what the future might bring.

            3. 4

              Thats a good point, though in the meantime, IPFS could be a great solution for the various static website generators. I think I’ll try making my hakyll website accessible on IPFS.

            4. [Comment removed by author]

              1. 1

                Through killfiles, like in Usenet.

                In the extreme, some may choose to whitelist, and only read posts from people they’ve pre-approved.