1. 18

I am one of the many millions of lemmings who are trying out Mastodon. Mastodon says it’s federated. Are there any social networks that work in a distributed fashion like bittorrent? A network where the information is stored across different peoples devices, truly no central server?

  1.  

  2. 28

    Ditto Scuttlebutt. Which I tried a few years ago but found the UI awful (in the default app at least), and more significantly, it was a resource hog. Just joining one ‘pub’ and following a few people resulted in a half gigabyte of data downloaded, and the app kept re-indexing over and over.

    Anyway, long story short, I’ve been building my own distributed system, Tendril, with a lot of influence from Scuttlebutt but with more efficient data structures & protocols, plus the core is C++ instead of JS and it uses SQLite instead of some bespoke JS database.

    I’ve been honestly procrastinating opening it up, adding “just one more feature” instead, but I’m super close now. I just uploaded the iPad version to TestFlight today (there’s also a Mac app and a bare CLI version that runs on Linux with a browser-based UI.)

    I guess I’ve just outed myself without thinking ahead :) It does seem like a good moment to be unveiling this. Anyway, message me privately if you want an invite.

    1. 3

      I would like to be notified when you have a site up explaining it. I may ask for an invite then.

    2. 7

      I don’t know much of Secure Scuttlebutt, but I do know it seems to be there to stay (and p2p, although of course in practice some people additionally run always-on nodes on VPS to improve connectivity)

      1. 4

        Yes! Scuttlebutt is the kind of thing I’m looking for. I should have though to look on Wikipedia too, because from Scuttlebutt, I landed on this page: https://en.wikipedia.org/wiki/Comparison_of_software_and_protocols_for_distributed_social_networking

        Thank you all.

        1. 4

          I looked into Scuttlebutt several years ago and the main reason I stopped investigating it is that the core protocol definition made some catastrophic design errors; the main on being where the content-addressable ID of data was based on specific implementation details of node.js’s JSON serializer. This left them in the awkward position of trying to popularize a standard with only one working implementation, as you couldn’t interoperate with the reference implementation without a JSON library that had the exact same behavior as node.js.

          I think people were right to be suspicious of any claimed standard which only has a single implementation, though I don’t know if that’s still the case nowadays. It’s possible they iterated on the standard and came up with a successor which didn’t have that design flaw.

      2. 7

        The challenge with a distributed namespace is Zooko’s Triangle.

        I found this is an interesting read on the high-level challenges of this topic: https://www.varunsrinivasan.com/2022/01/11/sufficient-decentralization-for-social-networks

        Which inspired the Farcaster Protocol that is focused on identity vs messaging, it has a lot of similar elements to Keybase’s design too.

        There’s also Lens Protocol that is more focused on the identity vs social graph aspect (and its ownership), less about the messaging.

        1. 5

          I second the mention of Secure Scuttlebutt. There are some things I don’t like very much about it, but it certainly meets your requirements.

          1. 2

            Is what you are thinking of functionally different from a federation where most or all users run their own server and federate with each other? One of the goals with federated networks is being able to own your own data, either by running your own node or exporting your data to move across nodes. Distributing the data itself would run somewhat counter to that goal, so I wouldn’t expect it to have as much backing as the current federated networks.

            Maybe ZeroNet is also similar to what you’re thinking of.

            1. 4

              One of the goals with federated networks is being able to own your own data, either by running your own node or exporting your data to move across nodes

              In a federated system you can’t own your own data unless you run your own node, because your data is tied to a single node. Running your own node is not something most people are capable of, making this sort of an elitist goal.

              In a truly decentralized system everyone owns their own data in that (a) it’s signed, so no one can alter or forge it, and (b) anything private is encrypted, so people who weren’t meant to read it can’t read it. Admittedly you do lose control over where public data is stored, since it’s cached by any number of peers, but I believe the same is true in Mastodon.

              1. 2

                I really liked the alternative that Jon Anderson proposed in his PhD thesis, where you build a platform on top of a content-addressable file system that you back with cloud storage. Files are stored (encrypted) in blocks named with their SHA256 hash and you have a thin layer for mapping to the location. You can host the data yourself or outsource it to a cloud. He worked out that a typical Facebook user would pay around $1/year to host in the cloud, which is about an order of magnitude cheaper than having a RPi powered on at home, even ignoring capital costs, and is more reliable.

                He built a proof of concept for his PhD but concluded that there was no market for people paying $1/year for social networking. That may no longer be the case.

                1. 1

                  Interesting. The thing I’m building has a CAS in it — every entry/post/message has a hard 4KB limit, and anything bigger goes in the CAS and is referenced by its digest. So far I haven’t put much work into the replication of the CAS beyond pairwise syncing.

                  1. 1

                    That’s exactly what we’ve done with Peergos, would love to talk and hear your thoughts if you’re interested. Do you have a link for the thesis? We talk about the S3 access and the social layer on top of the filesystem here:

                    https://peergos.org/posts/direct-s3

                    https://peergos.org/posts/decentralized-social-media

                    1. 2

                      I couldn’t immediately find it. His system was called Footlights and should be somewhere on the Cambridge web site.

                      It looks as if your use of S3 is similar to that proposed for AFS, so seems sensible. I’d love to see the rest of the protocol implemented in a way that could be run in FaaS offerings, so you could really use the pay-what-you-use model for cloud billing, rather than needing a VM.

                      1. 1

                        Thanks! That’s a very cool idea! Most of our complexity is client side, so the server is very simple, even more so if it is single user.

                  2. 1

                    You can go one step further and also control access to the ciphertext blocks themselves, and not just rely on encryption for privacy. We invented a capability based scheme for this for Peergos: https://peergos.org/posts/bats

                2. 2

                  There’s https://cabal.chat/ but I think it’s only chat (for now?)

                  1. 2

                    Twister is a Twitter-like microblogging platform that utilizes the same blockchain technology as Bitcoin, and the file exchange method from BitTorrent, both based on P2P technologies.

                    The project is mostly dead now but it was a cool concept.

                    1. 2

                      May want to check if someone built one with https://hypercore-protocol.org/. Probably not really big if it exists.

                      1. 2

                        Blog on ipfs?

                        1. 2

                          I’m surprised nobody has mentioned nostr yet. It isn’t peer-to-peer like BitTorrent, but your identity is stored on your client, and not on a server.

                          I haven’t really gotten into it much, but it does look elegant.

                          1. 1

                            I think the only fair way to have that social network is to set a standard for the data representation and the name resolution and then let peers do whatever locally. Scuttlebut made the mistake of relying on the json output of some specific lib so no one was able to make their own implementation for the longest time.

                            My approach is I think the convergence point but not much software exists as of now. I’ve been trying to find collaborators to work with but I have a problem with advertising because the ends is just another means … so my means has been as far as possible consistent with what I think is okay, so without the cooperation of media my only method of letting people know about what I am doing (where I live) is to meet people in reality and ask them to join my meetup or spread the word. Problem being that since I have this agenda I feel it is unfair of me to approach others with it and so I wait for others to approach me, this is possibly the least effective recruitment strategy of all time. However I am patient because I believe in the maths of it.

                            So a lot of my desperation for some people to play along overflows into HN and lobsters… anyway as you guys know; datalisp.is.

                            1. 1

                              What’s the point, other than technical curiosity, in storing this kind of stuff in a distributed way?

                              1. 4

                                If your social network is stored in a non-distributed way, whatever centralized entity stores the data that constitutes that graph has a good deal of power over your social life as mediated by their platform. This is why the technical and political choices that the large social networks Twitter and Facebook make are newsworthy events that genuinely affect the lives of lots of people.

                                1. 1

                                  I’m not fully tin-foil hat, but I do worry about relying on proprietary platforms for gossip, which is what Twitter is. For innocuous things like a community of writers, stupid posts that get suspiciously large amounts of engagement are not society breaking, but for political things, they are. A p2p social network with an open source implementation would possibly be free of the financial incentives for diddling what gets shown on your feed.

                                2. 1

                                  I’ve had a plan for this type of social media for a long time. It’s similar in concept to Secure Scuttlebutt, but more narrowly focused to just your friends. Hopefully, one of these days, I’ll actually finish it. It’s called MostPost:

                                  • Structured like Facebook or Google+: you have friends, and you communicate only with your direct friends. Not a “communicate with the whole world” network like Twitter/Mastodon/SSB.
                                  • Accounts are uniquely identified by an email address and a public key. Friend requests are sent over ordinary (encrypted) email; this is the first step that establishes identity.
                                  • Features like photo albums and Stories.
                                  • Extremely granular control of identity and privacy: G+ style circles, and you can have multiple identities, email addresses, and public keys merged into one interface, while appearing as a different person to different circles.
                                  • Direct p2p connections are established between friends, to share signed+encrypted updates with a gossip protocol. You only ever directly connect with friends.
                                  • …except for the initial process of finding a friend’s IP address, which uses a global DHT (for NAT busting or changing devices).
                                  • All of your data is stored in your app. There is no server, just the app database.
                                  • A twitter-like global social network could eventually be built on top of the friend graph, using a degrees-of-separation provenance system to determine which messages you relay, ensuring that (so long as you choose your friends wisely) spam is rare and problematic friends-of-friends can be blocked along with everything they relay.

                                  The most difficult part is the database, and syncing between multiple devices. The local database will be huge, must be encrypted at rest, and must be passively synced between devices so that switching devices is seamless. This led me to develop Osmosis… which has been in development limbo for years as I keep changing the feature set and rewriting it.

                                  1. 1

                                    There are lots of them, but I actually thing for this use case distributed is less flexible than federated. A federated protocol you can host your own, or not, and host it anywhere. Distributed you have to host your own, and often in up doing it on an end device like a mobile phone where it eats the battery.

                                    1. 1

                                      Are there any examples of battery-friendly P2P protocols?

                                      Perhaps protocols could have two tiers of P2P nodes: leaf nodes and supernodes (name stolen from Skype). Phones would be leaf nodes, and would not require constant connectivity. Perhaps the leaf-supernode trust relationship could be assured via Verifiable Data Structures ( https://security.googleblog.com/2021/07/verifiable-design-in-modern-systems.html ).

                                      Federated protocols (Mastondon, XMPP, Email) are just that, but without the cryptographic assurance between leaf and supernode. In practice that might be sufficient.

                                      1. 1

                                        Yeah, exactly. Before I got to your last bit I was thinking “you’re just describing federation” heh

                                    2. 1

                                      We have an interesting setup with Peergos (https://peergos.org): it’s P2P, but you can trustlessly use a remote server. It’s a self authenticated protocol and E2EE (even the web client doesn’t trust what the server returns). The idea is that most people don’t want to or are unable to self host. This design, combined with portable identity, means they get all the privacy and control benefits of self hosting, but whilst outsourcing storage to a server somewhere. It’s built on IPFS and libp2p.

                                      To read specifically about the social network (it’s much more than that) see:

                                      https://peergos.org/posts/decentralized-social-media