I am poking this problem space right now and there seem to be a mountain of unexplored territory, both in the abstract with how names, namespaces and taxonomies work to link together abstractions, but also in the technical and HCI – simply because DNS just is the go-to hierarchical solution and much of P2P tried to avoid user addressable entities in favour of data-driven search or juggling primitives (e.g. magnet links).
As part of a nlnet sponsorship I am working on A12 where the constraints are such that DNS is not an option (e.g. transiently airgapped networks) where fleets of user-facing devices still need to dynamically discover and join together to form a network transparent desktop.
One of the tactics is using petnames at border to/from cryptographic keymaterial then on the protocol level discovery in local broadcast domain from ts,nonce,H(Kpub, Ts, nonce), such as ‘foo’ was discovered or try to connect to ‘bar’. Then on the WAN level forming onion like circuits via rendezvous directories in the open, such as ‘found bar through foo’.
A “petname system” is a database and a set of interfaces which use that database to bidirectionally map human readable names to cryptographically secure names.
OK, each user will have a secure database which maps unique IDs to human-readable aliases.
Where does that database live? It definitely can’t live on any specific device, because I (as a user) have many devices, all of which need to have the same “petname resolution” behavior.
So this means the database needs to live at some third-party endpoint, which all of my devices can access. This (necessarily!) represents a delegation of trust.
And I’m definitely not going to run my own petname database, I’m going to outsource that work to a hosting provider. So then what is the difference between this outcome and, basically, DNS? The trust model is more or less the same, isn’t it?
My petname database is stored locally on devices I own and replicated across my devices.
Data doesn’t have to be centralized in order to be accessible from more than one location. Decentralized systems have existed for decades, and interest in them is growing. The Petnames paper introduces one small part of that, which is decentralized naming systems.
The reason my ad-hoc system is not as convenient as storing all my personal information in the Apple or Google cloud is due to the economic incentives of capitalists for whom “personal information is the new oil”. They can pump a lot of money into software with a polished UI, and they call sell hardware with that software preinstalled.
But I am hoping to transition to more convenient and easy to use software that is private and local-first. There are a number of projects. Today I am planning to dive into <anytype.io> and see if that meets my requirements. Please reply if you have opinions or experience with private, local-first, decentralized personal computing software.
Data doesn’t have to be centralized in order to be accessible from more than one location. Decentralized systems have existed for decades, and interest in them is growing. The Petnames paper introduces one small part of that, which is decentralized naming systems.
Data doesn’t have to be centralized to be accessible on multiple devices, true. But devices have to know where to send their local updates, and how to receive updates from other devices. And the only practical way to do this kind of synchronization does require a centralized coördinator.
For one reason, because updates between arbitrary devices like phones can’t rely on peer-to-peer, gossip-style communication if they’re expected to be reliable. You need a clearinghouse to take responsibility for message receipt and re-broadcast.
But more importantly, because you need a single source of truth for the set of valid devices. If a device is lost or compromised, you need to be able to revoke that device from the list with authority.
So, I’m curious! —
My petname database is stored locally on devices I own and replicated across my devices.
How do you replicate it?
Please reply if you have opinions or experience with private, local-first, decentralized personal computing software.
I’m down with local-first! But there’s no avoiding a centralized coördinator. If you don’t want to trust that coördinator, that’s fine, you can encrypt before sending, and decrypt after receiving — but that makes the payload opaque, which makes optimizations (delta compressions, etc.) really difficult.
—
Today I am planning to dive into <anytype.io> and see if that meets my requirements.
It looks like this service hosts user data encrypted with a single shared key on a (private?) set of IPFS nodes. Those nodes act as the centralized coördinator I describe above.
IPFS is a content-addressable storage system, so updating a document will create a new document with a different address. This kind of punts on all of the data synchronization issues, and means that updates to a document are not observable by consumers of that document, without some additional out-of-band effort. It doesn’t look like devices can be managed individually, or have unique keys. It doesn’t look like the key(s) can be rotated.
In the petnames essay, it says “names can have two out of three properties: decentralized, globally unique, human meaningful.”. Petnames are decentralized and human readable, but not globally unique. Decentralized architectures are a compromise: you gain useful properties, you lose other useful properties.
If I store all my personal data in a third party cloud, then I have a single source of truth (something you want) but I lose properties that are more important to me. First, I have to trust the third party that runs my cloud. Second, I can’t work offline. I want a local-first compute environment that is highly tolerant of being offline. What I am willing to give up is a single source of truth. If I edit documents on offline devices, then they become out of sync with their replicas on my other devices, until synchronization time arrives. If I make changes to the same document on different devices, I may have to resolve conflicts. I’m sure you’re aware that git can support this kind of decentralized workflow.
For a specific example, I have a cellphone, a laptop, and a desktop. They each contain copies of my personal data. I sync data manually between them, but there is no master. I’m looking for something more seamless and automatic, while remaining decentralized. Suppose I’m travelling with my laptop and cellphone. I have no internet or cellular connection. I’d like to be able to seamless sync my laptop and cellphone (via bluetooth or wifi hotspot) without a lot of ceremony. Neither one is the master; new changes on one get propagated to the other. My desktop at home won’t be synced in this situation, since it isn’t reachable. Maybe that explains why I want a decentralized system without a single source of truth.
Anytype.io uses different definitions of “local first”, “decentralized” and “open source” than the ones I was expecting.
For a specific example, I have a cellphone, a laptop, and a desktop. They each contain copies of my personal data. I sync data manually between them, but there is no master. I’m looking for something more seamless and automatic, while remaining decentralized. Suppose I’m travelling with my laptop and cellphone. I have no internet or cellular connection. I’d like to be able to seamless sync my laptop and cellphone (via bluetooth or wifi hotspot) without a lot of ceremony. Neither one is the master; new changes on one get propagated to the other. My desktop at home won’t be synced in this situation, since it isn’t reachable. Maybe that explains why I want a decentralized system without a single source of truth.
I get it. There is no authoritative source of truth for your personal data, there is only the individual view(s) of that data on each device, which are the result of applying all of the updates received by the device. And it should definitely be possible to share updates between devices directly, without going through a centralized coördinator. I agree!
The problem is: how? Specifically?
Are you aware of any mechanism for an iOS application to broadcast an update to a set of trusted peers, and for that update to be reliably received by a commandline application that’s running on a nearby laptop which identifies as one of those peers? And furthermore, that those updates can also be reliably delivered to an Android application which only connects to the internet (or comes close to the laptop or iPhone) some days later?
As far as I’m aware, there is no practical solution to this problem. Bluetooth, Bonjour, etc. just don’t provide the necessary discoverability and interoperability. Even if they did, it’s insufficient — I shouldn’t need to bring my Android physically next to my iPhone for them to have the same view of my data, I should only need to connect them both to the internet. Moreover, if I connect my iPhone to the internet on Monday, whatever updates it has should be received by my Android when I connect it on Tuesday. These kind of table-stakes requirements are only solved with a centralized and stateful coördinator.
This UX is what matters, and UX is bound by the constraints of the underlying technology. All available evidence suggests the best compromise solution for these kind of use cases is to proxy updates through a centralized third-party service over the internet. That sucks, sure! But fixing it requires changes the underlying protocols, and the APIs they provide to relevant devices.
Here’s what I would consider a practical solution for me, and it’s fully decentralized and peer-to-peer. If any two of my devices are both connected to the internet at the same time, then, using an appropriate protocol, they discover each other and synchronize with each other. In practice, my desktop is always running and connected to the internet, so it would normally be always available to act as a synchronization point. For other people, their cellphone is always turned on and connected to the internet, and could serve the same role.
Given appropriately designed software, the UX is simply the requirement to register each new device you get with an existing device in the peer group, so that they all know about each other and will sync with each other. This is not any more difficult than registering each new device with your third party centralized sync service.
I haven’t tried to actually implement this, but I know that suitable protocols exist, and that they can use distributed hash tables to allow peers to find each other without the need for static IP addresses on all peers. Tor Onion Services are an existence proof that this is possible.
I also said I want to sync locally if the internet isn’t available. You say that this isn’t practical, and claim that Bluetooth and Bonjour aren’t good enough. But you don’t convince me that this isn’t possible. Elsewhere on this page, @crazyloglad mentions they are working on “(Arcan) A12 where the constraints are such that DNS is not an option (e.g. transiently airgapped networks) where fleets of user-facing devices still need to dynamically discover and join together to form a network transparent desktop”. If they can figure out how to do that, then why can’t we have software that lets two of my devices discover each other on a LAN and sync? Maybe I should learn more about Arcan.
Here’s what I would consider a practical solution for me, and it’s fully decentralized and peer-to-peer. If any two of my devices are both connected to the internet at the same time, then, using an appropriate protocol, they discover each other and synchronize with each other. In practice, my desktop is always running and connected to the internet, so it would normally be always available to act as a synchronization point. For other people, their cellphone is always turned on and connected to the internet, and could serve the same role.
…
I haven’t tried to actually implement this, but I know that suitable protocols exist, and that they can use distributed hash tables to allow peers to find each other without the need for static IP addresses on all peers. Tor Onion Services are an existence proof that this is possible.
There’s a lot of complexity hiding behind “an appropriate protocol” :)
Discovery requires some kind of coördination. You can have everyone register with a centralized server, like DNS. Or, you can do some kind of P2P thing, like Tor/Onion. But you can’t avoid seeding each node with an initial set of peers, which you either have to specify manually (laborious, unreliable, bad UX) or — you guessed it — fetch from a centralized server.
(In fact, even if you’re manually specifying the initial set of peers, you’ve almost certainly just copy/pasted those IPs from a website somewhere, which is effectively the same thing as querying a centralized server for them!)
Given appropriately designed software, the UX is simply the requirement to register each new device you get with an existing device in the peer group, so that they all know about each other and will sync with each other. This is not any more difficult than registering each new device with your third party centralized sync service.
While end user devices can pretty easily make outgoing connections, it’s unfortunately very difficult to get them to reliably accept incoming connections. Most carriers and many ISPs will prevent it outright, and even when they don’t, there’s an infinite universe of complexity in dealing with NAT.
Systems that need end-user devices to be able to accept incoming connections (like Tailscale for example) often solve this problem by having the device make an outgoing connection to a — you guessed it — centralized server, which proxies incoming requests back through that connection.
I also said I want to sync locally if the internet isn’t available. You say that this isn’t practical, and claim that Bluetooth and Bonjour aren’t good enough. But you don’t convince me that this isn’t possible. Elsewhere on this page, @crazyloglad mentions they are working on “(Arcan) A12 where the constraints are such that DNS is not an option (e.g. transiently airgapped networks) where fleets of user-facing devices still need to dynamically discover and join together to form a network transparent desktop”. If they can figure out how to do that, then why can’t we have software that lets two of my devices discover each other on a LAN and sync? Maybe I should learn more about Arcan.
Oh there are all kinds of mesh network protocols and local network discovery mechanisms, almost uncountably many. The problem to solve here isn’t the technology. The problem to solve is the integration with the device and the user experience. Making it usable.
I say Bluetooth and Bonjour and all current technologies aren’t viable, because (as far as I’m aware) there is no software in existence today which has the synchronization behavior you describe. If it were feasible to do, then I have to believe that someone would already be doing it.
The ‘centralized versus distributed’ is a misdirect or a false dichotomy; there is a substantial difference from DNS as an externally authoritative distributed asynchronously mutating namespace versus internally (yourself) synchronised swarm of cutesy names you call things you like/dislike over ephemeral rendezvous points (coordinates really). Centralised as in pub, not as in tax office. Relatedly, scale isn’t millions of names but the much more manageable tens-to-hundreds – your silicone family.
Unless ‘hung drawn and quartered’ comes back into fashion your are also not geographically distributed (and if it is, well, other problems take priority. You might be in Japan, you might be in Germany or travel between the two but there is a substantial latency and tolerance depending on the time at airport checkin.
For that, pub/subscribe with the “trusted third party” being another instance of you and part of your namespace suffice as petname is transport securing pub-key doubles as state-authenticating signing key. Meet up at the pub at table #5 to coordinate the heist kind of a thing, leave ICANN out of it.
You can punch your swarm of devices as a synch operation (or NFC tap / ubikey for when kinetic energy fails to initiate key transfer) or huddle up before playing ball, You can’t punch DNS as easily as NAT (even though one might like to do that on a daily basis) as others get sad and disagree with your naming of things.
It seems like you’re suggesting that resolving a given identifier could yield different results depending on client identity, locality, or jurisdiction. Is that true? If so, can you give some examples of this kind of identifier?
Something which doesn’t qualify is, say, Google. When I type google.com I need to get the same logical result, regardless of where I am, or what my local pub is, or what I decide is my silicone family. The concept of google.com is invariant to all of those things. Any system which returns a different logical result for google.com based on any of those variables is broken. My local pub does not get to decide what google.com means.
(Different physical results that take advantage of e.g. CDN locality is a different thing, of course.)
The LAN discover case is somewhat easy, depending on threat model. compromised-by-default IoT devices or others on your corporate network building public key databases or leveraging discovery beacons for denial-of-service amplification is a somewhat spicier challenge. mDNS (Bonjour) comes from the era where this wasn’t much of a threat and pragmatism “I need to find my printer, whatever the cost” ruled (and the cryptographic tools needed weren’t exactly for public consumption at the time).
Unless you are the retrocommunications nerd running your network as a token ring or single chain coaxial, chances are kind-of high that the stricter ‘peer-to-peer’ definitions will fail by being bound to a physical layer that has a hub an spokes kind of a topology. There’s still a difference from that as centralisation to one of [FAANG] serving the entire world and forcing that as a design constraint for how the protocol is supposed to scale.
The HCI model is more involved than that (at least from my ‘networked desktop’ perspective) as it is not just about the different device having synchronised access to the same world state, but for them to act as different kinds of sources and sinks. This is why I have a local namespace of petnames (“connection points”) that correspond to access control models that links together with the networked form. Examples would be to connect “this part of my desktop is allowed to manage the clipboard and inject input” to a networked device that I am actually using to generate the input, or “my mobile phone is allowed to act as video source (webcam) on my desktop, but not synch my documents folder”.
For rediscovery on the global internet you will need some form of third party rendezvous, but that does not have to be an 8.8.8.8 like behemoth, but a virtual piconet like master of your very own terror cell. That one can also act as traffic relay or NAT punch, as well as state store / backup. Alas it also opens up the can of worms that is ‘transitive trust’ as the other option being manual management, which is .ssh/id_rsa annoying or the physical form of annoying with having a TPM in your keyring.
A Petname system implies nothing about your database being secure. It’s the underlying names in it that are secure, in the Zooko’s Triangle sense, i.e. no one else can use that name because they don’t know the corresponding private key.
You probably want the database to be secure for the same reason you want your existing contacts db to be secure, but it’s not a requirement of the system. In practice you can do things like add the cryptographic names as custom fields to address cards and keep using iCloud or Google or whatever.
A petnames database hosted for you somewhere requires trusting fewer people and machines. With DNS you delegate trust to the whole unencrypted path between your machine and the authoritative nameserver for whatever domain you want to reach.
You could also encrypt your database and then so long as you trust your software you could trust that your petname host could only do denial of service and rollback attacks rather than just feeding you arbitrary names.
Same for other sources of petnames, if they’re signed then they can be significantly higher trust than DNS.
I imagine what GP is thinking of is more like “Syncthing easy”, but you could even have your petname database sync with iCloud if you really wanted to.
So this means the database needs to live at some third-party endpoint, which all of my devices can access. This (necessarily!) represents a delegation of trust.
Not if the database is encrypted with a local key or password. Sure you’d have to download it and upload it again every time you need to synchronise it (I’m guessing it could be automated as a background task), but it would prevent any funny business from the provider.
OK, sure, but that just kind of moves the issue of trust away from the DB provider, and into the key manager. That is, any reasonable UX for managing those keys across all of your devices would necessarily need to live at some third-party endpoint, for the same reasons. And now we’re back to the same issue of trust delegation.
Sure, when I use KeepassXC I have to trust KeepassXC. But if the protocol between DB and key management is standard, the DB provider and key manager may be separated, and protect me from the DB provider. At least with this scheme any bug or backdoor from the key manager may be spotted (if it’s Open Source like any security product should be).
Not perfect, but still better.
Or just duplicate your pet names across your devices. I do that with my password database, no online anything involved, and the occasional desync has yet to be a deal breaker.
I appreciate that your password manager doesn’t sync through a third party. I hope it’s not controversial to observe that this is a fairly niche setup, and is really only viable for technically sophisticated users such as yourself.
These edge cases are interesting and useful, but the ultimate test is the general case. It needs to be feasible for grandma to use the thing. Grandma can definitely be convinced to use a password manager, but only if it works across her iMac, tablet, and phone without extraordinary effort. She’s definitely not going to sync a password DB between those devices manually.
The only way to solve this problem, AFAICT, is to delegate that work (and trust) to a reliably accessible third party.
The only way to solve this problem, AFAICT, is to delegate that work (and trust) to a reliably accessible third party.
Under Capitalism. Otherwise I could see manufacturers agree on a standard, such that grandma can have one client per device, and one service of her choosing, all independent from each other if she so choose (typically that would be a Microsoft and Apple clients with a Google server). She would have to trust her client (Microsoft and Apple) that her data is not leaked, but the server (Google) will not be able do decrypt anything without her password.
Now syncing is a bit expensive: download the entire DB, merge, upload the whole thing again. Or maybe chop it up in a content addressed manner, but then you’d leak quite a bit of metadata, such as the number of entries. In any case, I can see it working for Grandma, if the big FAANG agree on such a standard even though they probably have no interest to.
The only reason Grandma can’t have that is because this kind of setup is not the default. If it was it would be as easy as navigating Facebook.
Well, two points. One, capitalism won and it’s not going anywhere, so there’s not much point in talking about alternatives, you just gotta treat it as an invariant. Two, I don’t think it’s capitalism that makes this true, I think it’s just that the market has made it clear what a viable UX looks like. Manufacturers won’t agree on a standard unless doing so serves their interests. Decentralization doesn’t really serve anyone’s interest, so there’s no incentive. And why would Google host this opaque data? It can’t do anything with it. Does grandma care if it’s encrypted on the server? No! Grandma cares about product features and capabilities. Encryption is incidental.
Sadly I’m afraid I’ll have to agree with you. Which is why we’re all gonna die.
The problem with capitalism is that it cannot last. Finite boiling world and all. What comes after depends how it phases itself out. Unfortunately I’m anticipating a severe reduction in human population (we’re not all gonna die, but I can easily see most of us dying), and with it disruptive social changes. I don’t see capitalism surviving its own collapse. Or maybe the situation isn’t as dire as I think, and we’ll just continue the cyberpunk route we’re currently taking. Some kind of corporate fascism, if not good old fascism.
The rest of your analysis I also agree with: the incentives just aren’t there. They could make a nice UX for privacy-preserving decentralisation and all that jazz, but I agree they just won’t.
I haven’t lost all hope however. I do believe an alternative to capitalism has a chance. But just thinking that throws me outside the Overton window, and that’s a problem.
This is a really good intro to petnames. I was immediately struck by the utility of this when I first heard of them (20-ish years ago?) and I’ve incorporated them into my perennially-unready Tendril P2P social network.
I quite liked reading this design. My favorite part is that it’s a local-first design, not just in the fact that it supports offline usage first. The design itself is ready for experimentation by individuals. There’s no network protocol, consistency model, etc. to blur the core schema and motivation.
One thing confused me, though. Is the author proposing replacing (or at least augmenting) DNS with petnames? The text suggests a way for domain name registries to depend on widespread petnames adoption:
New registries with upgraded security approaches could also be added. For instance, a registry-oriented naming hub could have a strict policy of allocating all “nearby” names as a package whenever a domain name is sold, so that scammers cannot claim a lookalike name later.[8] For instance, “DomainSpace ⇒ paypa1.com” automatically redirects to “DomainSpace ⇒ paypal.com” because paypa1 was close enough to paypal it was sold as part of the same package.
Does this propose to solve the lookalike domain problem by asking registries to sell domain names in batches? To my mind, this breaks with the spirit of the rest of the design, where individuals are encouraged to think about their trust relationships. Even if we trust registries, this proposal just pushes the lookalike problem into a widely distributed web of registries that will certainly have their own lookalike domain name batches. Even if we converge on a lookalike batching procedure (perhaps using the cited Unicode confusables data), we would almost certainly see huge (10^7) batches for even the shortest domain names.
This proposal is an interesting inversion of the cited Petname Tool, where individuals set their own “edge names.” I wonder why the author chooses a different design. Perhaps the author is gesturing toward the distributed model, where individuals regularly subscribe to well-known lists of edge names. Instead of gesturing toward that, how about a new design on top of petnames that proposes using the Unicode confusables in a browser extension?
Maybe I’m blowing up this aside too much. I also see other people talking about PKI. What’s the connection? Is this some abstraction over some key management and distribution scheme? I’m not sure I got the “secure” part, especially since there’s lots of implied TOFU. For instance, why and how does Dr. Nym trust bizdir?
I am poking this problem space right now and there seem to be a mountain of unexplored territory, both in the abstract with how names, namespaces and taxonomies work to link together abstractions, but also in the technical and HCI – simply because DNS just is the go-to hierarchical solution and much of P2P tried to avoid user addressable entities in favour of data-driven search or juggling primitives (e.g. magnet links).
As part of a nlnet sponsorship I am working on A12 where the constraints are such that DNS is not an option (e.g. transiently airgapped networks) where fleets of user-facing devices still need to dynamically discover and join together to form a network transparent desktop.
One of the tactics is using petnames at border to/from cryptographic keymaterial then on the protocol level discovery in local broadcast domain from ts,nonce,H(Kpub, Ts, nonce), such as ‘foo’ was discovered or try to connect to ‘bar’. Then on the WAN level forming onion like circuits via rendezvous directories in the open, such as ‘found bar through foo’.
OK, each user will have a secure database which maps unique IDs to human-readable aliases.
Where does that database live? It definitely can’t live on any specific device, because I (as a user) have many devices, all of which need to have the same “petname resolution” behavior.
So this means the database needs to live at some third-party endpoint, which all of my devices can access. This (necessarily!) represents a delegation of trust.
And I’m definitely not going to run my own petname database, I’m going to outsource that work to a hosting provider. So then what is the difference between this outcome and, basically, DNS? The trust model is more or less the same, isn’t it?
My petname database is stored locally on devices I own and replicated across my devices.
Data doesn’t have to be centralized in order to be accessible from more than one location. Decentralized systems have existed for decades, and interest in them is growing. The Petnames paper introduces one small part of that, which is decentralized naming systems.
The reason my ad-hoc system is not as convenient as storing all my personal information in the Apple or Google cloud is due to the economic incentives of capitalists for whom “personal information is the new oil”. They can pump a lot of money into software with a polished UI, and they call sell hardware with that software preinstalled.
But I am hoping to transition to more convenient and easy to use software that is private and local-first. There are a number of projects. Today I am planning to dive into <anytype.io> and see if that meets my requirements. Please reply if you have opinions or experience with private, local-first, decentralized personal computing software.
Data doesn’t have to be centralized to be accessible on multiple devices, true. But devices have to know where to send their local updates, and how to receive updates from other devices. And the only practical way to do this kind of synchronization does require a centralized coördinator.
For one reason, because updates between arbitrary devices like phones can’t rely on peer-to-peer, gossip-style communication if they’re expected to be reliable. You need a clearinghouse to take responsibility for message receipt and re-broadcast.
But more importantly, because you need a single source of truth for the set of valid devices. If a device is lost or compromised, you need to be able to revoke that device from the list with authority.
So, I’m curious! —
How do you replicate it?
I’m down with local-first! But there’s no avoiding a centralized coördinator. If you don’t want to trust that coördinator, that’s fine, you can encrypt before sending, and decrypt after receiving — but that makes the payload opaque, which makes optimizations (delta compressions, etc.) really difficult.
—
It looks like this service hosts user data encrypted with a single shared key on a (private?) set of IPFS nodes. Those nodes act as the centralized coördinator I describe above.
IPFS is a content-addressable storage system, so updating a document will create a new document with a different address. This kind of punts on all of the data synchronization issues, and means that updates to a document are not observable by consumers of that document, without some additional out-of-band effort. It doesn’t look like devices can be managed individually, or have unique keys. It doesn’t look like the key(s) can be rotated.
In the petnames essay, it says “names can have two out of three properties: decentralized, globally unique, human meaningful.”. Petnames are decentralized and human readable, but not globally unique. Decentralized architectures are a compromise: you gain useful properties, you lose other useful properties.
If I store all my personal data in a third party cloud, then I have a single source of truth (something you want) but I lose properties that are more important to me. First, I have to trust the third party that runs my cloud. Second, I can’t work offline. I want a local-first compute environment that is highly tolerant of being offline. What I am willing to give up is a single source of truth. If I edit documents on offline devices, then they become out of sync with their replicas on my other devices, until synchronization time arrives. If I make changes to the same document on different devices, I may have to resolve conflicts. I’m sure you’re aware that git can support this kind of decentralized workflow.
For a specific example, I have a cellphone, a laptop, and a desktop. They each contain copies of my personal data. I sync data manually between them, but there is no master. I’m looking for something more seamless and automatic, while remaining decentralized. Suppose I’m travelling with my laptop and cellphone. I have no internet or cellular connection. I’d like to be able to seamless sync my laptop and cellphone (via bluetooth or wifi hotspot) without a lot of ceremony. Neither one is the master; new changes on one get propagated to the other. My desktop at home won’t be synced in this situation, since it isn’t reachable. Maybe that explains why I want a decentralized system without a single source of truth.
Anytype.io uses different definitions of “local first”, “decentralized” and “open source” than the ones I was expecting.
I get it. There is no authoritative source of truth for your personal data, there is only the individual view(s) of that data on each device, which are the result of applying all of the updates received by the device. And it should definitely be possible to share updates between devices directly, without going through a centralized coördinator. I agree!
The problem is: how? Specifically?
Are you aware of any mechanism for an iOS application to broadcast an update to a set of trusted peers, and for that update to be reliably received by a commandline application that’s running on a nearby laptop which identifies as one of those peers? And furthermore, that those updates can also be reliably delivered to an Android application which only connects to the internet (or comes close to the laptop or iPhone) some days later?
As far as I’m aware, there is no practical solution to this problem. Bluetooth, Bonjour, etc. just don’t provide the necessary discoverability and interoperability. Even if they did, it’s insufficient — I shouldn’t need to bring my Android physically next to my iPhone for them to have the same view of my data, I should only need to connect them both to the internet. Moreover, if I connect my iPhone to the internet on Monday, whatever updates it has should be received by my Android when I connect it on Tuesday. These kind of table-stakes requirements are only solved with a centralized and stateful coördinator.
This UX is what matters, and UX is bound by the constraints of the underlying technology. All available evidence suggests the best compromise solution for these kind of use cases is to proxy updates through a centralized third-party service over the internet. That sucks, sure! But fixing it requires changes the underlying protocols, and the APIs they provide to relevant devices.
Here’s what I would consider a practical solution for me, and it’s fully decentralized and peer-to-peer. If any two of my devices are both connected to the internet at the same time, then, using an appropriate protocol, they discover each other and synchronize with each other. In practice, my desktop is always running and connected to the internet, so it would normally be always available to act as a synchronization point. For other people, their cellphone is always turned on and connected to the internet, and could serve the same role.
Given appropriately designed software, the UX is simply the requirement to register each new device you get with an existing device in the peer group, so that they all know about each other and will sync with each other. This is not any more difficult than registering each new device with your third party centralized sync service.
I haven’t tried to actually implement this, but I know that suitable protocols exist, and that they can use distributed hash tables to allow peers to find each other without the need for static IP addresses on all peers. Tor Onion Services are an existence proof that this is possible.
I also said I want to sync locally if the internet isn’t available. You say that this isn’t practical, and claim that Bluetooth and Bonjour aren’t good enough. But you don’t convince me that this isn’t possible. Elsewhere on this page, @crazyloglad mentions they are working on “(Arcan) A12 where the constraints are such that DNS is not an option (e.g. transiently airgapped networks) where fleets of user-facing devices still need to dynamically discover and join together to form a network transparent desktop”. If they can figure out how to do that, then why can’t we have software that lets two of my devices discover each other on a LAN and sync? Maybe I should learn more about Arcan.
There’s a lot of complexity hiding behind “an appropriate protocol” :)
Discovery requires some kind of coördination. You can have everyone register with a centralized server, like DNS. Or, you can do some kind of P2P thing, like Tor/Onion. But you can’t avoid seeding each node with an initial set of peers, which you either have to specify manually (laborious, unreliable, bad UX) or — you guessed it — fetch from a centralized server.
(In fact, even if you’re manually specifying the initial set of peers, you’ve almost certainly just copy/pasted those IPs from a website somewhere, which is effectively the same thing as querying a centralized server for them!)
While end user devices can pretty easily make outgoing connections, it’s unfortunately very difficult to get them to reliably accept incoming connections. Most carriers and many ISPs will prevent it outright, and even when they don’t, there’s an infinite universe of complexity in dealing with NAT.
Systems that need end-user devices to be able to accept incoming connections (like Tailscale for example) often solve this problem by having the device make an outgoing connection to a — you guessed it — centralized server, which proxies incoming requests back through that connection.
Oh there are all kinds of mesh network protocols and local network discovery mechanisms, almost uncountably many. The problem to solve here isn’t the technology. The problem to solve is the integration with the device and the user experience. Making it usable.
I say Bluetooth and Bonjour and all current technologies aren’t viable, because (as far as I’m aware) there is no software in existence today which has the synchronization behavior you describe. If it were feasible to do, then I have to believe that someone would already be doing it.
The ‘centralized versus distributed’ is a misdirect or a false dichotomy; there is a substantial difference from DNS as an externally authoritative distributed asynchronously mutating namespace versus internally (yourself) synchronised swarm of cutesy names you call things you like/dislike over ephemeral rendezvous points (coordinates really). Centralised as in pub, not as in tax office. Relatedly, scale isn’t millions of names but the much more manageable tens-to-hundreds – your silicone family.
Unless ‘hung drawn and quartered’ comes back into fashion your are also not geographically distributed (and if it is, well, other problems take priority. You might be in Japan, you might be in Germany or travel between the two but there is a substantial latency and tolerance depending on the time at airport checkin.
For that, pub/subscribe with the “trusted third party” being another instance of you and part of your namespace suffice as petname is transport securing pub-key doubles as state-authenticating signing key. Meet up at the pub at table #5 to coordinate the heist kind of a thing, leave ICANN out of it.
You can punch your swarm of devices as a synch operation (or NFC tap / ubikey for when kinetic energy fails to initiate key transfer) or huddle up before playing ball, You can’t punch DNS as easily as NAT (even though one might like to do that on a daily basis) as others get sad and disagree with your naming of things.
As for the integration side, it is absolutely not OSI layered, hence why few seem to already be doing it: https://www.youtube.com/watch?v=_RSvk7mmiSE (2019)
I’m not totally sure what you’re talking about.
It seems like you’re suggesting that resolving a given identifier could yield different results depending on client identity, locality, or jurisdiction. Is that true? If so, can you give some examples of this kind of identifier?
Something which doesn’t qualify is, say, Google. When I type google.com I need to get the same logical result, regardless of where I am, or what my local pub is, or what I decide is my silicone family. The concept of google.com is invariant to all of those things. Any system which returns a different logical result for google.com based on any of those variables is broken. My local pub does not get to decide what google.com means.
(Different physical results that take advantage of e.g. CDN locality is a different thing, of course.)
The LAN discover case is somewhat easy, depending on threat model. compromised-by-default IoT devices or others on your corporate network building public key databases or leveraging discovery beacons for denial-of-service amplification is a somewhat spicier challenge. mDNS (Bonjour) comes from the era where this wasn’t much of a threat and pragmatism “I need to find my printer, whatever the cost” ruled (and the cryptographic tools needed weren’t exactly for public consumption at the time).
Unless you are the retrocommunications nerd running your network as a token ring or single chain coaxial, chances are kind-of high that the stricter ‘peer-to-peer’ definitions will fail by being bound to a physical layer that has a hub an spokes kind of a topology. There’s still a difference from that as centralisation to one of [FAANG] serving the entire world and forcing that as a design constraint for how the protocol is supposed to scale.
The HCI model is more involved than that (at least from my ‘networked desktop’ perspective) as it is not just about the different device having synchronised access to the same world state, but for them to act as different kinds of sources and sinks. This is why I have a local namespace of petnames (“connection points”) that correspond to access control models that links together with the networked form. Examples would be to connect “this part of my desktop is allowed to manage the clipboard and inject input” to a networked device that I am actually using to generate the input, or “my mobile phone is allowed to act as video source (webcam) on my desktop, but not synch my documents folder”.
For rediscovery on the global internet you will need some form of third party rendezvous, but that does not have to be an 8.8.8.8 like behemoth, but a virtual piconet like master of your very own terror cell. That one can also act as traffic relay or NAT punch, as well as state store / backup. Alas it also opens up the can of worms that is ‘transitive trust’ as the other option being manual management, which is .ssh/id_rsa annoying or the physical form of annoying with having a TPM in your keyring.
A Petname system implies nothing about your database being secure. It’s the underlying names in it that are secure, in the Zooko’s Triangle sense, i.e. no one else can use that name because they don’t know the corresponding private key.
You probably want the database to be secure for the same reason you want your existing contacts db to be secure, but it’s not a requirement of the system. In practice you can do things like add the cryptographic names as custom fields to address cards and keep using iCloud or Google or whatever.
A petnames database hosted for you somewhere requires trusting fewer people and machines. With DNS you delegate trust to the whole unencrypted path between your machine and the authoritative nameserver for whatever domain you want to reach.
You could also encrypt your database and then so long as you trust your software you could trust that your petname host could only do denial of service and rollback attacks rather than just feeding you arbitrary names.
Same for other sources of petnames, if they’re signed then they can be significantly higher trust than DNS.
You can store and synchronize with encryption easily if it’s only between your own devices.
Are we talking iCloud easy or rsync easy? 😉
I imagine what GP is thinking of is more like “Syncthing easy”, but you could even have your petname database sync with iCloud if you really wanted to.
Not if the database is encrypted with a local key or password. Sure you’d have to download it and upload it again every time you need to synchronise it (I’m guessing it could be automated as a background task), but it would prevent any funny business from the provider.
OK, sure, but that just kind of moves the issue of trust away from the DB provider, and into the key manager. That is, any reasonable UX for managing those keys across all of your devices would necessarily need to live at some third-party endpoint, for the same reasons. And now we’re back to the same issue of trust delegation.
Sure, when I use KeepassXC I have to trust KeepassXC. But if the protocol between DB and key management is standard, the DB provider and key manager may be separated, and protect me from the DB provider. At least with this scheme any bug or backdoor from the key manager may be spotted (if it’s Open Source like any security product should be).
Not perfect, but still better.
Or just duplicate your pet names across your devices. I do that with my password database, no online anything involved, and the occasional desync has yet to be a deal breaker.
I appreciate that your password manager doesn’t sync through a third party. I hope it’s not controversial to observe that this is a fairly niche setup, and is really only viable for technically sophisticated users such as yourself.
These edge cases are interesting and useful, but the ultimate test is the general case. It needs to be feasible for grandma to use the thing. Grandma can definitely be convinced to use a password manager, but only if it works across her iMac, tablet, and phone without extraordinary effort. She’s definitely not going to sync a password DB between those devices manually.
The only way to solve this problem, AFAICT, is to delegate that work (and trust) to a reliably accessible third party.
Under Capitalism. Otherwise I could see manufacturers agree on a standard, such that grandma can have one client per device, and one service of her choosing, all independent from each other if she so choose (typically that would be a Microsoft and Apple clients with a Google server). She would have to trust her client (Microsoft and Apple) that her data is not leaked, but the server (Google) will not be able do decrypt anything without her password.
Now syncing is a bit expensive: download the entire DB, merge, upload the whole thing again. Or maybe chop it up in a content addressed manner, but then you’d leak quite a bit of metadata, such as the number of entries. In any case, I can see it working for Grandma, if the big FAANG agree on such a standard even though they probably have no interest to.
The only reason Grandma can’t have that is because this kind of setup is not the default. If it was it would be as easy as navigating Facebook.
Well, two points. One, capitalism won and it’s not going anywhere, so there’s not much point in talking about alternatives, you just gotta treat it as an invariant. Two, I don’t think it’s capitalism that makes this true, I think it’s just that the market has made it clear what a viable UX looks like. Manufacturers won’t agree on a standard unless doing so serves their interests. Decentralization doesn’t really serve anyone’s interest, so there’s no incentive. And why would Google host this opaque data? It can’t do anything with it. Does grandma care if it’s encrypted on the server? No! Grandma cares about product features and capabilities. Encryption is incidental.
Sadly I’m afraid I’ll have to agree with you. Which is why we’re all gonna die.
The problem with capitalism is that it cannot last. Finite boiling world and all. What comes after depends how it phases itself out. Unfortunately I’m anticipating a severe reduction in human population (we’re not all gonna die, but I can easily see most of us dying), and with it disruptive social changes. I don’t see capitalism surviving its own collapse. Or maybe the situation isn’t as dire as I think, and we’ll just continue the cyberpunk route we’re currently taking. Some kind of corporate fascism, if not good old fascism.
The rest of your analysis I also agree with: the incentives just aren’t there. They could make a nice UX for privacy-preserving decentralisation and all that jazz, but I agree they just won’t.
I haven’t lost all hope however. I do believe an alternative to capitalism has a chance. But just thinking that throws me outside the Overton window, and that’s a problem.
This is a really good intro to petnames. I was immediately struck by the utility of this when I first heard of them (20-ish years ago?) and I’ve incorporated them into my perennially-unready Tendril P2P social network.
I quite liked reading this design. My favorite part is that it’s a local-first design, not just in the fact that it supports offline usage first. The design itself is ready for experimentation by individuals. There’s no network protocol, consistency model, etc. to blur the core schema and motivation.
One thing confused me, though. Is the author proposing replacing (or at least augmenting) DNS with petnames? The text suggests a way for domain name registries to depend on widespread petnames adoption:
Does this propose to solve the lookalike domain problem by asking registries to sell domain names in batches? To my mind, this breaks with the spirit of the rest of the design, where individuals are encouraged to think about their trust relationships. Even if we trust registries, this proposal just pushes the lookalike problem into a widely distributed web of registries that will certainly have their own lookalike domain name batches. Even if we converge on a lookalike batching procedure (perhaps using the cited Unicode confusables data), we would almost certainly see huge (10^7) batches for even the shortest domain names.
This proposal is an interesting inversion of the cited Petname Tool, where individuals set their own “edge names.” I wonder why the author chooses a different design. Perhaps the author is gesturing toward the distributed model, where individuals regularly subscribe to well-known lists of edge names. Instead of gesturing toward that, how about a new design on top of petnames that proposes using the Unicode confusables in a browser extension?
Maybe I’m blowing up this aside too much. I also see other people talking about PKI. What’s the connection? Is this some abstraction over some key management and distribution scheme? I’m not sure I got the “secure” part, especially since there’s lots of implied TOFU. For instance, why and how does Dr. Nym trust bizdir?