1. 31
  1.  

  2. 34

    I don’t really agree with a lot of the claims in the article (and I say this as someone who was very actively involved with XMPP when it was going through the IETF process and who wrote two clients and continued to use it actively until 2014 or so):

    Truly Decentralized and Federated (meaning people from different servers can talk to each other while no central authority can have influence on another server unlike Matrix)

    This is true. It also means that you need to do server reputation things if your server is public and you don’t want spam (well, it did for a while - now no one uses XMPP so no one bothers spamming the network). XMPP, unlike email, validates that a message really comes from the originating domain, but that doesn’t stop spammers from registering millions of domains and sending spam from any of them. Google turned off federation because of spam and the core problems remain unsolved.

    End-To-End Encryption (unlike Telegram, unless you’re using secret chats)

    This is completely untrue for the core protocol. End-to-end encryption is (as is typical in the XMPP world) multiple, incompatible, extensions to the core protocol and most clients don’t support any of them. Looking at the list of clients almost none of them support the end-to-end encryption XEP that the article recommends. I’d not looked at XEP-0384 before, but a few things spring to mind:

    • It’s not encrypting any metadata (i.e. the stuff that the NSA thinks is the most valuable bit to intercept), this is visible to the operators of both party’s servers.
    • You can’t encrypt presence stanzas (so anything in your status message is plaintext) without breaking the core protocol.
    • Most info-query stanzas will need to be plain-text as well, so this only affects direct messages, but some client-to-client communication is via pub-sub. This is not necessarily encrypted and clients may or may not expose which things are and aren’t encrypted to the user.
    • The bootstrapping thing involves asking people to trust new fingerprints that exist. This is a security-usability disaster: users will click ‘yes’. Signal does a good job of ensuring that fingerprints don’t change across devices and manages key exchange between clients so that all clients can decrypt a message encrypted with a key assigned to a stable identity. OMEMO requires a wrapped key for every client.
    • The only protection against MITM attacks is the user noticing that a fingerprint has changed. If you don’t validate fingerprints out-of-band (again, Signal gives you a nice mechanism for doing this with a QR code that you can scan on the other person’s phone if you see them in person) then a malicious server can just advertise a new fingerprint once and now you will encrypt all messages with a key that it can decrypt.
    • There’s no revocation story in the case of the above. If a malicious fingerprint is added, you can remove it from the advertised set, but there’s no guarantee that clients will stop sending things encrypted with it.
    • The XEP says that forward secrecy is a requirement and then doesn’t mention it again at all.
    • There’s no sequence counter or equivalent so a server can drop messages without your being aware (or can reorder them, or can send the same message twice - no protection against replay attacks, so if you can make someone send a ‘yes it’s fine’ message once then you can send it in response to a request to a different question).
    • There’s no padding, so message length (which provides a lot of information) is available.

    This is without digging into the protocol. I’d love to read @soatok’s take on it. From a quick skim, my view is that it’s probably fine if your threat model is bored teenagers.

    They recommend looking for servers that support HTTP upload, but this means any file you transfer is stored in plain text on the server.

    Cross-Platform Applications (Desktop, Web, and Mobile)

    True, with the caveat that they have different feature sets. For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.

    Multi-Device Synchronization (available on some servers)

    This, at least, is fairly mature. There are some interesting interactions between it and the security gurantees claimed by OMEMO.

    Voice and Video Calling (available on most servers)

    Servers are the easy part (mostly they do STUN or fall back to relaying if they need to). There are multiple incompatible standards for voice and video calling on top of XMPP. The most widely supported is Jingle which is, in truly fractal fashion, a family of incompatible standards for establishing streams between clients and negotiating a CODEC that both support. It sounds as if clients can now do encrypted Jingle sessions from their article. This didn’t work at all last time I tried, but maybe clients have improved since then.

    1. 8

      Strongly agree – claiming that XMPP is secure and/or private without mentioning all the caveats is surprising! There’s also this article from infosec-handbook.eu outlining some of the downsides: XMPP: Admin-in-the-middle

      The state of XMPP security is a strong argument against decentralization in messengers, in my opinion.

      1. 7

        Spam in XMPP is largely a solved problem today. Operators of open relays, servers where anyone can create an account, police themselves and each other. Anyone running a server that originates spam without dealing with it gets booted off the open federation eventually.

        Another part of the solution is ensuring smaller server operators don’t act as open relays, but instead use invites (like Lobste.rs itself). Snikket is a great example of that.

        but that doesn’t stop spammers from registering millions of domains and sending spam from any of them.

        Bold claim. Citation needed. Where do you register millions of domains cheaply enough for the economics of spam to work out?

        Domains tend to be relatively expensive and are easy to block, just like the IP addresses running any such servers. All I hear from server operators is that spammers slowly register lots of normal accounts on public servers with open registration, which are then used once for spam campaigns. They tend to be deleted by proactive operators, if not before, at least after they are used for spam.

        Google turned off federation because of spam and the core problems remain unsolved.

        That’s what they claim. Does it really seem plausible that Google could not manage spam? It’s not like they have any experience from another federated communications network… Easier for me to believe that there wasn’t much in the way of promotion to be gained from doing anything more with GTalk, so they shut it down and blamed whatever they couldn’t be bothered dealing with at the time.

        1. 3

          Your reasonning about most clients not supporting OMEMO is invalid because noone cares about most clients: it’s all about the marketshare. Most XMPP clients probably don’t support images but that doesn’t matter.

          For replays, this may be dealt with the double ratchet algorithm since the keys change fairly often. Your unknown replay would also have to make sense in an unknown conversation.

          Forward secrecy could be done with the double ratchet algorithm too.

          Overall OMEMO should be very similar to Signal’s protocol, which means that it’s quite likely the features and flaws of one are in the other.

          Conversations on Android also offers showing and scanning QR codes for validation.

          As for HTTP upload, that’s maybe another XEP but there’s encrypted upload with an AES key and a link using the aesgcm:// scheme (as you can guess: where to retrieve the file plus the key).

          I concur that bootstrapping is often painful. I’m not sure it’s possible to do much better without a centralized system however.

          Finally, self-hosting leads to leaking quite a lot of metadata because your network activity is not hidden in large amounts of network activity coming from others. I’m not sure that there’s really much more that is available by reading the XMPP metadata. Battery saving on mobile means the device needs to tell the server that it doesn’t care about status messages and presence from others but who cares if it’s unencrypted to the server (on the wire, there’s TLS) since a) it’s meant for the server, b) even if for clients instead, you could easily spot the change in network traffic frequency. I mean, I’m not sure there’s a lot more that is accessible that way (not even mentionning that if you’re privacy-minded, you avoid stuff like typing notifications and if you don’t, traffic patterns probably leak that anyway). And I’m fairly sure that’s the same with Signal for many of these.

          1. 3

            now no one uses XMPP so no one bothers spamming the network

            I guess you’ve been away for awhile :) there is definitely spam, and we have several community groups working hard to combat it (and trying to avoid the mistakes of email, not doing server/ip rep and blocking and all that)

            1. 3
              Cross-Platform Applications (Desktop, Web, and Mobile)
              

              True, with the caveat that they have different feature sets. For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.

              Or they’ve also calcified (see: Pidgin). Last time I tried XMPP a few years ago, Conversations on Android was the only tolerable one, and Gajim was janky as hell normally, let alone on Windows.

              1. 3

                True, with the caveat that they have different feature sets. For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.

                This was the reason I couldn’t get on with XMPP. When I tried it a few years ago, you really needed quite a lot of extensions to make a good replacement for something like WhatsApp, but all of the different servers and clients supported different subsets of the features.

                1. 3

                  I don’t know enough about all the details of XMPP to pass technical judgement, but the main problems never were the technical decisions like XML or not.

                  XMPP had a chance, 10-15 years ago, but either because of poor messaging (pun not intended) or not enough guided activism the XEP thing completely backfired and no two parties really had a proper interaction with all parts working. XMPP wanted to do too much and be too flexible. Even people who wanted it to succeed and run their own server and championed for use in the companies they worked for… it was simply a big mess. And then the mobile disaster with undelivered messages to several clients (originally a feature) and apps using up to much battery, etc.pp.

                  Jitsi also came a few years too late, sadly, and wasn’t exactly user friendly either at the start. (Good people though, they really tried).

                  1. 5

                    I don’t know enough about all the details of XMPP to pass technical judgement, but the main problems never were the technical decisions like XML or not.

                    XML was a problem early on because it made the protocol very verbose. Back when I started working on XMPP, I had a £10/month plan for my phone that came with 40 MB of data per month. A few extra bytes per message added up a lot. A plain text ‘hi’ in XMPP was well over a hundred bytes, with proprietary messengers it was closer to 10-20 bytes. That much protocol overhead is completely irrelevant now that phone plans measure their data allowances in GB and that folks send images in messages (though the requirement to base64-encode images if you’re using in-band bytestreams and not Jingle still matters) but back then it was incredibly important.

                    XMPP was also difficult to integrate with push notifications. It was built on the assumption that you’d keep the connection open, whereas modern push notifications expect a single entity in the phone to poll a global notification source periodically and then prod other apps to make shorter-lived connections. XMPP requires a full roster sync on each connection, so will send a couple of megs of data if you’ve got a moderately large contact list (first download and sync the roster, then get a presence stanza back from everyone once you’re connected). The vcard-based avatar mechanism meant that every presence stanza contained the base64-encoded hash of the current avatar, even if the client didn’t care, which made this worse.

                    A lot of these problems could have been solved by moving to a PubSub-based mechanism, but PubSub and Personal Eventing over PubSub (PEP) weren’t standardised for years and were incredibly complex (much more complex than the core spec) and so took even longer to get consistent implementations.

                    The main lessons I learned from XMPP were:

                    • Federation is not a goal. Avoiding having an untrusted admin being able to intercept / modify my messages is a goal, federation is potentially a technique to limit that.
                    • The client and server must have a single reference implementation that supports anything that is even close to standards track, ideally two. If you want to propose a new extension then you must implement it at least once.
                    • Most users don’t know the difference between a client, a protocol, and a service. They will conflate them, they don’t care about XMPP, they care about Psi or Pidgin - if the experience isn’t good with whatever client you recommend that’s the end.
                    1. 2

                      XMPP requires a full roster sync on each connection, so will send a couple of megs of data if you’ve got a moderately large contact list (first download and sync the roster, then get a presence stanza back from everyone once you’re connected).

                      This is not accurate. Roster versioning, which means that only roster deltas, which are seldom, are transferred, is used widely and also specified in RFC 6121 (even though, not mandatory to implement, but given that it’s easy to implement, I am not aware of any mobile client that doesn’t use it)

                      1. 1

                        Also important to remember that with smacks people are rarely fully disconnected and doing a resync.

                        Also, the roster itself is fully optional. I consider it one of the selling points and would not use it for IM without, but nothing prevents you.

                        1. 1

                          Correct.

                          I want to add that, it may be a good idea to avoid using XMPP jargon to make the test more accessible to a wider audience. Here ‘smacks’ stands for XEP-198: Stream Management.

                    2. 2

                      XMPP had a chance, 10-15 years ago, but either because of poor messaging (pun not intended) or not enough guided activism the XEP thing completely backfired and no two parties really had a proper interaction with all parts working. XMPP wanted to do too much and be too flexible.

                      I’d argue there is at least one other reason. XMPP on smartohones was really bad for a very long time, also due to limitations on those platforms. This only got better later. For this reason having proper messaging used to require spending money.

                      Nowadays so you “only” need is too pay a fee to put stuff into the app store and in case of iOS development buy an overpriced piece of hardware to develop on. Oh and of course deal with a horrible experience there and be at the risk of your app being banned from the store, when they feel like. But I’m drifting off. In short: Doing what the Conversation does used to be harder/impossible on both Android and iOS until certain APIs were added.

                      I think that gave it a pretty big downturn when it started to do okay on the desktop.

                      I agree with the rest though.

                    3. 2

                      I saw a lot of those same issues in the article. Most people don’t realize (myself included until a few weeks ago) that when you stand up Matrix, it still uses matrix.org’s keyserver. I know a few admins who are considering standing up their own keyservers and what that would entail.

                      And the encryption thing too. I remember OTR back in the day (which was terrible) and now we have OMEMO (which is ….. still terrible).

                      This is a great reply. You really detailed a lot of problems with the article and also provided a lot of information about XMPP. Thanks for this.

                      1. 2

                        It’s not encrypting any metadata (i.e. the stuff that the NSA thinks is the most valuable bit to intercept), this is visible to the operators of both party’s servers. You can’t encrypt presence stanzas (so anything in your status message is plaintext) without breaking the core protocol.

                        Do you know if this situation is any better on Matrix? Completely honest question (I use both and run servers for both). Naively it seems to me that at least some important metadata needs to be unencrypted in order to route messages, but maybe they’re doing something clever?

                        1. 3

                          I haven’t looked at Matrix but it’s typically a problem with any Federated system: you need at least an envelope that tells you the server that a message needs to be routed to to be public. Signal avoids this by not having federation and by using their sealed-sender mechanism to avoid the single centralised component from knowing who the sender of a message is.

                          1. 1

                            Thanks.

                          2. 1

                            There is a bit of metadata leaking in matrix, because of federation. But it’s something the team is working to improve.

                          3. 2

                            Fellow active XMPP developer here.

                            I am sure you know that some of your points, like Metadata encryption, are a deliberate design tradeoff. Systems that provide full metadata encryption have other drawbacks. Other “issues” you mention to be generic and apply to most (all?) cryptographic systems. I am not sure why XEP-0384 needs to mention forward secrecy again, given that forward secrecy is provided by the building blocks the XEP uses and discussed there, i.e., https://www.signal.org/docs/specifications/x3dh/. Some points of yous are also outdated and no longer correct. For example, since the newest version of XEP-0384 uses XEP-0420, there is now padding to disguise the actual message length (XEP-0420 borrows this again from XEP-0373: OpenPGP for XMPP).

                            From a quick skim, my view is that it’s probably fine if your threat model is bored teenagers.

                            That makes it sound like your threat model shouldn’t be bored teenagers. But I believe that we should also raise the floor for encryption so that everyone is able to use a sufficiently secured connection. Of course, this does not mean that raising the ceiling shouldn’t be researched and tried also. But we, that is, the XMPP community of volunteers and unpaid spare time developers, don’t have the resources to accomplish everything in one strike. And, as I said before, if you need full metadata encryption, e.g., because you are a journalist in a suppressive regime, then the currently deployed encryption solutions in XMPP are probably not what you want to use. But for my friends, my family, and me, it’s perfectly fine.

                            They recommend looking for servers that support HTTP upload, but this means any file you transfer is stored in plain text on the server.

                            That depends on the server configuration, doesn’t it? I imagine at least some servers use disk or filesystem-level encryption for user-data storage.

                            For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.

                            It got better. But yes, this is the price we pay for the modularity of XMPP due to its extensibility. I also believe it isn’t possible to have it any other way. Unlike other competitors, most XMPP developers are not “controlled” by a central entity, so they are free to implement what they believe is best for their project. But there is also a strong incentive to implement extensions that the leading Implementations support for compatibility. So there are some checks and balances in the system.

                          4. 3

                            What’s centralized about Matrix compared to XMPP?

                            1. 8

                              In principle, nothing. In practice:

                              • 35% of all users on the network are on a single homeserver (matrix.org).
                              • Most people who are not on the matrix.org homeserver are still using their identity server (maps matrix IDs to other IDs like email addresses or phone numbers).
                              • Nearly everyone is using the matrix.org integration server (provides room widgets like stickers and video calls).

                              Beyond that, the Matrix specification is largely “whatever Synapse and Element do”, so there are effectively no compatible third-party servers, and third-party clients vary in their support, with relatively few providing end-to-end encryption and none that I know of supporting “spaces”.

                              Obviously, XMPP has a lot of compatibility issues too (it can be a crapshoot whether your client and your server both support the XEPs needed for the features you want). But the cause is different.

                              1. 5

                                I think that’s fairly ok. Everything is open source and the system actually managed to take off at bigger installations and corporations. Move fast and break things is ok for something that is still going this much. It’s kinda the same with rustc and the calls for a spec.

                                I’d say most people don’t actually want federation, at least my workplace doesn’t. They either use the big system where everyone is, or they want their private space formed for a specific organization. Although element really does lack a multi-account feature to actually be able to use this kind of operation.

                                1. 4

                                  most people don’t actually want federation

                                  To be honest, I don’t think too many really want federation and the problems it brings w/ an open world. I suspect people want more single signon and a single client, which can be achieved via SSO/APIs.

                                  1. 3

                                    I’d say most people don’t actually want federation

                                    Maybe not, but most people don’t want a single point of failure.

                                    1. 2

                                      shrug

                                      It sounds like you don’t need or want decentralized chat, just on-premises hosted chat. But a lot of people do, I think.

                                      1. 6

                                        First of all I want something that works for everyone and isn’t whatsapp. I already have over 5 messengers installed, so I’d much rather have something that has actually more users than a cargo cult of nerds.

                                        Then I’d like to host some instances for myself and others. But I can already find solutions for the second part at various niche software solutions, which won’t actually work for anyone else than me and the other people here on lobste.rs. If convenience, features and ease of use weren’t a thing I wouldn’t have so many contacts on telegram, which is behind whatsapp and threema in regards to E2E. But it has much more (contact related) privacy, notification and group settings, has anonymous surveys, an official bot api, doesn’t require you to pass a phone number, can handle more files and sizes, has better preview integration, bigger groups, more admin features and a much cleaner desktop experience.

                                        So yes please, first break up the market and deliver the required features, then you can concentrate on a specification set in stone. For now there are already some very useful 3rd party clients for matrix.

                                        Edit: And all of these easy solutions require someone to pay for these space and bandwidth monstrosities that are modern chat solutions with files, photos, full text search, gifs, user (animated) stickers, push notifications, voice messages and possibly video chat. So I won’t hold any grudge against matrix if they first try to get up in the marked and handle their clients needs, which surely isn’t federation or a full fledged specification for the army or government instances.

                                        I don’t even know how they want to make money to keep all of this running, while all the people are used to paying “nothing” in whatsapp, signal, telegram & co.

                                        We’ve got a matrix instance at the institution. I don’t need want a “job” for my day to day communication, which will ping me at any point in time when it goes down. Because now it’s bound to you personally and not your job as your family/friends/.. can’t connect anymore.

                                        1. 5

                                          I completely agree. Federation was pushed as a goal with XMPP, without stepping back and understanding the problem that it was trying to solve. I want to be able to run a chat app and talk to other people without some other entity being able to:

                                          • Read my messages.
                                          • Delete my account.
                                          • Tamper with my messages.

                                          Running my own XMPP server gives me this guarantee. Someone else running their own server that is federated with mine gives them this guarantee. If the person using the server is not running the server then some of this breaks down: if their account is tied to a domain that they don’t own, then it’s possible for the server admin to unilaterally kick them off. The server admin can always delete their messages and without end-to-end encryption can read or tamper with them. With XMPP’s end-to-end encryption, the admin can still drop messages, record all metadata for their conversations, and launch replay attacks (send the same message twice).

                                          Signal gives me all of these guarantees, plus the really important one: A single app that I can tell my mother to install and that works for her out of the box with no configuration. Threema has some questionable security claims and spews FUD, Telegram is a semi-proprietary version of Signal without some of the newer security features.

                                          As a colleague of mine said a couple of years ago when I did a survey of open source messaging solutions, the correct answer is ‘Just use Signal’. It’s easy to use, provides user-friendly features like reactions, stickers (I don’t actually know what these are, but apparently people care about them), in-band picture messaging, and encrypted voice / video chat, and has mobile and desktop apps.

                                          Signal has a single entity running it, but they’re a registered charity, they’re well funded by donations (and the cost of running the service is really small per user at scale - WhatsApp was charging $1/year and was very profitable), and even if they were evil the protocol is designs so that they don’t even see most of the metadata, let alone the content.

                                    2. 2

                                      35% of all users on the network are on a single homeserver (matrix.org).

                                      Is it really only 35%? Basically everyone I talked to who tried setting up their own home server gave up on it, so I wonder if the reality is more like … 35% of accounts created, but not 35% of accounts in active use? Of the matrix users I see in our bridged Libera channel, well over half are on matrix.org.

                                      1. 3

                                        Well, new vector runs a hosted homeserver thing now, so some people are “not on matrix.org” but still on new vector infrastructure of course

                                        1. 3

                                          Basically everyone I talked to who tried setting up their own home server gave up on it

                                          That sounds really weird to me. I set up my own home server ages ago, back when it was harder. Now there are ansible playbooks https://github.com/spantaleev/matrix-docker-ansible-deploy/ and tutorials https://matrix.org/docs/guides/free-small-matrix-server/

                                          1. 3

                                            Anecdotally, I would expect it to be higher. I read an article last week that gave the 35% figure, which surprised me at the time, but I don’t remember their methodology.

                                          2. 1

                                            The identity server also serves out keys, right?

                                            1. 2

                                              I do not think so, but I could be wrong. I think your own homeserver does key backup.

                                        2. 3

                                          Yes, I’m glad to see XMPP being revitalized!

                                          I have more confidence in the longevity in messaging approaches that have ecosystems with multiple participants, rather than messaging approaches that rely on BDFL.

                                          I use XMPP clients Quicksy on Android and Dino on Linux to chat to my partner and some friends.

                                          1. 2

                                            Is there a reliable way to bridge Matrix and XMPP so that a matrix user can talk to an XMPP user and vice versa? I see that there will be downsides in a way as only the common subset of both protocols would be supportable, but it would still be nice.

                                            1. 4

                                              There’s this: https://github.com/matrix-org/matrix-bifrost

                                              I got it running but never got it to connect correctly with my Matrix server and kinda gave up. I’m not sure if it works.

                                              1. 3

                                                It has bugs, and for obvious reasons new vector isn’t putting much into fixing them, but for some use cases it works. The instance on aria-net is the most stable and has many bug fixes not in upstream

                                                1. 3

                                                  Thank you! Looks intimidating. I’m not sure I’d really want to run this on my small XMPP server…

                                                  1. 2

                                                    No reason you need to run anything matrix related. Just connect to addresses that go via the aria-net or matrix.org instance. Running it yourself gains you nothing since you’ll just be using it to send messages to matrix.org users anyway.

                                                    1. 1

                                                      You mean, as in “it just works”? Looking it up in DNS I am entirely surprised indeed:

                                                      $ dig _xmpp-server._tcp.matrix.org SRV
                                                      
                                                      ; <<>> DiG 9.16.15-Debian <<>> _xmpp-server._tcp.matrix.org SRV
                                                      ;; global options: +cmd
                                                      ;; Got answer:
                                                      ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6254
                                                      ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
                                                      
                                                      ;; OPT PSEUDOSECTION:
                                                      ; EDNS: version: 0, flags:; udp: 512
                                                      ;; QUESTION SECTION:
                                                      ;_xmpp-server._tcp.matrix.org.	IN	SRV
                                                      
                                                      ;; ANSWER SECTION:
                                                      _xmpp-server._tcp.matrix.org. 300 IN	SRV	0 5 5269 lethe.matrix.org.
                                                      
                                                      ;; Query time: 48 msec
                                                      ;; SERVER: 192.168.1.1#53(192.168.1.1)
                                                      ;; WHEN: Fr Nov 19 17:55:31 CET 2021
                                                      ;; MSG SIZE  rcvd: 93
                                                      

                                                      I have not tested if there is actually something listening on port 5269 on lethe.matrix.org, but if it is, it indeed just ought to work. That’s… interesting. So I can assume matrix.org Matrix users will just be reachable via XMPP without further setup? That’s kind of cool. I have no Matrix user in my XMPP roster yet, so I was unaware of this and just assumed the two universes are entirely incompatible to one another. Thanks for rectifying this. I will remember that the next time I talk to someone about XMPP vs. Matrix.

                                                      1. 1

                                                        https://github.com/matrix-org/matrix-bifrost/wiki/Address-syntax for the syntax

                                                        You are likely to have a better experience with this one https://aria-net.org/SitePages/Portal/Bridges.aspx though

                                                        1. 1

                                                          Thanks!