This is a great guide for techy tinkers like us, but if you want something compatible that friends and family can “just use” and send them a single invite link that “just works” I suggest checking out Snikket.
Time-tested protocol that has demonstrated real-world extensibility to meet the needs of each era as it comes
Community-first with development not primarily funded by a single entity
Lightweight servers and federation that can run on any hardware and over almost any network link
If you have a personal preference for Matrix, it’s a heck of a lot better than many other things you could choose though. Rising tide lifts all boats and all that.
As someone who was involved a lot in XMPP back near the start, most of these points are also downsides:
Time-tested protocol that has demonstrated real-world extensibility to meet the needs of each era as it comes
Unfortunately, the flip side of this is that it’s also prone to fragmentation. The JSF tried to have a reference server[1], but didn’t have a reference client or client library. This led to every client adding features like file transfer and video calling in incompatible ways. This may be better now, but I last tried about five years ago and finding a pair of an iOS and Android client that could do voice calls, send photos, and send rich text messages to each other was hard. And that’s long before you get to things like end-to-end encryption (I think we’re now in the fourth standard for that and, because of the age of the core protocol, it means that you still end up sending a load of metadata in clear text where an intermediate server can record it).
The core protocol was kind-of fine, but then PubSub came along. And then people realised that presence stanzas were a form of pub-sub and that they had all sorts of problems with amplification (presence stanzas are broadcast to everyone. Servers are allowed to drop all except the last one, but a lot of things ended up embedded in presence stanzas and so they got very large). So then personal-eventing over PubSub (PEP) came along and started to move all of these things out of presence. Except that PEP depended on PubSub and none of the servers supported PubSub, so we had years of standards being proposed with no implementations until we were at the point where a client using the modern version of the spec needed to implement 10 different huge XEPs to build the core functionality before it could actually implement any of the ones that exposed this to the user. And none of the servers actually supported any of these. At that point, I stopped paying attention.
Jabber had its chance but managed to completely squander it.
Community-first with development not primarily funded by a single entity
This is what led to the fragmentation in the first place. It’s a community without strong leadership and so everyone implements their own thing. Big companies come along with extensions (e.g. Jingle) and everyone has to jump to implement something compatible because it’s suddenly deployed on a load of devices. Or 5 different XEPs for the same feature, using incompatible mechanisms, end up being proposed and sit as informational XEPs because no one is leading the process of driving consensus towards a single standard.
Lightweight servers and federation that can run on any hardware and over almost any network link
This is true and ejabberd is an impressive engineering achievement. It also supports Matrix now. The clients are a lot more variable .
[1] This was always a bit of a mess. jabberd was fine but then they decided to rewrite it as jabberd2, which broke transports and eventually changed their mind and then decided that ejabberd was the reference implementation (after most people had switched to it anyway), then decided something else was the reference implementation after I stoped paying attention.
It sounds like your main issue is feature fragmentation, something which matrix does not fix (though it’s a bit smaller due to less implementations? But there are still incompatibilities already).
The question I was responding to was specific to my recommendation of Snikket, which is a server+Android app+iOS app and if you use the officially branded apps and server there will of course be no fragmentation.
As a techy tinker I actually quite like the opportunity to try new things, especially when they can be implemented in a backwards compatible way, but agree that for regular users who want “just work” you need a cohesive brand like Snikket.
Well after reading, I am now unclear what’s actually different from XMPP. They even have client compatibility.
Edit: It’s XMPP with guaranteed extensions folded in and an attempt to give those extensions (and some blessed clients) a new name to distinguish it all from the Jabber of old.
I did this for many years. Initially, most of my contacts were on ICQ and I accessed them via a bridge. A couple were on AIM. MSN Messenger took off and a load of them were there too. A few friends used my server and I had a few folks in my roster who were on other Jabber servers. The Google Talk launched and suddenly everyone with a gmail account (well over half of the people I knew) had a native Janber client. For a while, it was great, then Google started silently dropping presence stanzas from federated servers, so I always appeared offline to Google users. Then they ended federation and, overnight, I went to about 3 people I could connect to.
I didn’t jump on any of the proprietary IM services and so I maintained a Jabber server for a few years for no real purpose. Eventually, I gave up and just didn’t use IM for a few years. I tried to use Jabber again for my family, but the state of clients was pretty poor and, much as it was 15 years ago, they all implemented different XEPs and so you only got good interop between users with the same clients (I had to have two Android clients installed, one for being able to compose rich text messages, the other for sending photos with one of the many file transfer protocols, which happened to be the one the iOS client that my partner was using could talk).
A few years ago, I was encouraged to try Signal. I was able to get my parents to install it and my mother now uses it to talk to her 70-year-old neighbour. I don’t get any benefits from a private IM server relative to just using Signal. It has end-to-no encryption, a protocol that ensures that the service operators have no access to most of the metadata that anyone malicious might care about, and a single consistent client experience. Sometimes they do stupid things like decide to bundle a scam payment system, but it has the big advantage that it’s easy for to get people to use it.
They already are - this is a big part of what the EU’s Digital Markets Act (DMA) is about: mandating federation. In particular, I recommend looking at this write-up on the technical standards work:
In short, it looks like the EU may pick something from IETF that supports MLS. Unfortunately, there aren’t any widely-used implementations of MLS, but my personal hope is that MLS support can be added to XMPP and we can then use that.
XMPP is definitely up to the task - it’s just a matter of IETF picking it for this. You probably want to follow MIMI (mentioned there) for how the working group is going. There are summaries of some of their meetings, such as this one from an XMPP community member who was there:
This is a great guide for techy tinkers like us, but if you want something compatible that friends and family can “just use” and send them a single invite link that “just works” I suggest checking out Snikket.
Why this over Matrix and Element?
If you have a personal preference for Matrix, it’s a heck of a lot better than many other things you could choose though. Rising tide lifts all boats and all that.
As someone who was involved a lot in XMPP back near the start, most of these points are also downsides:
Unfortunately, the flip side of this is that it’s also prone to fragmentation. The JSF tried to have a reference server[1], but didn’t have a reference client or client library. This led to every client adding features like file transfer and video calling in incompatible ways. This may be better now, but I last tried about five years ago and finding a pair of an iOS and Android client that could do voice calls, send photos, and send rich text messages to each other was hard. And that’s long before you get to things like end-to-end encryption (I think we’re now in the fourth standard for that and, because of the age of the core protocol, it means that you still end up sending a load of metadata in clear text where an intermediate server can record it).
The core protocol was kind-of fine, but then PubSub came along. And then people realised that presence stanzas were a form of pub-sub and that they had all sorts of problems with amplification (presence stanzas are broadcast to everyone. Servers are allowed to drop all except the last one, but a lot of things ended up embedded in presence stanzas and so they got very large). So then personal-eventing over PubSub (PEP) came along and started to move all of these things out of presence. Except that PEP depended on PubSub and none of the servers supported PubSub, so we had years of standards being proposed with no implementations until we were at the point where a client using the modern version of the spec needed to implement 10 different huge XEPs to build the core functionality before it could actually implement any of the ones that exposed this to the user. And none of the servers actually supported any of these. At that point, I stopped paying attention.
Jabber had its chance but managed to completely squander it.
This is what led to the fragmentation in the first place. It’s a community without strong leadership and so everyone implements their own thing. Big companies come along with extensions (e.g. Jingle) and everyone has to jump to implement something compatible because it’s suddenly deployed on a load of devices. Or 5 different XEPs for the same feature, using incompatible mechanisms, end up being proposed and sit as informational XEPs because no one is leading the process of driving consensus towards a single standard.
This is true and ejabberd is an impressive engineering achievement. It also supports Matrix now. The clients are a lot more variable .
[1] This was always a bit of a mess. jabberd was fine but then they decided to rewrite it as jabberd2, which broke transports and eventually changed their mind and then decided that ejabberd was the reference implementation (after most people had switched to it anyway), then decided something else was the reference implementation after I stoped paying attention.
It sounds like your main issue is feature fragmentation, something which matrix does not fix (though it’s a bit smaller due to less implementations? But there are still incompatibilities already).
The question I was responding to was specific to my recommendation of Snikket, which is a server+Android app+iOS app and if you use the officially branded apps and server there will of course be no fragmentation.
As a techy tinker I actually quite like the opportunity to try new things, especially when they can be implemented in a backwards compatible way, but agree that for regular users who want “just work” you need a cohesive brand like Snikket.
Well after reading, I am now unclear what’s actually different from XMPP. They even have client compatibility.
Edit: It’s XMPP with guaranteed extensions folded in and an attempt to give those extensions (and some blessed clients) a new name to distinguish it all from the Jabber of old.
I did this for many years. Initially, most of my contacts were on ICQ and I accessed them via a bridge. A couple were on AIM. MSN Messenger took off and a load of them were there too. A few friends used my server and I had a few folks in my roster who were on other Jabber servers. The Google Talk launched and suddenly everyone with a gmail account (well over half of the people I knew) had a native Janber client. For a while, it was great, then Google started silently dropping presence stanzas from federated servers, so I always appeared offline to Google users. Then they ended federation and, overnight, I went to about 3 people I could connect to.
I didn’t jump on any of the proprietary IM services and so I maintained a Jabber server for a few years for no real purpose. Eventually, I gave up and just didn’t use IM for a few years. I tried to use Jabber again for my family, but the state of clients was pretty poor and, much as it was 15 years ago, they all implemented different XEPs and so you only got good interop between users with the same clients (I had to have two Android clients installed, one for being able to compose rich text messages, the other for sending photos with one of the many file transfer protocols, which happened to be the one the iOS client that my partner was using could talk).
A few years ago, I was encouraged to try Signal. I was able to get my parents to install it and my mother now uses it to talk to her 70-year-old neighbour. I don’t get any benefits from a private IM server relative to just using Signal. It has end-to-no encryption, a protocol that ensures that the service operators have no access to most of the metadata that anyone malicious might care about, and a single consistent client experience. Sometimes they do stupid things like decide to bundle a scam payment system, but it has the big advantage that it’s easy for to get people to use it.
What would it take for an ActivityPub/Mastodon moment to happen for instant messaging? Twitter had to die for Mastodon to take off.
XMPP is up to the task. Regulation, maybe?
Do you really perceive Twitter as dead?
Yes it is pretty much completely dead to me. My circle of leftist, gay, computery people have found a cozy new home in the Fediverse.
They’re also literally auctioning off the office furniture right now. It doesn’t look good.
Maybe the EU could do it. I wonder if they would, when you consider that most of the various chat service companies are US?
They already are - this is a big part of what the EU’s Digital Markets Act (DMA) is about: mandating federation. In particular, I recommend looking at this write-up on the technical standards work:
https://openforumeurope.org/wp-content/uploads/2022/11/Ian_Brown_Private_Messaging_Interoperability_In_The_EU_DMA.pdf#page=32
In short, it looks like the EU may pick something from IETF that supports MLS. Unfortunately, there aren’t any widely-used implementations of MLS, but my personal hope is that MLS support can be added to XMPP and we can then use that.
XMPP is definitely up to the task - it’s just a matter of IETF picking it for this. You probably want to follow MIMI (mentioned there) for how the working group is going. There are summaries of some of their meetings, such as this one from an XMPP community member who was there:
https://mail.jabber.org/pipermail/members/2022-November/009472.html