Threads for arathorn

  1. 5

    I’m a little worried about the constant focus on features when the basic experience of just text chats and such can be really rough, in terms of inconsistency and client/server performance. It needs more polish, but what attention is there to give when you’re on the next big thing?

    1. 7

      literally the whole point of the original post is to spell out the emphasis we’re putting on perf and usability atm. Only the second half of the post talks about new features (which happens in entirely different teams)

    1. 9

      I’ve really enjoyed using Matrix, hopefully when sync is faster I can lure more of my friends to use it.

      1. 3

        meh…Startup (initial connection delay) is considerably slower that IRC. It’s a bit of a downgrade in terms of overall polish of clients too (but again, IRC is soo much bare-bones so it’s simpler by design)

        1. 5

          IRC is nice but I prefer Matrix over it due to the ease of use: I got my fiancee to use Matrix with me on my homeserver, but I doubt i would’ve ever got her use IRC. :)

          1. 2

            I got my parents to use IRC in early 2000s. It wasn’t that difficult, just installed an irc client that automatically connects to a server and channel.

            Matrix would be a hard sell now that Telegram and Whatsapp exist.

            1. 8

              the point of the OP is that Matrix clients have to be better than TG or WhatsApp to win, and that’snwhat we’re aiming for.

              1. 4

                In my experience using Matrix for the last several years, most Matrix clients seem to be struggling to keep up in terms of features/functionality. Which is unfortunate, because the official clients that are web/browser based are slow and frustrating to use on older devices. That said, the Android Element.io app is not too bad :)

                I think supporting multiple (unofficial…) clients is very important, since it prevents “vendor lock in”, which can totally happen even when the protocol is federated if everyone ends up depending on the official client and (hypothetically, but not totally impossible…) it is sold/acquired by some nefarious company in the future… I don’t know if the current situation is from Matrix being some quickly moving target, or if implementing the features is just… hard. In any case, it’s not great having such limited options for clients.

                1. 5

                  I wouldn’t say that “most” matrix clients are struggling to keep up in features/functionality - it’s more that we’re still figuring out some features (as per the OP) and everyone (including Element) is playing catchup to a fast moving target.

                  Totally agreed that vendor lockin is a total antigoal. Element is not the “official” Matrix client - it’s just a client, like Netscape was just a browser in the early days of the web. It happens to be written by the team who created Matrix, but the two are separate these days: matrix.org/foundation v element.io/careers.

                  In terms of native Desktop apps, we’re hoping matrix-rust-sdk will power a new generation of excellent native apps - ElementX iOS supports macOS too, for instance, and Fractal-next is already a GTK app based on rust-sdk.

                  1. 2

                    and Fractal-next is already a GTK app based on rust-sdk.

                    Yeah… but that doesn’t even support E2EE[1]. I consider that to be a major feature of Matrix, without it Matrix is just a slower way to exchange unencrypted text online. Last time I looked, a few Matrix clients were struggling to implement E2EE.

                    1. https://gitlab.gnome.org/GNOME/fractal/-/issues/717
                    1. 2

                      If you look at the checkboxes on that bug, all the hard bits are already done (thanks to leaning on matrix-rust-sdk). You can literally use Fractal-next for E2EE today. They just need to hook up UI for the remaining edge cases (eg key backups). In terms if why that hasn’t happened yet… it’s a FOSS project; PRs welcome.

                    2. 1

                      Just want to give shoutout to Nheko! I use it daily on my desktop.

        1. 4

          While the way to go forward is interesting, and Matrix is an excellent piece of engineering, it lacks in several areas still, which I would have expected to work flawlessly by now. This is from my own experimenting with the protocol and surrounding server/clients:

          • 1:1 calling is lacking, TURN appears to rarely work. Calling from mobile (Android) to Desktop and reverse works <50% of the time
          • If you’re logged into both mobile (Android) and desktop, an incoming call picked up by Desktop will continue ringing on the mobile, until it drains the battery. The call pickup even is aparently not properly handled when mobile lockscreen is active.
          • Recently, my mobile notifications and message history on mobile is completely off. It appears to be “receiving” messages minutes after they appear on my desktop. Comparing mobile and desktop message history reveals completely out-of-order messages, or simply not-received-on-mobile ones

          It feels to me that every time the core features are stable, at least on mobile, something happens in the matrix world that triggers new features that replace the stable features with something new (often incomplete) leaving core functionality flawed. It happened with Riot -> RiotX, and it seems to happen again with Element -> Element X. This is one of the reasons I’ve stopped reporting issues with mobile clients, as there’s no point, a “new” mobile client will most likely soon appear and the cycle will start again.

          1. 6

            Agreed that RiotX on Android still has some major bugs (which is frustrating, given how long it’s been around). ElementX on Android will likely not be a rewrite however: it is “just” replacing the Kotlin SDK with the Rust SDK, and replacing the calling implementation with Element Call (so we get native conferencing as well as 1:1) - which should address both issues you’ve mentioned here.

            1. 1

              Will the voice/video call functionality be part of the Rust SDK? How are you implementing WebRTC on the mobile platforms, where presumably running a browser engine’s WebRTC stack inside a web view wouldn’t be a good solution? Are you using the Google WebRTC C++ library directly, do you have some kind of wrapper over it, or are you using a different WebRTC implementation?

              1. 4

                currently the plan is to run webrtc inside a webview, as per the matryoshka section of the OP. the current mobile apps use libwebrtc directly, with limited success (as you can see from the original comment here), so given we’re switching to multiway native Matrix VoIP the idea is to switch to embedding the Element Call webapp, and then replace that with native impls only if performance actually requires it. So far, webrtc in a webview is actually working fine, and avoids us having to build a new native impl of the relatively complex multiway calling on each platform.

                Alternatively, others are very welcome to go wild with libwebrtc or webrtc-rs on top of matrix-rust-sdk or others. After all, it’s all open…

            2. 4

              The slow receiving messages on mobile will probably be fixed by the new sync. Mobile clients don’t currently keep syncing all the time in order to save battery. When a push notification arrives, the mobile client needs to sync in order to receive all the relevant data. That takes a long time and lots of data on sync v2. Exactly what sync v3 is supposed to fix.

            1. 11

              These days, I am unsure what Matrix is heading for. This post explains that they want to have VoIP video conferencing and decentralised virtual reality. Then I open the lobste.rs comments and the first thing I see is a comparison to IRC.

              It seems as if Matrix’ mission statement today is going far beyond the goal to open up walled text message gardens. From this post it looks as if they want to make Matrix a decentralised platform for everything. The post talks explicitely about the success of the open web and how Matrix strives to copy it, and that makes me think: don’t we already have the open web? It’s built on a protocol called HTTP. Does this mean Matrix wants to replace HTTP?

              If Matrix is indeed inferior even to IRC (I cannot judge as I do not use Matrix) in the domain IRC occupies (text messaging), such a wide approach seems doomed.

              1. 10

                We’ve always tried to be clear that Matrix is a general purpose comms protocol for realtime data - not just chat. For instance, right from the original launch in Sept 2014 we had VoIP signalling in there too, and did a very basic demo of 3D over Matrix on day 1 too: https://techcrunch.com/video/animatrix-presents-disrupt-sf-2014-hackathon/

                The post talks explicitely about the success of the open web and how Matrix strives to copy it, and that makes me think: don’t we already have the open web? It’s built on a protocol called HTTP. Does this mean Matrix wants to replace HTTP?

                Obviously we’re not trying to replace HTTP. Matrix is an API layered on top of HTTP (or other transports) to provide a communication layer for realtime data. If anything it competes with ActivityStreams as a way to link streams of activity over the open web - except with a completely different architecture. The reason for invoking the open web is that we simply want to be the comms layer for the open web: a global realtime virtual space where folks can chat, talk, interact, and publish/subscribe to realtime data of any kind.

                W3C simply doesn’t provide an API for that, yet - and if they did, hopefully it might be Matrix.

                1. 4

                  The open web is not a federated eventually consistent database. That’s what matrix provides. https://matrix.org/ for more info. An update for people following the blog doesn’t cover the introduction.

                  Text chat is the first application, but matrix can be used for much more.

                1. 4

                  https://matrix.org/blog/2022/08/15/the-matrix-summer-special-2022#wysiwyg

                  However, given that users are now used to WYSIWYG in Teams and Slack, we’ve now decided to have another go at it

                  You link to a blog post that shows issues with Slack WYSIWYG, but creating richly formatted messages in MS Teams is also an incredible daily source of frustrations, with no way to opt-out and just use Markdown without that awful dynamic interpolation. I hope just writing Markdown to format messages will still be an option in Element when/if this WYSIWYG editor is introduced.

                  1. 4

                    don’t worry - markdown will not be going away. this is just adding wysiwyg as an option for those who want it (and for parity with Teams)

                  1. 3

                    Thank you for writing this up, there’s an excellent amount of detail here. The proliferation of silo’ed chat protocols has been one of my pet peeves and has definitely (thus far) been heading the wrong direction; as bad as the proprietary protocols were back in the aughts, they were at least neutrally interoperable in their heyday – these days most companies are downright hostile in how their enforce their ToS when it comes to third-party clients etc.

                    I’m hoping the relatively smaller ecosystems (e.g. Discord) take note and at least loosen their ToS to allow for calling user APIs without fear of a perma-ban.

                    A couple of additional questions: is it known if companies might attempt to limit the exposure of their APIs to EU markets only, or does the DMA cover that explicitly? Is the DMA a pre-requisite for fully scaling out use of Matrix Bridging Services – i.e. does the interoperability climate pre-DMA preclude you from offering bridging as a commercial service?

                    1. 3

                      Thanks for the positive feedback :)

                      is it known if companies might attempt to limit the exposure of their APIs to EU markets only, or does the DMA cover that explicitly?

                      I don’t believe the DMA covers that explicitly, but IANAL. Much like some sites decided to cut off EU traffic rather than implement GDPR, I guess it’s possible that the gatekeepers might only offer open APIs to EU IP addresses - but it feels like the negative PR of doing so (and the theatre of doing so, given how easy it is to get a EU IP) would not be worth it.

                      Is the DMA a pre-requisite for fully scaling out use of Matrix Bridging Services – i.e. does the interoperability climate pre-DMA preclude you from offering bridging as a commercial service?

                      Any kind of bridging to a closed service from Matrix (or XMPP) is pretty miserable today, given you have to do adversarial interoperability, which massively reduces the interest in building bridges or relying on them. So yes, DMA would be transformative for bridging and interop in general :)

                      1. 1

                        So yes, DMA would be transformative for bridging and interop in general :)

                        How much of this do you suspect will be bridges for alternative open protocols vs alternative clients? Also, how do you foresee abuse/spam issues being handled?

                    1. 1

                      Personally I’d feel a lot more excited about this if you could give a working demo. I’m glad to see you’re doing VRM support though. I really hate having to re-rig my avatar for every chat app.

                      1. 1

                        so we’re in the middle of switching stack from matrix-js-sdk to hydrogen-sdk (hence https://thirdroom.io/ being a bit of a mess right now, although if you try hard enough it might work). There’s a video of the initial demo up at https://www.youtube.com/watch?v=e26UJRCGfGk&t=2263s however. I posted the intro post for the project today because we’re finally working on it fulltime as of this week.

                      1. 1

                        I enabled beta and feel Spaces is a much polished version of Communities. On the Spaces Beta discussion on the orange site @Arathorn said that the Discord style communities is the boring obvious bit.

                        For what it’s worth, the thing I find most exciting about Spaces is that they provide a decentralised hierarchical namespace with decentralised access controls for every room (ie pubsub topic) in Matrix. So it’s like we’ve sprouted an openly federated global hierarchical filing system for freeform realtime data streams of all flavours - where people can go crazy defining their own trees, applying their own curation ideals; perhaps we’ll even see a single global tree emerge (although the implementation may need some more optimisation first).

                        It’s like a multiplayer hybrid of DMOZ and USENET and the read/write Web all rolled together. Once we start storing more interesting data streams than instant messages in it (eg forums, email, bulletin boards, DOMs, scene graphs, ticker data, IOT sensor data…) it really gets interesting :)

                        “wow we accidentally created the realtime read/write web”

                        What is the non-obvious and interesting part? His rest of comment went over my head.

                        1. 2

                          I was trying to explain that while we wrote spaces to let users group their rooms together, in practice you can create hierarchies of spaces to group all the rooms together. For instance, I could create a space called #root:matrix.org and then a space within it called #opensource:matrix.org and then a space within that called #linux:matrix.org and then fill the space in that with all the linux chatrooms I know about. I could then give ops to other linux experts in the #linux space, and they could delegate ops onwards… until you’ve built a hierarchy that contains all the best chatrooms anyone knows about. It’s a multiplayer way to curate and categorise all the conversations of the world, including those bridged in from other networks and platforms.

                          Hope that makes more sense!

                          1. 1

                            That makes more sense, thank you for taking the time.

                        1. 8

                          Why not jump from old and quirky IRC protocol to e.g. Matrix? Also, matrix is an open federation, so this kind of grab shouldn’t be possible.

                          1. 15

                            We are ourselves old and quirky.

                            Freenode had a ~25 year run, which is significantly better than the median free tier on an online service.

                            1. 5

                              It is indeed quite the accomplishment. But IRC is clearly on the decline.

                              1. 1

                                IRC works well with very slow connections like dialup and archaic machines, but not unstable connections unfortunately, my main complaint is the lack of at least a small chat log without using 3rd party services/sw. Any small disruption will make me lose messages in a rural area internet.

                            2. 7

                              My main issue with Matrix is the lack of a client that I can run easily on extremely low-powered hardware. Just about all the major, well-supported clients are built on Electron. Compare that to IRC: you can have useful IRC client in just about ~5k lines of C (yes, I’ve written my own IRC client).

                              1. 4

                                I have to wonder if this is really the limiting factor for IRC - if we’re measuring protocols based on what you can write on a coke can, IRC might win, but is that what people actually want?

                                1. 2

                                  i do. the irc client i use is very fast and configurable. i don’t want to run a full web browser and 100000 tons of javascript just to exchange text with people. i currently use the weechat-matrix plugin for weechat to access matrix but it is unmaintained and missing many features i guess.

                                  1. 4

                                    weechat-matrix isn’t unmaintained - it’s stable. the author is prioritising matrix-rust-sdk, but weechat should work great.

                                    1. 2

                                      Yeah ok, ‘unmaintained’ is a little strong. My point is, nothing new is being added to improve support for matrix (multiline messages, etc) and there are lots and lots of quirks, having used it daily for many months now. And the author has made it clear they have no interest in improving the existing plugin while they go off and RWIR..

                                      It’s “good enough” for me, but a far cry from supporting everything matrix has to offer. That’s the case for almost all matrix clients though, as I’m sure you are aware.

                                    2. 3

                                      Fwiw I heard yesterday about https://github.com/poljar/weechat-matrix-rs . When it’s cooked it might be a good way for me to try Matrix seriously.

                                      1. 1

                                        Yeah, that has been around for a bit, and seems to be progressing along slowly. I don’t think it’s anywhere close to replacing the old python version of the plugin in its current state, and seems to be a long ways off from being there.

                                        1. 1

                                          Ah, good to know, thank you. Will keep an eye on it :)

                                    1. 1

                                      https://github.com/tulir/gomuks is roughly 15k lines of Go (+ non-trivial LOC from dependencies of course).

                                    2. 4

                                      try hosting matrix

                                      1. 3

                                        What makes you think I haven’t?

                                        1. 6

                                          Everyone I’ve talked to personally who’s tried this has nothing but horror stories when it comes to running their own homeserver. The consensus I’ve heard is that it’s only practical if you have staff to look after it or if you prevent your channels from federating.

                                          I admire the vision but they have a long way to go before actually realizing the benefits of a decentralized system.

                                          1. 4

                                            I’ve had very few issues running it myself. I have Synapse, Postgres, and Nginx running along with IRC, Discord, and Slack app services on a 2 GB VPS. Other than the occasional upgrade, I’ve had minimal issues. I manage everything through my service manager, so usually it’s as simple as running an upgrade task and then restarting the service. That said, I have a lot of experience running web services, so that might contribute.

                                            1. 2

                                              I’ve configured synapse by hand and using https://github.com/spantaleev/matrix-docker-ansible-deploy/ . Both work well provided you read the documentation.

                                            2. 1

                                              you are using matrix.org

                                              1. 3

                                                I have an account on matrix.org, true. That doesn’t prevent me from having accounts elsewhere. A matrix.org account is sometimes useful.

                                                1. 3

                                                  w.kline.sh/

                                                  I run my own too (@sumit:battlepenguin.im). It works pretty well, and I even have bridges working. Overall I think it’s way easier to stand up than XMPP (everything is over HTTP; there is that weird federated port but you can now use a normal LetsEncrypt cert and stick it behind a Traefik or HAProxy frontend).

                                                  I will say, scaling it would be difficult. I’ve heard other people complain about larger matrix servers with a lot of users and matrix.org has had issues with theirs after multiple huge refactors that dropped CPU usage. I think Matrix would be way better if there were multiple server implementations like ActivityPub does (Mastodon, Pleroma, Peertube, etc.) but it looks like development on the Go implementation is still slow going.

                                          2. 1

                                            Yes, go to Matrix, let the eternal september end here.

                                          1. 4

                                            Some interesting comments from the project lead over at HN.

                                            And here’s the user-facing post. I created a space for Haskell here (it includes the IRC bridge to #haskell).

                                            1. 1

                                              I created a space for Haskell here (it includes the IRC bridge to #haskell).

                                              URL changed to: https://matrix.to/#/#haskell-space:matrix.org

                                              1. 1

                                                How did you set the URL of the space? Been wanting to do this for Pikelet, which is currently a random hash…

                                                1. 1

                                                  You add an alias for the room that is the space. The UI is probably missing currently, but you can use the API. https://matrix.org/docs/spec/client_server/r0.6.1#put-matrix-client-r0-directory-room-roomalias

                                                  1. 1

                                                    Is there a way to send this request from element in the browser, or so I need to do some more involved shenanigans for this?

                                                    1. 5

                                                      More involved shenanigans, sadly. Adding aliases to Spaces is top of the list for the next wave of work in the beta though.

                                                      1. 1

                                                        No worries, looking forward to it! Thanks for your efforts!

                                            1. 1

                                              I have a hard time figuring out how to get matrix setup and working. Like what the backend and frontend are and how they work. Am I not understanding what it is?

                                              1. 10

                                                TL;DR: If you want to try it out, download the Element client and let it walk you through making an account.

                                                You’ll have to choose a Matrix homeserver (like an email provider). If you won’t use it that frequently, the free matrix.org homeserver is good but slow. For more serious use, consider a subscription to Element Matrix Services, where they host a homeserver for you. Or you can try to self-host synapse. I wouldn’t.

                                                Other homeservers are being developed right now (Conduit is pretty cool). But none are ready for production just yet. And unfortunately the choice of homeserver is still important because your account will be tied to it. In the future, the “multi-homed accounts” feature will make this initial choice less important (hopefully).


                                                There are two basic components to understand if you’re just getting into Matrix, and the two components are best understood as an analog to email, which is really the only popular federated protocol today.

                                                There’s the Matrix homeserver, which is like your email provider. It’s where the messages and account information are stored. It’s what actually takes care of sending and receiving your messages. It’s where you sign up. Multiple people can have an account on the same homeserver. Synapse is the most popular homeserver right now, it’s developed by the team that founded Matrix, and it’s considered to be slow (Python) and is slated to be replaced.

                                                Then there’s Matrix clients. Just like email, Matrix is standardized. You can use any email client to get/send your Gmail, and you can use any Matrix client to get/send Matrix messages. Element is the most popular Matrix client (again made by the team that created Matrix). It’s the most feature-complete by far. It’s written in Electron, so it’s bloated. But it works fairly well.

                                                1. 3

                                                  I wouldn’t.

                                                  Can you elaborate ? We use a synapse server at work and it works.

                                                  1. 1

                                                    I should have been clearer. I meant that I don’t advise trying to self-host the homeserver at all. Self-hosting anything is a ton of work if done properly. Timely security updates, migrations, frequent (and tested) backups, and reliability all come to mind as things I personally don’t want to have to worry about. Element Matrix Services seems like a good deal for just a few people.

                                                    1. 3

                                                      But these challanges aren’t at all Synapse-specific, are they? Updates. migrations and proper backups are something you have to do with any server that you self-host. And after running a homeserver for a few years, the only migration I ever had to do is from an older to a newer PostgreSQL version by simply dumping the whole database and reading it back in. All schema migrations are done automatically by Synapse and I never had any problems with that. Hosting a Matrix server with Synapse is so easy if you compare it e.g. to hosting your own email server. And Synapse really is battle-tested because it’s dogfooded at the huge matrix.org homeserver instance.

                                                      1. 1

                                                        No they’re definitely not specific to Synapse. That was pretty much my point.

                                                        And I know Synapse has put a ton of work into being easy to deploy. But I still won’t ever recommend managing infrastructure to anyone. It’s awesome that Synapse makes it as easy as possible for people like us to self-host it, but $5/month is probably well worth the lack of headache for most people.

                                                  2. 2

                                                    As far as I can tell, none of the homeserver implementations are ready for self-hosting – unless you disable federation, and then what’s the point?

                                                    1. 3

                                                      I’m not sure where you’re getting that impression. I’m hosting two different Synapse instances myself. I just update them when new releases come out; it’s been relatively painless.

                                                      1. 1

                                                        Can you please give a reason why you don’t think Synapse is ready for self-hosting? I’ve been doing it for years with enabled federation and I never had any serious problems.

                                                        1. 1

                                                          Sure. I’ve heard again and again that if you enable federation on Synapse and someone joins a large room, the server bogs down and starts chewing through resources. Has that changed?

                                                          Also note that I’d be running it on an old laptop or a raspberry pi, just like I would run any chat server – IRC, Jabber, etc.

                                                    2. 1

                                                      .. I mean, probably? What exactly are you struggling with?

                                                      1. 2

                                                        Uh oh, now I feel even dumber. The main website has information about something called Synapse and there is “element” which is a frontend I believe, but how do you install a matrix server and start using it?

                                                        1. 13

                                                          My attempt at clarification:

                                                          • Matrix is a protocol for a federated database, currently primarily used for chat
                                                          • Synapse is the reference home server (dendrite, conduit, construct etc. are other alternatives)
                                                          • Element is the reference client (there are versions of element for the web (electron), android and ios)
                                                          • A user account is (currently) local to a home server
                                                          • A chat room is federated and not located on a specific home server. The state is shared across home servers of all users that have joined the room.
                                                          • There are P2P tests where the client and home server are bundled together on e.g. a mobile phone
                                                          • Spaces are a way to organize rooms. Spaces are just a special case of a room and can include other rooms and spaces.
                                                          1. 4

                                                            Thank you! That clarifies a lot. I was stuck thinking Matrix is the server. So, Matrix is a protocol for a federated database, that’s very interesting and cool.

                                                            1. 1

                                                              Is it legitimate for me, as a user rather than someone who’s interested in the infrastructure, to just think of Matrix being like a finer-grained version of IRC, where instead of (mainly) a few big networks there are more smaller networks and instead of joining e.g. #linux on freenode, I’d join e.g. #linux:somewhere …

                                                              Would I now discover ‘rooms’ by starting from a project’s website, for example, rather than just joining some set of federated servers and looking for rooms with appropriate names?

                                                              I just searched for ‘linux room matrix’ and the top hit was an Arch Linux room #archlinux:archlinux.org

                                                              (I don’t really want to join a general Linux room - just using it as an example)

                                                              1. 3

                                                                Well, generally NO. Most all matrix home servers are all joined together via the federated protocol. So if you join #archlinux:archlinux.org on homeserver A, and your BFF uses homeserver B, you will still see each other and communicate with each other in that room like if you were both on homeserver A.

                                                                One COULD create a non-federated home server, but that’s not the typical use case, and the reasons to do so would be odd. If you are doing for example a chat server for internal chat @ $WORK, using Matrix is probably a terrible idea. Zulip, Mattermost, etc are all better solutions for that use-case.

                                                                1. 2

                                                                  Discovering rooms is currently a bit problematic, as room directories are per server. But a client can query room directories from any server (that allows public queries). Spaces will probably help a lot with room discovery, as they can form deep hierarchies.

                                                              2. 8

                                                                I did a video to try to explain it last year (which i should update, but it’s still usable, even if it calls Element by its old name of Riot): https://www.youtube.com/watch?v=dDddKmdLEdg

                                                                1. 3

                                                                  I recommend starting off by just creating an account at app.element.io and using the default homeserver so you don’t have to host anything youself

                                                                  1. 2

                                                                    Synapse is a server implementation and Element is one of the clients.

                                                                    Installing Synapse: https://matrix.org/docs/guides/installing-synapse

                                                                    1. 1

                                                                      Uh oh, now I feel even dumber.

                                                                      Don’t. The Matrix project is pretty terrible at both naming and the new user experience.

                                                                      1. 2

                                                                        Not trying to hate on them or anything. @ptman ‘s comment above really helped.

                                                                        1. 1

                                                                          Yeah, I wish them every success - but what I guess I’ll call the introductory surface of the system is fairly painful.

                                                                1. 3

                                                                  Could this be used to implement (seat underneath) the CapTP (Capability Transport Protocol )?

                                                                  1. 3

                                                                    Pinecone is a very generic overlay - while it’s hooked up in Matrix as a way of connecting nodes as a bearer for uTP requests, you could equally well layer IPv6 over it similarly to how Yggdrasil works. CapTP is at a much higher level - aiui it’s effectively distributed RPC with a capability security model, promise pipelining and distributed GC. I see it as more similar to the higher layers of Matrix, which give you decentralised access controls and eventually consistent data replication (albeit no RPC). So: yes, you could absolutely put a CapTP implementation over Pinecone.

                                                                    1. 3

                                                                      If the network is generic enough, I don’t see why not

                                                                      1. 2

                                                                        Maybe? It depends on the ultimate goal. The typical goal of CapTP implementations is to deliver messages from one machine to another, but that is also what Pinecone claims to do; therefore, it might not be necessary to build CapTP on top of Pinecone.

                                                                        1. 1

                                                                          Hmm, you might be right. Perhaps, I did have the right ‘categorization’ of capabilities of these systems.

                                                                          I was looking at CapTP as sort of a modern day CORBA, with the clever methodology to manage trusts across applications representing user/technical account identities (CORBA did not do that).

                                                                          While Pinecone is a system that manages trust and connectivity at the level of networked devices (nodes) – which in my, perhaps, superficial understanding, CapTP did not do.

                                                                          At the end of the day one need to trust network nodes, the pipes, the applications, and the user identities.

                                                                          1. 2

                                                                            From an introduction to capabilities:

                                                                            When a system, such as an OS or a website, is presented with a request for a service it provides, it needs to decide if it should actually do what the requestor is asking for. The way it decides is what we’re talking about when we talk about access control. If you’re like most people, the first thing you’re likely to think of is to ask the requestor “who are you?” The fundamental insight of the capabilities paradigm is to recognize that this question is the first step on the road to perdition.

                                                                            We do not need to trust nodes, applications, nor identities. Capability-oriented computing arises out of an understanding that we do not need to trust remote machines in order to achieve useful computation with them.

                                                                      1. 2

                                                                        I am cautiously optimistic that Spaces will get me something I desperately need: a way to have a set of private rooms that an allow list of accounts can enumerate and join. This is super easy with Slack and seems to be basically not possible with Matrix today, even if you set up the existing Community structure. Fingers crossed!

                                                                        1. 3

                                                                          yup, that’s very much the hope. https://github.com/matrix-org/matrix-doc/blob/matthew/msc1772/proposals/1772-groups-as-rooms.md#restricting-room-membership-based-on-space-membership is the section of the spec that outlines how it should work - and yes, you can’t do it today in Matrix without a bot to manage it (a bit like how folks used eggdrop decades ago on IRC before chanservs provided access lists).

                                                                        1. 1

                                                                          As bloat is my main annoyance, I checked out Hydrogen.

                                                                          It does superficially look nice, but I quickly found there’s no sane way to move encryption keys from other clients (including importing from Element), nor does it seem to have a way for doing the needed verification (initiated by Element).

                                                                          1. 5

                                                                            Hydrogen is beta, so: patches welcome. That said, sharing encryption keys with other clients works fine, if you are encrypting them and storing them in secure storage on the server. Manual import/export ironically would be much easier; just needs to be written. You’re right that it doesn’t do verification yet though.

                                                                            1. 1

                                                                              I understand it is beta, but I still expected a path to switch from using Element (the most popular client) to using Hydrogen.

                                                                          1. 7

                                                                            Such a big fan of Matrix. Glad to see this acquisition go well!

                                                                            1. 4

                                                                              Glad to see this acquisition go well!

                                                                              Oh no, this is not an acquisition, I think you mean adoption. Warm and fuzzy open source community adoption.

                                                                              1. 4

                                                                                In cased you’ve missed it - Matrix bought Gitter from GitLab.

                                                                                https://blog.gitter.im/2020/09/30/gitter-element-acquisition/

                                                                                1. 11

                                                                                  Matrix itself is an open source project + non-profit foundation (https://matrix.org/foundation), and hasn’t acquired anything :)

                                                                                  Element (the startup which makes the flagship Matrix client) did acquire Gitter from Gitlab though, in order to support Gitter and its communities and fund the integration into Matrix.

                                                                                  1. 3

                                                                                    Are those meaningful distinctions? It looks like a startup pushing the protocol they are developing by buying the competitors.

                                                                                    1. 11

                                                                                      It is a meaningful distinction, which is the only reason i was making it.

                                                                                      Matrix predates Element by 3 years, and Matrix is dedicated non-profit software foundation. There are loads of (massive) commercial companies building on the protocol; better known ones include Thales, Kudelski & Ericsson.

                                                                                      Separately, Element is a startup created by the team who created Matrix. It’s true that Element invests a lot of time into pushing Matrix (given if Matrix is successful, Element is successful), and Element acquired Gitter in order to push the protocol - not because it’s a competitor, but to use it a showcase for integrating Matrix into an existing chat platform, to encourage more projects to do so.

                                                                                      To be crystal clear, The Matrix.org Foundation (https://find-and-update.company-information.service.gov.uk/company/11648710) and Element (aka New Vector, https://find-and-update.company-information.service.gov.uk/company/10873661) are literally different companies.

                                                                                      But just as you wouldn’t describe Google or Mozilla acquiring Foo as “The Web acquired Foo” or “The W3C acquired Foo”, it’s very inaccurate to describe Element acquiring Gitter as “Matrix acquiring Gitter”.

                                                                                      1. 2

                                                                                        Element is a startup created by the team who created Matrix.

                                                                                        This is probably where the ‘confusion’ comes from. Just because the Matrix foundation thing and Element are technically/legally separate companies doesn’t mean that the people involved are separate, motivations are separate, etc…

                                                                                    2. 1

                                                                                      in order to support Gitter and its communities

                                                                                      was the Gitter community under distress prior to the acquisition?

                                                                                      1. 3

                                                                                        Considering the rising number of projects that only offer support through Discord, or other proprietary platforms, I would assume so.

                                                                                        1. 2

                                                                                          I think the most charitable description would be that GitLab largely left Gitter to its own devices.

                                                                                        2. 1

                                                                                          Thanks for clarifying!

                                                                                    3. 1

                                                                                      Yeah? I’m still lukewarm on Matrix since moving over from Keybase when Zoom bought the latter.

                                                                                    1. 2

                                                                                      If anyone from gitter is around, please remove the big ugly black [m] from every message, it is really distracting.

                                                                                      1. 4

                                                                                        Yup, the visuals need to be tweaked.

                                                                                        1. 2

                                                                                          If anyone wants to take a stab at updating this, the CSS is here.

                                                                                        1. 3

                                                                                          It would be nice if the core functionality of the iOS client[1] and accessibility of the ecosystem were fixed too, or at least if some visible movement was seen on those. The basic ability of two mutually encrypted users to send each other messages has been broken for months with very little responsiveness from the maintainers.

                                                                                          One of the key issues I have found in general with the matrix ecosystem is that it is highly inaccessible to the people who need it most – non-technical but tech-literate end users. For instance its SDKs and APIs and protocol are all very well (even beautifully) documented for the most part, but client functionality, less so – there is no documentation on the website or anywhere I could find in the repos discussing e.g. how to properly set up cross signing, or even going through the basic steps of how to verify another user’s device (not to mention why one would want to bother, or what E2EE is in layman’s terms)!

                                                                                          Coupling that with removing “escape hatch” functionality like the ability to manually verify someone from the mobile clients, and it becomes exceedingly difficult for non-technical users – even for more technical users like me – to understand when something isn’t working because of improper setup on the part of the user, or because the client has been broken for months as the iOS client has been.

                                                                                          The end result is that the entire ecosystem goes from being a genuinely useful tool for activists, journalists, and vulnerable demographics to being basically a toy whose benefits can only be properly enjoyed by more technical users for whom none of this is a matter of life and death. Blog posts like this, while interesting for someone like me who has the background to understand them, only reinforce this: very few of the blog posts on matrix.org are comprehensible to end users who aren’t also engineers.

                                                                                          [1] https://github.com/vector-im/element-ios/issues/3762

                                                                                          1. 6

                                                                                            So there’s a big difference between Matrix the protocol, Synapse the server implementation (subject of the OP here) and Element the client (subject of the comment here). The Matrix.org website and Synapse and the OP are unashamedly focused on developers and sysadmins.

                                                                                            Separately, totally agreed that matrix clients like Element need to do better at usability - the intention is of course to be as easy as WhatsApp / Telegram / Slack etc to use, and there should have some major changes in the next weeks to show visible progress there. Meanwhile, the bug you linked is a plain old bug which we’re hunting down (thank you for providing the logs).

                                                                                            1. 1

                                                                                              I appreciate the reply. I’m aware of the differences between the protocol and the server and client implementations. That said, I’m genuinely excited to hear about those updates, and to know that you agree there’s a usability problem. Thanks for working on that bug.

                                                                                            2. 1

                                                                                              Synapse was dog-slow with rooms with a large number of people for a long time, which made the experience of using Matrix frustrating and unreliable. Fixing that is as important for usability by anyone, including nontechnical people, as fixing client-side encryption.

                                                                                              The end result is that the entire ecosystem goes from being a genuinely useful tool for activists, journalists, and vulnerable demographics to being basically a toy whose benefits can only be properly enjoyed by more technical users for whom none of this is a matter of life and death.

                                                                                              People who understand encryption well enough to deal with Matrix client encryption usability issues can just as easily be activists, journalists, and people who would be harmed by having their private communications leaked, as people who don’t. These are orthogonal, not disjoint, categories.

                                                                                              1. 1

                                                                                                Thanks for the point about performance being relevant from an accessibility perspective

                                                                                                With regard to your second point, the set theoretic language obscures more than it helps here since we’re talking about accessibility. These sets are fuzzy: clearly activists, journalists, and others who aren’t software engineers, security researchers, or otherwise have technical (or tech-adjacent) day jobs can and do deal with Matrix client encryption. My claim is that Element is, at the time of writing this, orders of magnitude more challenging to learn, use, and trust if you aren’t in the latter category. For some, my observed experience is that the usability concerns make it completely non-viable.

                                                                                                The fact that some e.g. journalists can persevere and make it work for them is not relevant to my point – that is similar to saying “even though Acme Cafe isn’t wheelchair accessible, it’s still accessible to other people”.

                                                                                            1. 2

                                                                                              I’m probably missing something, but I fail to see how this solution helps combating propagation of child pornography. If I’m reading this correctly, it basically assumes that enough poorly-vetted participants will join and down-score before leaving (what happens with the score if they leave, which they have to?) and/or that there will be enough external indicators to find them, which really would be just punting the problem elsewhere. It seems to me to be based on the usual bad assumption that there will be enough “good actors” present to deter “bad actors” (for whatever that means).

                                                                                              What am I missing?

                                                                                              1. 2

                                                                                                I was thinking the same. People joining chats with illegal material will typically be the ones who are also interested in keeping it on the down low.

                                                                                                1. 2

                                                                                                  i think the confusion is over the idea of “down-score”. so say that you’re a govt chasing child abusers around Matrix: if you infiltrate a bad room, you might well publish a blocklist of the hashes of the users in that room, or the hash of that room ID, or the hash of content in that room. you could then leave the room, and that blocklist will hang around for as people want to use it. if people trust your list (and trust you not to abuse it by overreach or retaliation or whatever) then they will continue to use it as a way to block content they don’t want off the servers. this score then lasts for as long as people choose to trust it; if folks got ‘unfairly’ maligned by innocently lurking in a child abuse room then they’ll need to abandon that identity.

                                                                                                  in terms of external indicators: if you’re a random ‘good’ user on some room directory and stumble across a room full of obnoxious stuff, then you might flag it to the authorities, who would check it, ban it, and blocklist it. in practice, the external indicators do exist in order for people to find these rooms, so this seems to be a valid assumption (and not a case of punting the problem elsewhere).

                                                                                                  1. 3

                                                                                                    Thank you for your explanation. I understand now how this would help with content blocking, but not really in how this would help collect evidence against perpetrators. Blocking content is not nearly as important than stopping its creation (at least when it comes to child abuse). Could be I need to think about it more.

                                                                                                    I continue to disagree about punting. External indicators exist in large part because so far e2e encryption hasn’t been used as widely as its proponents would like it to be. I don’t see how one can both argue that e2e encryption protects you from snooping and not stopping legitimate cases of observing as well. You’d basically left to serendipitous discoveries of entries to those networks.

                                                                                                    1. 2

                                                                                                      I understand now how this would help with content blocking, but not really in how this would help collect evidence against perpetrators.

                                                                                                      By making it easier to identify abusive content, you can both filter it out (as an end user) or infiltrate/investigate it (as law enforcement).

                                                                                                      External indicators exist in large part because so far e2e encryption hasn’t been used as widely as its proponents would like it to be.

                                                                                                      It’s a contradiction in terms to e2e encrypt publicly visible content. If you’ve built a community for whatever purpose, you need to advertise it somehow - which means telling random people about it, which means consciously relaxing your encryption to do so.

                                                                                                1. 18

                                                                                                  The cited law doesn’t seem like it’s about combating abuse. It seems like it’s about providing a way to obtain evidence for prosecution. This article seems to fail to address the particular concerns the law is addressing.

                                                                                                  I believe backdoors are fundamentally unworkable and wrong. But the arguments against it need to do better than this.

                                                                                                  1. 10

                                                                                                    The point of the article is to prove that there is no viable solution to the particular concern the proposed law is suggesting (see also https://lobste.rs/s/ntyvtw/combating_abuse_matrix_without#c_52uuvc) and proposes an alternative which could help mitigate abuse across the board if embraced properly.

                                                                                                    But the arguments against it need to do better than this.

                                                                                                    go on then, we’re all ears :)

                                                                                                    1. 9

                                                                                                      I think where these arguments seem to go off the rails is in two ways.

                                                                                                      • They treat the backdoor proposals as a black and white proposition.
                                                                                                      • They propose solutions to a different problem than the proponents are trying to solve.

                                                                                                      If I had the audience to effect change here I would be focusing my arguments instead on the tradeoffs and I would be honest about them. We can’t allow shining a light on bad behavior like child exploitation, hate groups, and terrorism without sacrificing privacy/security in the long term. What we are actually talking about are tradeoffs that society has to make.

                                                                                                      To be clear the authorities actually don’t care about the 1/1 gpg encrypted communication between bad parties here. For them that’s a straw man argument. It’s such a small skillset that by definition it’s a smaller problem. The danger in there eyes is the mass enablement of more people to hide their bad behavior and the accompanying fear that the ability to hide results in an explosion of bad behavior because it’s safer and easier to hide it.

                                                                                                      When we refuse to engage them on that point we lose the argument. One small part of the argument is touched on in your article. The cat is out of the bag for something like Matrix. You only need one competent administrator in a different juristiction to provide a way to hide for other.

                                                                                                      But the other part of the argument that seems to be shied away from is that we regard the risk of to society of a backdoor as greater than the risk to children that good privacy and encryption poses. But unless you convince the rest of society that this is the case and get them to own it you will lose the argument.

                                                                                                      1. 9

                                                                                                        They treat the backdoor proposals as a black and white proposition

                                                                                                        That’s because they are a black and white proposition. There’s no such thing as half a back door. And “ability to decrypt anything on command and we promise to have a good reason” is a back door, no matter how many times politicians pretend otherwise.

                                                                                                        1. 2

                                                                                                          I think you misunderstand what I mean by black and white. Ubiquitious encryption is not an unambiguous good in many peoples eyes. It’s a tradeoff.

                                                                                                        2. 5

                                                                                                          Governments care about the ability to spy on their citizens. They are hiding this agenda behind the more politically potent cause of preventing child abuse.

                                                                                                          1. 6

                                                                                                            But you shouldn’t be trying to convince the government. The people you need to convince are the governed. The government are on one side of the argument and you are the other.