I find it a bit depressing. When I was an undergrad, email for the entire university (a bit over ten thousand accounts) was handled on a single powerful machine: a 500 MHz Pentium III with 512 MiB of RAM. Before Facebook bought them, WhatsApp was hosting over ten million users per server. Mastodon seems to be about three orders of magnitude less efficient than either of these.
I totally agree that Mastodon is way too heavy for what it is.
Fortunately, the Mastodon API used by the web and mobile clients is well enough defined that alternatives are being developed that are compatible with Mastodon clients (like Tusky or Pinafore for example).
GoToSocial is an example, coded in Go, it uses much less resources to achieve the same result, though it is still in development. I use it and it works well enough for me.
I recently moved into an apartment with symmetrical 1 Gbps fiber internet so acquired an older (but still punchy!) intel NUC off ebay to sit in the fiber closet and host things. It would be fun (?) to host my own mastodon instance, but I wonder whether standing up an instance and federating with others actually makes things worse for those instances, resource-wise, compared to just setting up an account on one of them.
You could also try one of the lighter-weight alternatives. I’ve been running a GotoSocial node out of my home for a few months and it’s only used around 100MB compared to 4GB for Mastodon. Much easier to set up too, since it’s just one self-contained binary.
I’ve been debating doing that, but their docs right now are screaming that they’re only alpha and not to use them. What’s your take been? Not too many bugs?
I’ve got a gotosocial up and the main thing I’ve noticed is that it seems to have weird federation / follow / etc. glitches - probably because whilst ActivityPub is a nominal standard, people seem to have implemented things differently (basically “implement the Mastodon API”). e.g. I can’t find one of my Pleroma accounts from the gotosocial one; the follow I have to a Mastodon server doesn’t show any posts on the gotosocial one. But they’re aware of these issues and seem to be actively working on fixing things.
I have only seen one bug so far, in the rate limiting system. I’m behind two layers of NAT so the client IP address identification is not reliable. Disabling rate limiting didn’t work; I had to raise the limit to 999999 instead. The maintainers have been very responsive so far, so I believe it will be fixed soon.
Did you evaluate Pleroma as well? I’ve been playing with that in on a free Oracle cloud ARM instance and it seems pretty performant. 1 GB memory total used for entire VM (FreeBSD, Postgres, app). It could certainly run on a RaspberryPi
I used to run Pleroma on my Raspberry Pi (in fact I have implemented new features for it: https://technomancy.us/191) but I can no longer recommend that project due to its association with certain high-profile bigots. I am watching the Akkoma project with interest, which forked the Pleroma codebase and has a more sensible maintainer, but it’s still early days.
I think all you’d miss out on from a community perspective is you wouldn’t have anyone else in the “local” section, you’d still have the federated section, trending if you enable it and anyone you follow.
I don’t think it’s obviously better for the big instance to handle a new local account compared to a new remote instance.
With the remote instance, they only have to send you events once for each activity of some of the people with accounts there. If you start an account there on the other hand, they have to start listening to and storing data from anyone you follow.
I don’t think it’s obviously better for the big instance to handle a new local account compared to a new remote instance.
I don’t think it is but, if I understand the moderation model correctly, it is easier for them to handle a new user on a remote server than a new remote server. In all cases, if the new user is benign then it’s fairly easy. There’s a bit more c2s traffic if the new user is local and s2s if the new user is remote and communicates with local users (follows in either direction, direct messages, and so on). For a malicious user, it’s different:
A local account needs the local admin to do something, to either close the account or educate the user.
A remote account on an existing server will be handled by the moderator(s) on that server. If they don’t do anything, then you can block their server and the problem goes away.
A new user on a new remote server requires handling in the same way as a new federated server with malicious users.
I don’t know how scalable the server block lists are, but the Mastodon instances that I’ve seen are typically blocking 2-3 servers and list them publicly. If ActivityPub becomes as popular as Twitter, then the easiest thing for a spammer will be creating a new server and then following a load of accounts and then sending them spam. If the only way that this is handled is by every single server needing to manually block the server, then I don’t see any way that this doesn’t turn into a massive problem.
The alternative is generally some form of reputation system. For example, I allow any messages from server A until someone on my server or someone on a server I trust follows someone from A (or until some number of people do). That would work, but provides some friction for new servers: you either know someone who is running one and ask them to trust you at the start, or you need to communicate that you have good people out of band and then have people on other servers follow them. I don’t know if existing Fediverse servers have anything like this.
The alternative is generally some form of reputation system. […] That would work, but provides some friction for new servers: you either know someone who is running one and ask them to trust you at the start, or you need to communicate that you have good people out of band and then have people on other servers follow them.
So basically the same issue as with SMTP servers, then.
following a load of accounts and then sending them spam.
Pleroma, at least, offers you a “following-only” timeline view - a spammer following me couldn’t get any spam in front of me. I could also write a filter which hides/content-warnings posts from previously unseen or “I am not following anyone on” instances using the MRF system.
I don’t know if existing Fediverse servers have anything like this.
I’ve not seen anything regarding “reputation” - but yeah, it would be exceedingly handy to have the ability to assign scores for posts / accounts / instances (+10 for mutual follow, +5 for I am following, -10 for a new instance I am not following anyone on, -100 for “account posts every 5 seconds bloody hell” etc.) much like SMTP spam scores.
Pleroma, at least, offers you a “following-only” timeline view - a spammer following me couldn’t get any spam in front of me
Does that prevent them from being able to send you direct messages? I guess that’s fine for some uses, but it means that people that follow you can’t reach you via the Fediverse.
Done some testing - looks like people you don’t follow can’t send to you, either public or direct posts. Which definitely cuts down on the spam angle. But yes, it does limit interaction from people you don’t follow.
Better check your ISP’s terms of use first. As I heard, a lot of ISP’s deny the use of their service for server hosting and you can end up with your contract broken :(
I’m not a Mastodon user. So maybe it is marketed wrong or something.
Reading through the article it sounds like it’s only to a certain degree about Mastodon and mostly about properly confusing PostgreSQ and Sidekiq. And the changes made seem extremely basic for anyone having weekend with either. Changing the shared buffer to something appropriate is the first thing you tend to learn and while I wouldn’t call myself an expert on Rails these Sidekiq configuration changes were also the first things I’d have looked into simply because that makes sense for the situation, pretty much with any kind of message queue or even some application.
I would assume there’s more things to improve and it sounds like Mastodon isn’t the most optimized thing but I also think that running anything like that pretty much requires looking into the used technologies (PostgreSQL and Sidekiq) instead of expecting it to magically rune itself. PostgreSQL doesn’t really have a way to know how much resources it should use and how. It’s highly configurable and has excellent documentation in that area.
The very fact that people are able to run vibrant, busy communities with tens of thousands of users on such small low powered servers fantastic.
I’ve seen instances running quite busy instances run reasonably well on a single little 4 vCPU, 16GB instance.
I find it a bit depressing. When I was an undergrad, email for the entire university (a bit over ten thousand accounts) was handled on a single powerful machine: a 500 MHz Pentium III with 512 MiB of RAM. Before Facebook bought them, WhatsApp was hosting over ten million users per server. Mastodon seems to be about three orders of magnitude less efficient than either of these.
I totally agree that Mastodon is way too heavy for what it is.
Fortunately, the Mastodon API used by the web and mobile clients is well enough defined that alternatives are being developed that are compatible with Mastodon clients (like Tusky or Pinafore for example).
GoToSocial is an example, coded in Go, it uses much less resources to achieve the same result, though it is still in development. I use it and it works well enough for me.
I recently moved into an apartment with symmetrical 1 Gbps fiber internet so acquired an older (but still punchy!) intel NUC off ebay to sit in the fiber closet and host things. It would be fun (?) to host my own mastodon instance, but I wonder whether standing up an instance and federating with others actually makes things worse for those instances, resource-wise, compared to just setting up an account on one of them.
You could also try one of the lighter-weight alternatives. I’ve been running a GotoSocial node out of my home for a few months and it’s only used around 100MB compared to 4GB for Mastodon. Much easier to set up too, since it’s just one self-contained binary.
I’ve been debating doing that, but their docs right now are screaming that they’re only alpha and not to use them. What’s your take been? Not too many bugs?
I’ve got a gotosocial up and the main thing I’ve noticed is that it seems to have weird federation / follow / etc. glitches - probably because whilst ActivityPub is a nominal standard, people seem to have implemented things differently (basically “implement the Mastodon API”). e.g. I can’t find one of my Pleroma accounts from the gotosocial one; the follow I have to a Mastodon server doesn’t show any posts on the gotosocial one. But they’re aware of these issues and seem to be actively working on fixing things.
I have only seen one bug so far, in the rate limiting system. I’m behind two layers of NAT so the client IP address identification is not reliable. Disabling rate limiting didn’t work; I had to raise the limit to 999999 instead. The maintainers have been very responsive so far, so I believe it will be fixed soon.
There are not that many bugs but a lot of missing features. I have a PR open here to make them clearer so people know what they’re getting into: https://github.com/superseriousbusiness/gotosocial/pull/1086
The most annoying is the lack of backfill; sometimes you have to rely on permalinks to read conversations that your server isn’t fully aware of.
Did you evaluate Pleroma as well? I’ve been playing with that in on a free Oracle cloud ARM instance and it seems pretty performant. 1 GB memory total used for entire VM (FreeBSD, Postgres, app). It could certainly run on a RaspberryPi
I used to run Pleroma on my Raspberry Pi (in fact I have implemented new features for it: https://technomancy.us/191) but I can no longer recommend that project due to its association with certain high-profile bigots. I am watching the Akkoma project with interest, which forked the Pleroma codebase and has a more sensible maintainer, but it’s still early days.
Thanks!
And there’s PleromaPi for that.
I mean, it can, but federation is also sort of the point of them existing, and owning your own domain is a substantial advantage.
I think all you’d miss out on from a community perspective is you wouldn’t have anyone else in the “local” section, you’d still have the federated section, trending if you enable it and anyone you follow.
I guess I could convince some of my friends to join. Only if I can come up with a good name (I have historically been bad at this).
I don’t think it’s obviously better for the big instance to handle a new local account compared to a new remote instance.
With the remote instance, they only have to send you events once for each activity of some of the people with accounts there. If you start an account there on the other hand, they have to start listening to and storing data from anyone you follow.
I don’t think it is but, if I understand the moderation model correctly, it is easier for them to handle a new user on a remote server than a new remote server. In all cases, if the new user is benign then it’s fairly easy. There’s a bit more c2s traffic if the new user is local and s2s if the new user is remote and communicates with local users (follows in either direction, direct messages, and so on). For a malicious user, it’s different:
I don’t know how scalable the server block lists are, but the Mastodon instances that I’ve seen are typically blocking 2-3 servers and list them publicly. If ActivityPub becomes as popular as Twitter, then the easiest thing for a spammer will be creating a new server and then following a load of accounts and then sending them spam. If the only way that this is handled is by every single server needing to manually block the server, then I don’t see any way that this doesn’t turn into a massive problem.
The alternative is generally some form of reputation system. For example, I allow any messages from server A until someone on my server or someone on a server I trust follows someone from A (or until some number of people do). That would work, but provides some friction for new servers: you either know someone who is running one and ask them to trust you at the start, or you need to communicate that you have good people out of band and then have people on other servers follow them. I don’t know if existing Fediverse servers have anything like this.
So basically the same issue as with SMTP servers, then.
Pleroma, at least, offers you a “following-only” timeline view - a spammer following me couldn’t get any spam in front of me. I could also write a filter which hides/content-warnings posts from previously unseen or “I am not following anyone on” instances using the MRF system.
I’ve not seen anything regarding “reputation” - but yeah, it would be exceedingly handy to have the ability to assign scores for posts / accounts / instances (+10 for mutual follow, +5 for I am following, -10 for a new instance I am not following anyone on, -100 for “account posts every 5 seconds bloody hell” etc.) much like SMTP spam scores.
Does that prevent them from being able to send you direct messages? I guess that’s fine for some uses, but it means that people that follow you can’t reach you via the Fediverse.
Done some testing - looks like people you don’t follow can’t send to you, either public or direct posts. Which definitely cuts down on the spam angle. But yes, it does limit interaction from people you don’t follow.
Better check your ISP’s terms of use first. As I heard, a lot of ISP’s deny the use of their service for server hosting and you can end up with your contract broken :(
Yeah I’m on google fiber which as far as I can tell allows hosting for non-business stuff. No static IP though.
I’m not a Mastodon user. So maybe it is marketed wrong or something.
Reading through the article it sounds like it’s only to a certain degree about Mastodon and mostly about properly confusing PostgreSQ and Sidekiq. And the changes made seem extremely basic for anyone having weekend with either. Changing the shared buffer to something appropriate is the first thing you tend to learn and while I wouldn’t call myself an expert on Rails these Sidekiq configuration changes were also the first things I’d have looked into simply because that makes sense for the situation, pretty much with any kind of message queue or even some application.
I would assume there’s more things to improve and it sounds like Mastodon isn’t the most optimized thing but I also think that running anything like that pretty much requires looking into the used technologies (PostgreSQL and Sidekiq) instead of expecting it to magically rune itself. PostgreSQL doesn’t really have a way to know how much resources it should use and how. It’s highly configurable and has excellent documentation in that area.