1. 36
  1. 19

    Mastodon used about ~2.5 GB out of the 4 I have on my Pi. With Pleroma, the total used RAM is only about ~700 MB. That’s crazy!

    I agree it’s crazy. Crazy less bloated, and crazy still bloated.

    700MB. Christ.

    1. 27

      To be clear, the 700 MB is the total RAM usage, i.e. by all programs and not Pleroma alone.

      1. 21

        That 700MB includes a Postgres database and a webserver.

        1. 9

          I wonder if we can still run pleroma in a 256mb ram system. Most of the ram is used by postgres, and that can be configured to use a lot less.

          1. 11

            I bet you can but PostgreSQL is also very tricky to limit RAM usage to a certain cap. First off the defaults are very conservative in most cases you would be cranking all the values up not down but you already know that as if I recall correctly I saw some great articles regarding PostgreSQL inner-workings from your blog posts on pleroma development.

            That said there are several configs that have direct and indirect influence on how much memory PostgreSQL will use: shared_buffers - the actual working set of the data the DB hacks on, that will be the largest immediate RAM allocation. Then we have the tricky parts like work_mem which is a per connection allocation but not a per connection limit. If your work mem is 8 MB and you execute a query which has 4 nodes in the resulting plan you can allocate up to 4*8 MB for that one connection. If you add to that parallel query execution then multiply that by concurrently running workers. I assume pleroma uses a connection pool so that alone can bump RAM usage a lot. Add to that things like maintenance_work_mem for tasks like vacuums and index rebuilds and you quickly can see how the actual memory usage can fluctuate on a whim.

            To the point.

            I agree it’s crazy. Crazy less bloated, and crazy still bloated.

            700MB. Christ.

            I simply think @ethoh is wrong. 700 MB usage is crazy low for a RDBMS and we are talking about RDBMS + a whole app using it. Databases are designed to utilize memory and avoid hitting the disk when not necessary. Unused memory is wasted memory.

            1. 3

              700 MB usage is crazy low for a RDBMS

              I don’t really get how you can make this claim with no reference at all to the data storage needs of the application. A fair metric would be the overhead of the DB relative to the application data. In this case we’d need to know some things about how Mastodon and Pleroma work, and how OP managed his instances of them.

              1. 4

                I don’t really get how you can make this claim with no reference at all to the data storage needs of the application.

                In similar fashion the OP claimed that 700 MB is crazy bloated. I was making a reference to that. However to back up my claims with some quick napkin calculations:

                Default shared_buffers for PostgreSQL 12 is 128 MB. Per PostgreSQL documentation the recommended setting is roughly 25% of available system RAM then measure.

                If you have a dedicated database server with 1GB or more of RAM, a reasonable starting value for shared_buffers is 25% of the memory in your system.

                source: https://www.postgresql.org/docs/12/runtime-config-resource.html

                The system in question has 4 GB of RAM so by that logic 1 GB for shared_buffers would be a reasonable setting - hence 700 MB at that point could be considered crazy low.

                Default work_mem is 4 MB, max_worker_processes is set to 8 and max_connections by default is 100 (https://www.postgresql.org/docs/12/runtime-config-connection.html#GUC-MAX-CONNECTIONS). This means that query execution can easily eat up to 3.2 GB by default in the absolutely unlikely worst case scenario.

                maintenance_work_mem is by default an additional 64 MB.

                So we are looking at PostgreSQL itself using anywhere between 128 MB and 3 GB of RAM with it’s default settings that are ultra conservative and usually the first thing everyone increases. This is before considering the actual data and application workload.

                By this logic, personally for me 700 MB for PostgreSQL on a running Pleoroma instance including the memory used by Pleroma itself is crazy low.

                1. 5

                  But, this little Pi is not a dedicated database server, it at least hosts the app too? And defaults are just defaults. Maybe indicative of PG usage in general, across every application that uses it, but that’s a really broad brush to be painting such a tiny picture with! I still think there are a few different species of fruit being compared here. But I do appreciate your explanation, and I think I understand your reasoning now.

                2. 1

                  Fwiw, my Pleroma database is approaching 60GB in size.

                  1. 1

                    Due to shit posting or bot? You can clean it up a little bit by expiring remote messages older than 3months

                    1. 2

                      I have a dedicated 500GB NVMe for the database. Storage isn’t a problem and it’s nice for search purposes.

              2. 2

                I’m still not convinced that postgresql is the best storage for ActivityPub objects. I remember seeing in pleroma that most of the data is stored in a jsonb field, and that makes me think that maybe key-value stores based on object’s IDs would be simpler and maybe(???) faster.

                I’m currently implementing a storage “engine” based on this idea and I’m saving the plain json as plain files in a directory structure. It, of course, is missing ACID[1] and other niceties, but I feel like the simplicity of it is worth for an application that just wants to serve content for a small ActivityPub service without any overhead.

                [1] IMHO ACID is not a mandatory requirement for storing ActivityPub objects, as the large part of them (activities) are immutable by design.

                1. 5

                  Misskey used to use a nosql / document store. They switched to postgresql because of performance issues. I’m sure you could build an AP server with a simpler store, but you we do make heavy use of relational features as well, so the relatively ‘heavy’ database part is worth it for us.

                  1. 2

                    Yes. One problem with a off the shelf key value store in this setup is that scanning over the whole keyspace to be able to filter objects is way less efficient than a good indexed db. (Even though I’m not there yet), I’m thinking of adding some rudimentary indexes based on bloom filters on properties that might require filtering.

                    1. 4

                      postgresql provides indexing for json objects, so it makes a lot of sense to use it even for this kind of use case. Even sqlite has some json support these days.

                  2. 2

                    I am not convinced to store tons of small files individually, they are usually less than 1kb. The overhead from inode will waste 75% of a 4k, and you will also run out of inodes pretty quickly if your fs is not tuned for tons of small files.

                    1. 3

                      inodes are a legacy filesystem problem. Use ZFS :)

                      1. 1

                        The idea behind federation would be that most instances would have a small number of users with small local storage needs.

                    2. 1

                      Not really for recent releases, you need at least 512MB for a stable instance. Pleroma itself use <200MB RAM, and postgresql can use another 200MB, depends on your configuration.

                  3. 10

                    Total RSS for my Pleroma instance on Arch x86_64 (which is extremely lightly used) is ~115MB. There’s a bunch of other RSS being used by the Postgres connections but that’ll depend on your precise configuration.

                    For comparison, my honk instance on Arch armv7l is using 17MB (but it admittedly bare-bones compared to Pleroma.)

                    1. 2

                      How is honk working for you? Do you want to share a link to your instance? I’ve been considering installing it myself. It seems cool, but the only honk deployment I’ve seen in the wild is the developer’s. If we’re talking about saving resources, honk seems to be better for that than Pleroma :)

                      1. 3

                        I run it for my single user instance. Haven’t tried upgrading since I installed it.

                        It generally works as expected, if a little rough - I edited a bunch of the default templates and found the terminology a little obtuse, and threads where some replies are private don’t show any indication which can be a bit confusing.

                        I may setup Plemora at some point, as I would like the extra features, but I might never get around to it because honk is so trouble-free and works alright.

                        1. 2

                          Pretty well - I just run the binary in a screen session on one of my servers for now.

                          https://honk.rjp.is/ - mainly using it as a publish-on-your-own-space feeder for tweets via IFTTT.

                          1. 3

                            Have you looked into crossposting using one of the open source crossposters?

                            I’m assuming that they won’t work because honk has fewer features than Mastodon, but I don’t actually know.

                            1. 2

                              I did try moa for a while but the link [from moa to twitter] kept disappearing for some reason - I did intend to self-host it but never got around to it. IFTTT is easier for now and if I want to shift off IFTTT, I’ve already got “RSS to Twitter” code for other things I can easily repurpose.

                              [edited to clarify “link”]

                      2. 4

                        Fwiw it’s a bit over 300MBs on my (single user) instance.

                        1. 3

                          I still think that 300MB is a lot, especially when cheaper VPS can have only 500MB of RAM.

                          1. 3

                            In fairness, 512 mb is a ridiculously low amount of memory.

                            Nowadays it’s possible to configure a physical system with 128c/256t and literally terabytes of ram and we’re still paying 2013 prices for 1c/512mb VPS instances.

                            Think about this.

                            1. 1

                              I’ve been mostly using (and recommending) the smallest hetzner vps instances, which have 2gb of ram and cost just under 3 euro per month. although looking at lowendbox, i see that you can get a 1gb vps for $1 per month.

                      3. 18

                        I didn’t know about Pleroma. Here’s a description of it.

                        https://blog.soykaf.com/post/what-is-pleroma/

                        1. 6

                          So I am growing increasingly dissatisfied with even the minimal utility I get out of Facebook (sharing pictures of my kids with family members) and have thought about standing up a fediverse instance and trying to use that, instead. The problem for me is with the clients – how are they, for elderly parents with a strong interest in granddaughters and none at all with fiddling with technology? How is the fediverse for non-nerds?

                          1. 10

                            You should absolutely not share pictures of your kids on ActivityPub. Depending on the software you are using its either more like Twitter or more like blogging software like Wordpress (wordpress has an AP plugin actually, and dedicated blogging instance software like write freely and plume exist and rock)

                            Can’t speak for the current state of Diaspora, but THAT is what you are looking for.

                            1. 5

                              The Fediverse, in general, is closer to Twitter than it is Facebook. That being said, Tusky for Android and Mast for iOS are both (IMO) better than the Twitter client for both platforms. Both incredibly polished and intuitive. There’s a number of high quality clients, but these are the two that I’ve personally settled on.

                              1. 3

                                Interesting. I wonder if there is anything that is more of a Facebook replacement.

                                1. 4

                                  Diaspora perhaps?

                                  1. 3

                                    Yeah, Disaspora looks more facebook-like. I don’t think it’s quite as popular as the Fediverse, and it’s not built on ActivityPub to my knowledge.

                                    If you’re into more Instagram-like (ie. photosharing), there’s always Pixelfed as well, which is part of the Fediverse

                                    1. 3

                                      ActivityPub. And no, it’s not a great protocol for a Facebook-like; the existing projects are all fairly nascent and have been struggling with follower/friend mechanics.

                                  2. 3

                                    friendica could be interesting for you

                                    1. 3

                                      Thanks, I’ll check it out. I’ve always wondered about building something like Facebook but focussed on the needs of families, particularly families with small kids. Yeah, it would never be a billion dollar thing, but nowadays it seems like there might be an appetite for something less crap than Facebook.

                                      1. 2

                                        For the social network, I always thought that the Google circles concept was way better. I wonder if there are any successors to that concept.

                                  3. 2

                                    I don’t use iOS so I haven’t been able to try it myself, but I just want to mention that Mast does still seem to be open source, the repo is here: https://github.com/ShihabMe/Mast2 It’s annoyingly difficult to find that repo, I found the link on the developer’s Mastodon account. Many open source projects don’t adequately advertise their open source nature, which is a source of significant frustration for me.

                                    1. 1

                                      And the guy is doing a lot of support directly on mastodon.

                                2. 4

                                  Thanks for this. I tried running a masto instance and it ran for a few months until it ate itself. Some subtle failed database upgrade problem that caused the back end and the front end to stop being on speaking terms - so my instance RAN. You could log in, but you couldn’t send toots or receive toots, just kinda observe the main timeline :(

                                  It’s just TOO BIG TOO MUCH AAAH.

                                  1. 3

                                    thanks for the tips. I run an instance for a single user and I also plan to do same to free some resources on my machine (700 Mb is still a lot). Mastodon is indeed an heavy beast (for my use case at least).

                                    1. 5

                                      why should you migrate? Because Pleroma is cleaner, leaner and prettier looking. Oh, and we have chats.

                                      This fails to mention why you shouldn’t migrate.

                                      Because Pleroma is easy to set up, it is therefore easier for fash and nazis to set up. There’s a whole lot of abuse that admins and mods have to deal with coming almost exclusively from Pleroma instances, so rather than dealing with that individually, which is time-consuming, frustrating, and stressful, a lot of mod teams have been blocking pleroma instances on sight. None of this is even mentioning the rumours of unsavory and *phobic opinions that have been voiced by members on the dev team. So it’s not a surprise that marginalized communities will act like this, the mod teams want to protect themselves and their users from having to deal with that kind of bullshit, and this is the option that doesn’t involve constant reports and months of stress added to their (already likely-hectic) lives.

                                      1. 25

                                        You’re saying Pleroma is inherently bad because bad actors use it? That’s not a very good argument.

                                        1. 18

                                          FWIW I read it more as a warning that you could get caught in the crossfire due to overworked moderators that make overly-broad policies as a measure of desperation, not “pleroma bad”

                                          1. 8

                                            I think this is a misunderstanding - I’ve avoided migrating to Pleroma despite wanting to host my own instance because

                                            1. a lot of people I’m friends with, including very technical people, have had a terrible time getting assistance from the maintainers, and

                                            2. a lot of the rest of the Fediverse blocks Pleroma instances on sight, or nearly, because of some genuinely awful stuff that the developers have said and because of the preponderance of fashy instances. This is decidedly unfair to users but it’s not something that can be swept under the rug, not is it “politics over technology”.

                                            1. 17
                                              1. The Pleroma folks have only been helpful if anything at all. “My friends” isn’t a valid source.

                                              2. Again, if you’re going to call a group of people “fashy”, the least you could do is provide sources. “Pleroma = Nazi” seems like a running meme on the fedi, always backed by ZERO evidence. Quit spreading FUD.

                                              1. 1

                                                Quit spreading FUD.

                                                I don’t think you understand what FUD means. The point of FUD is to bolster other products, or remove the product from the ecosystem. How does removing Pleroma from the ecosystem make it better? Mastodon already has a monopoly on the fediverse, but regardless – they are both open projects with large contributor bases? So what exactly is gained here?

                                                FUD implies that the person has something to gain from spreading Fear, Uncertainty, and Doubt. What exactly do random people on the fediverse gain from spreading this?

                                                1. 4

                                                  What exactly do random people on the fediverse gain from spreading this?

                                                  That Mastodon remains the primary Fediversal network and thus shutting down whoever the network’s hivemind slightly disagrees with.

                                                  1. 2

                                                    To be quite honest, I don’t think “The Mastodon developers want to stop any competition by spreading that the developers of Pleroma are fascists” can be considered any less of a conspiracy theory than you consider “The Pleroma developers are fascists” to be.

                                                    Except the latter has a couple of hundred people in my network saying that they have had personal, negative interactions with the Pleroma developers along the lines of fascism. People who have literally nothing to gain from saying this. People who do not give a shit whether or not Mastodon is the primary node.

                                            2. 5

                                              You’re not giving a very charitable interpretation of their post.

                                              Their comment isn’t that Pleroma is bad because bad people use it. Their comment is that, one disadvantage of Pleroma is that many mod teams ban Pleroma instances on sight.

                                              The underlying reason why Pleroma instances are banned on sight might be that bad people use it, but “bad people use it, therefore it’s bad” wasn’t the argument.

                                            3. 5

                                              Because Pleroma is easy to set up, it is therefore easier for fash and nazis to set up.

                                              This is weak and, imo, a bad way to open this (otherwise very much correct) post. It’s not that there are fashy Pleroma instances becuase it’s easier to set up; rather, it’s because the devs are, truly, terrible bigoted people, and because the software is, in my experience, used a lot more by fashy communities than other fediverse software is.

                                              1. 15

                                                That’s funny, my impression always was that Pleroma instances were a lot more pleasant, and their devs seem more interested in federation and otherwise sane technical practices. My theory is that this was because they didn’t promote themselves on twitter as a “twitter with better moderation”, which necessarily leads to a more toxic user base.

                                                1. [Comment removed by author]

                                                  1. 19

                                                    Sources please. This is a meme on Fedi, in my experience.

                                                2. 15

                                                  the devs are, truly, terrible bigoted people

                                                  Example of this?

                                                  1. 15

                                                    It’s all misdirection and FUD. Nobody ever posts examples they just keep saying it and insisting that it’s true.

                                                    1. 14

                                                      That’s my point, if you can’t show me at least screenshots of someone saying or doing something terrible, I have no reason to believe you. There’s a ton of heresay and second-hand gossip on the Fediverse (and the internet in general) that I take basically no one’s word at face value.

                                                  2. 12

                                                    it’s because the devs are, truly, terrible bigoted people

                                                    Please cite examples of something they’ve done. This is the most bizarre baseless accusation.

                                                3. 2

                                                  I have an account on a mastodon instance on sdf. I barely ever use it, but I can’t imagine why a service like that (that runs on limited budget) is wasting resources running mastodon instead of pleroma.

                                                  1. 4

                                                    I expect it’s because Mastodon came out first, and more people have experience running it in production.

                                                    Additionally, if you’re running a service for a fair amount of users, there’s a bit more involved migrating that many users than migrating a single user, which is what the OP did in their post. More can go wrong. Maybe this falls in line with “if it ain’t broke, don’t fix it”.

                                                    That said, if the cost of running Pleroma was significantly less than Mastodon it might be worth the hassle.

                                                  2. 0

                                                    The DevOps tag is to computer systems as the Culture tag is to lobste.rs