1. 6

    I’ve also tried to be open about my Sidekiq business and how it has grown over the past 5 years. It’s one thing to build a business on open source, it’s another to be open about the business itself. A recent interview:

    https://launched.app/blog/how-a-solo-founder-grew-an-open-source-tool-to-over-1m-arr/

    1. 2

      I appreciate you writing up posts on makig money with open source. More peope need to at least know it’s out there and think about it even if they might ultimately not go for money. I think there’s an awareness problem on top of ideological motivations.

      1. 1

        as someone looking to eventually start their own business, this is an awesome resource, thank you!

      1. 9

        I respect how they came to their old name but “bro” does have unfortunate baggage these days. I’m glad they found a new name they can be proud of.

        1. 3

          This is true.

          A product name is what’s supposed to represent you to the rest of the world. Connotations, type-ability, and relative uniqueness are basically what a name is for: if it carries negative connotations, then it isn’t serving its purpose.

          And trying to put a halt to the euphemism treadmill is like bravely standing your ground against a tsunami: the people who claim to do that tend to be all wet.

          1. 7

            My hope for the future of Go is that it will continue to embrace simplicity in the face of cries for complexity.

            I agree. I used to be a large proponent of adding generics to Go. But since Rust has taken off, I’d like it if Go and Rust explore two different approaches: Go with a drastic approach to simplicity and GC, Rust with parametric polymorphism + memory safety without GC. So with that in mind, I wouldn’t mind if Go did not add generics.

            I am using Rust as my primary language, but I would like it if there was a fall-back option in case Rust or Rust libraries become over-engineered. This is just personal opinion and I don’t want to be controversial, but I think Haskell went from a simple functional language to type wizardry. Scala followed a similar path, except that it already started as a more complex language by aiming to be a functional language, while simultaneously implementing many OO concepts. Type wizardry can be fun, but it typically results in libraries that are hard to use by diverse teams or set of contributors. Rust libraries are generally ok in this respect, but there is definitely a lot of room of over-engineering with generics and traits.


            The thing that I miss the most in Go, besides less repetitive error handling, is deterministic destruction. Especially when binding C. There is no guarantee when/if finalizers run and asking that callers call or defer a Close() function is a bit annoying. Destructors and GC are not mutually exclusive. A minor shortcoming is the lack of more powerful sum types.

            Outside the language, it would be nice to have a slower compiler that inlines and optimizes more aggressively. As far as I understand, gccgo performance is currently not much better yet than the native Go compiler.

            But everyone probably has a different set of wishes and if they were all added go Go, we’d end up with something that is not Go. So, it’s probably best for Go’s designers to be conservative ;).

            1. 1

              I’m getting more and more disenchanted with Rust’s memory safety claims. Seems like every week now we are hearing about another overflow in a Rust library due to unsafe. If you build a footgun, someone will use it.

              1. 19

                That is a good sign. The bad things you hear in the news are those that are rare and thus not dangerous. The really dangerous stuff is not reported in the news because these bad things happen all time.

                For a comparison, statistically you will probably die from some form of cancer in a hospital. That does not get reported in the news. Being killed from a plane crashing into a skyscraper. That is newsworthy, but nothing to be afraid of.

                When was the last time an overflow in a C program was on the lobsters frontpage?

                1. 2

                  Compared to the status quo (C, C++), Rust is in a far better position, and far easy to audit. It’s great that some of these other bugs are being sorted out, but it would be far worse in a language with no separation between safe and unsafe code. Yes, I would love to see a systems language that has been formally verified from the ground up, with dependent types that we can prove properties about some of the more tricky low level stuff, but this is hard to do and will take more years, and even then it’d be prudent to install escape hatches for practicalities sake.

              1. 5

                Sys-V style daemons have been deprecated for over a decade now, since Upstart and Systemd became things. Don’t use this.

                http://0pointer.de/public/systemd-man/daemon.html

                1. 1

                  A very interesting read. Thanks for the link! I work with embedded sound technology where its not uncommon to enter platforms that are driven from a highly hardware oriented technology house - here you often have no systemd, no upstart, no runit, no nothing but a pile of confusing intricate init scripts done in ash. From my perspective daemonizing the old fascion way is still a skill needed to be mastered. But as your link very well descripes, the pitfalls are many, and I certainly hope that eventually all of us will enter the new-style deamon way :-)

                1. 6

                  So, what’s the business model now?

                  1. 3

                    We are looking at a few options based on initial engagements with folks:

                    • the standard “support” model
                    • running Wallaroo as a hosted option in the cloud for folks. So for example, we install Wallaroo into your AWS account, hook it up to your data sources and run it for you.
                    • enterprise version where we build around the core product and create value that isn’t of interest to the open source community but are to large enterprises. for example, i doubt many folks in the open source community are interested in using Wallaroo with Kdb but its come up on more than 1 occassion with larger enterprises.
                    1. 4

                      Third option works the best. I mentioned that in the licensing discussion in another thread. Active Directory integration was the example I gave. Things like that stay proprietary. Good luck regardless.

                      1. 3

                        Yeah, the thing we want to do is not have it be features that are integrated into the open source product. We don’t want to keep things from the open source option, rather we really want to build around it. It’s hard. It’s going to be interesting but, I think it’s the right thing to do.

                        1. 6

                          Sounds risky to me. I respect your team’s commitment to principles, though.

                          Kind of adding to other comment, one thing I forgot to mention is you can just license it to businesses. That may sound counterintuitive since who would pay for open-source software they can get for free, right? A number of people on HN told me they license GPL software to businesses. One was at OpenSUSE but another just did custom apps for businesses that they allowed to go FOSS. They weren’t critical to competitive advantage, developer highlighted benefits of FOSS like extra features at lower cost, and they were cool with it. One pointed out that even Stallman encouraged people to charge for free software.

                          So, that’s possible at who knows what odds of success or level of revenue for Wallaroo. The main benefit they told me is businesses like having someone accountable for what they’re using. Someone to call, sue, or whatever. The other factor is that many believe you have to pay for good software. The “cheap” software is garbage. Free software must be even worse. Instead of arguing with them, those selling FOSS said they’d rather just let them pay a lot of money for the software and their own foolishness. :)

                          1. 3

                            O yes, that “license” with a “neck to wring” is part of what we cover in “support”. Sorry, we’ve been discussing it for so long that I forgot that many people rightly consider them to be different items.

                            It is risky but personally, I see the alternative as more risky. The infrastructure tooling space is dominated by open source products. Even if we create a better product, would we really be able to outpace a large open source project? Probably not. Could we lose lots of business to “its good enough” and people using another tool? Probably.

                            Those are also risks. So ya, risky but, we think worth the gamble.

                            1. 3

                              Good thoughts on infrastructure. I had one more idea but would’ve worked better if you GPL’d it. That is embedding it into commercial products that want to layer on top of your project or use one of its components. In the GPL model, they buy a copyleft exemption so they don’t have to GPL their product. Apache breaks that model given companies can just repackage it for free without legal risk from giving nothing back. Since you want it permissive, I tumbled idea around in head and it still might work.

                              There might be companies that want to integrate your stuff with theirs who would prefer to pay your company to do it faster (your experts) and with less integration risk. You could mention integration of Wallaroo into commercial offerings as a service Wallaroo Labs offers. I don’t think this would be common. It could be a nice supplement to main revenues if priced well to encourage the practice, you avoid any deals that look like a resource drain (or just charge extra for it), and Wallaroo kept modular enough to make that easy.

                              1. 2

                                I’m a huge fan of the GPL but man, it really scares large enterprises and makes selling to them much harder.

                                1. 4

                                  Oh I hear you. I was mainly mentioning it for other readers that might be evaluating pro’s and con’s of various options in licensing their software. The paid service for embedding was for you. I could see why you’d want to avoid GPL if dealing with big business. I also mostly “thank” Microsoft for that.

                                  1. 1

                                    Why not offer Wallaroo Pro which is Wallaroo/GPL relicensed with a commercial license? That’s literally part of my business model with Sidekiq (LGPL) and Sidekiq Pro (commercial).

                                    1. 2

                                      It’s something we considered but again, fear of GPL will keep some developers from using Wallaroo for free in some enterprises. In the end, we wanted to leave that open as a possibility.

                                      I’m comfortable with the GPL. Many people are not and their lack of comfort definitely ended up leading to a lack of comfort from other non-engineers at Wallaroo Labs. Apache 2 is a “safe license”. No one balks at it.

                                      All of that said, I think the model you have is a completely valid one.

                    1. 37

                      What about dependencies? If you use python or ruby you’re going to have to install them on the server.

                      How much of the appeal of containerization can be boiled directly down to Python/Ruby being catastrophically bad at handling deploying an application and all its dependencies together?

                      1. 6

                        I feel like this is an underrated point: compiling something down to a static binary and just plopping it on a server seems pretty straightforward. The arguments about upgrades and security and whatnot fail for source-based packages anyway (looking at you, npm).

                        1. 10

                          It doesn’t really need to be a static binary; if you have a self-contained tarball the extra step of tar xzf really isn’t so bad. It just needs to not be the mess of bundler/virtualenv/whatever.

                          1. 1

                            mess of bundler/virtualenv/whatever

                            virtualenv though is all about producing a self-contained directory that you can make a tarball of??

                            1. 4

                              Kind of. It has to be untarred to a directory with precisely the same name or it won’t work. And hilariously enough, the --relocatable flag just plain doesn’t work.

                              1. 2

                                The thing that trips me up is that it requires a shell to work. I end up fighting with systemd to “activate” the VirtualEnv because I can’t make source bin/activate work inside a bash -c invocation, or I can’t figure out if it’s in the right working directory, or something seemingly mundane like that.

                                And god forbid I should ever forget to activate it and Pip spews stuff all over my system. Then I have no idea what I can clean up and what’s depended on by something else/managed by dpkg/etc.

                                1. 4

                                  No, you don’t need to activate the environment, this is a misconception I also had before. Instead, you can simply call venv/bin/python script.py or venv/bin/pip install foo which is what I’m doing now.

                                2. 1

                                  This is only half of the story because you still need a recent/compatible python interpreter on the target server.

                              2. 8

                                This is 90% of what I like about working with golang.

                                1. 1

                                  Sorry, I’m a little lost on what you’re saying about source-based packages. Can you expand?

                                  1. 2

                                    The arguments I’ve seen against static linking are things like you’ll get security updates etc through shared dynamic libs, or that the size will be gigantic because you’re including all your dependencies in the binary, but with node_packages or bundler etc you’ll end up with the exact same thing anyway.

                                    Not digging on that mode, just that it has the same downsides of static linking, without the ease of deployment upsides.

                                    EDIT: full disclosure I’m a devops newb, and would much prefer software never left my development machine :D

                                    1. 3

                                      and would much prefer software never left my development machine

                                      Oh god that would be great.

                                2. 2

                                  It was most of the reason we started using containers at work a couple of years back.

                                  1. 2

                                    Working with large C++ services (for example in image processing with OpenCV/FFmpeg/…) is also a pain in the ass for dynamic libraries dependencies. Then you start to fight with packages versions and each time you want to upgrade anything you’re in a constant struggle.

                                    1. 1

                                      FFmpeg

                                      And if you’re unlucky and your distro is affected by the libav fiasco, good luck.

                                    2. 2

                                      Yeah, dependency locking hasn’t been a (popular) thing in the Python world until pipenv, but honestly I never had any problems with… any language package manager.

                                      I guess some of the appeal can be boiled down to depending on system-level libraries like imagemagick and whatnot.

                                      1. 3

                                        Dependency locking really isn’t a sufficient solution. Firstly, you almost certainly don’t want your production machines all going out and grabbing their dependencies from the internet. And second, as soon as you use e.g. a python module with a C extension you need to pull in all sorts of development tooling that can’t even be expressed in the pipfile or whatever it is.

                                      2. 1

                                        you can add node.js to that list

                                        1. 1

                                          A Node.js app, including node_modules, can be tarred up locally, transferred to a server, and untarred, and it will generally work fine no matter where you put it (assuming the Node version on the server is close enough to what you’re using locally). Node/npm does what VirtualEnv does, but by default. (Note if you have native modules you’ll need to npm rebuild but that’s pretty easy too… usually.)

                                          I will freely admit that npm has other problems, but I think this aspect is actually a strength. Personally I just npm install -g my deployments which is also pretty nice, everything is self-contained except for a symlink in /usr/bin. I can certainly understand not wanting to do that in a more formal production environment but for just my personal server it usually works great.

                                        2. 1

                                          Absolutely but it’s not just Ruby/Python. Custom RPM/DEB packages are ridiculously obtuse and difficult to build and distribute. fpm is the only tool that makes it possible. Dockerfiles and images are a breeze by comparison.

                                        1. 2

                                          Seems like there’s a blog post on lobste.rs every six months about this from someone “discovering” the problem.

                                          1. 5

                                            You can also freeze constant Strings to minimize GC pressure. I blogged about using a magic comment recently:

                                            https://www.mikeperham.com/2018/02/28/ruby-optimization-with-one-magic-comment/

                                            1. 3

                                              Sweet baby Jesus, string literals are mutable? This is why we can’t have nice things.

                                              1. 1

                                                TIL! Thanks for sharing 👍

                                              1. 38

                                                I like that the blog title and the domain name are directly at odds.

                                                1. 5

                                                  Headlines are hard. In the actual prose I say “almost always” :)

                                                  1. 14

                                                    So… an exception to your rule? Check mate.

                                                1. 3
                                                  1. It can’t do anything technologically useful.
                                                  2. It should be fast at doing what it does, so that it is safe to call LOLWUT on production instances.
                                                  3. The output should be entertaining in some way.

                                                  This is irritating to all of us with open PRs for real improvements, bugfixes, and enhancements that are starved for attention.

                                                  Because of the master-slave fiasco and intense work involved with scrubbing everything, he now needs a diversion to like working on his own project again? That’s great, it’s over – now let’s get back to brass tacks and fix defects, finally finish replication enhancements now being worked on for 2 years, finally wrap up streams (Salvatore’s pet project nobody asked for), and improve performance to play catch-up with other datastores.

                                                  1. 30

                                                    Or, antirez can spend his time as he sees fit, just like any other FOSS maintainer.

                                                    1. 9

                                                      I’m sorry your PRs weren’t getting attention, but it was very important to change a word and the negative discourse associated with that (and this resulting cry for help) were much more critical to spend cycles on.

                                                      You can’t prioritize software quality over Twitter grumping, that’d be silly.

                                                      1. 6

                                                        I haven’t followed the details of the master-slave kerfuffle, but it seems to be simply a failure of community management. As a lazy maintainer, it seems like the proper route is just to say “I acknowledge that people feel so strongly about this sort of terminology and will happily accept PR’s to fix the problem; this seems like a good candidate for new contributors to get their feet wet with” and link to a connected issue tracker.

                                                        Then after a year or two of no nobody submitting fixes you close the issue as WONTFIX and get on with life.

                                                        Edit: That said, taking a break from the stuff you have to do and making something functioning, silly and completely irrelevant in the grand scheme of things can be deeply satisfying. Sometimes you need that sort of thing to remind us why we love doing what we do.

                                                        1. 3

                                                          Yeah, I still don’t understand how Twitter discussions can have this impact on people’s lives.

                                                          Together with just some wording used since the beginning of IT. So why can’t words acquire new meanings? Isn’t that the definition of a living language?

                                                          1. 3

                                                            Because the people are suffering from some kind of delusion.

                                                            I would like to see the budding young DBA-to-be who veered off IT/CS/STEM because he or she got triggered by the word “slave”.

                                                            Or even honest testimonies along the lines of “I was always interested in code, but I’d suffer immense panic attacks when I saw the word ‘blacklist’. I’m a Person of Color so this touches me deeply. The maintainer was kind enough to rename it (we settled on disallowlist) and now I commit a few times a year to the project. I have never felt this included and welcome in my life. Best of all? No one even complained about the broken APIs after the rename!”

                                                            1. 1

                                                              What I find weird is the “matter of fact” tone plus many contributors (not necessarily the maintainer) caving in right away, as if they also felt dirty while typing “git push - u origin master” or felt all the weight of worldwide injustices when the main DB switched to the “slave” one. I was relieved reading sane supporters though.

                                                            2. 2

                                                              Yes words do this constantly. But only in a negative way. Negative connotations tend to hang around. That’s why using these words is problematic. Also impractical, since you’d have to add a disclaimer that you mean the word in the new sense not the broadly known old one. If someone knows an example of some words changing from negative to positive connotations I’d be happy to hear about this.

                                                              For the current example I honestly don’t understand the quarrels people have. Why is dropping language that’s strongly tied to a history of colonisation and oppression a bad thing? Especially when other words can describe the same architectural pattern as accurately as the original words?

                                                              And every programmer knows that choice of words is important, we shouldn’t use them too lightly.

                                                              To turn the question around: So why can’t concepts get new names if the old ones are problematic?

                                                              1. 7

                                                                If someone knows an example of some words changing from negative to positive connotations I’d be happy to hear about this.

                                                                • Fun - to cheat or hoax
                                                                • Smart - a sharp stinging pain
                                                                • Fond - a fool
                                                                1. 4

                                                                  “Nice” is one of such words. It meant foolish and silly.

                                                                  The problem isn’t much about naming by itself: you want to use “disallowedList” and “allowedList”? Want to use “primary” and “replica”? Want to use “kaka” and “popo”, do as you wish. But going ahead and picking a project at seemingly random and force them to change words that have been picked as they are (or used to be) the standard name of those elements in a somewhat aggressive way then that’s where I’m drawing the line.

                                                                  Or creating chaos on the issues section or on social media, extending the use of such standard words to the morals of the maintainers, that’s totally disgusting and absolutely useless.

                                                                  What I find even more shocking isn’t the vocal part that enjoys stirring up drama for whatever reason (there is clearly a reason why these groups are doing this and it cannot be to make the tech world a friendlier place, that’s for sure) but it is people who just shrugs them off or who, just like you, now supports their point because they are somewhat technically correct.

                                                                  Let’s start harassing all Spanish speaker developers who use the word “negro” for their black elements, how about the word “sheet” that reads as poo in Swedish, how about the word “pitch” that is extremely similar to a bad word in some Slavic languages, and “git”?

                                                                  That is and would be dumb to do. Also, why is the “American” morality be pushed down the throats of the rest of the world? Just because they still cannot cope with their past, is that a good reason to force everyone to change their behaviour?

                                                                  1. 3

                                                                    I share the idea that this debate is totally dumb. There were intellectuals in the 90s that brought this up with Intel and IDE controllers. PATA went away and that debate didn’t matter, but now we see the same bullshit arising again.

                                                                    The most vocal people in this fight are extremely childish and troll/harass developers. That’s not right. If you take a reasonable stance, you’re a racist.

                                                                    I’m tired of outrage culture.

                                                                    I highly recommend Brendan O’Neil’s speech on offence: https://www.youtube.com/watch?v=BtWrljX9HRA

                                                                    and I also recommend the original post about this debate with ver well reasoned arguments:

                                                                    http://antirez.com/news/122

                                                                    1. 2

                                                                      “In fact, pretty much every leap forward in history… pretty much every freedom we enjoy is a product of individuals having given offense. Having offended against the orthodoxies of their age. Offensiveness is not something we have to begrudgingly accept. Offensiveness is the motor of human progress.”

                                                                      I’ll add the people against offending folks are defending beliefs created by offending folks ranging from speech to progress. Just like he said. They wouldn’t exist if their rules were enforced by the orthodoxy of the day when people were trying to get those reforms started. So, they glorify the reformers who offended piles of people creating their current belief systems but say nobody is allowed to offend their orthodoxies suggesting alternative systems. Double standards are a common sign of bullshit.

                                                                      Edit: Oh, wait, he just said that, too, later in the speech. I’m still watching it.

                                                                    2. 2

                                                                      how about the word “sheet” that reads as poo in Swedish

                                                                      I don’t want to get into the broader discussion again, but I speak fluent Swedish; sheet /ʃiːt/ does not look or sound like skit /ˈɧiːt/.

                                                                      1. 1

                                                                        Can’t argue with you, I just looked up English words which sounded/looked like bad words in other languages. Maybe they had something specific in mind, don’t know :)

                                                                      2. 2

                                                                        IMNSHO the use of the word “slave” isn’t as simple as being “American”. The African slaves were often sold off by their own chieftains, and different types of slavery have existed and (may be argued to) exist around the world.

                                                                        The point about stirring up shit is more relevant. These are homonyms; a word that sounds the same or is spelled the same as another can have different meanings, as rhe examples above.

                                                                        The slave in an IT or mechanical context simply isn’t the same type of slave that picked cotton in the southern states.

                                                                        I’m sure there are plenty of Slavic lobsters here, but I haven’t read a single comment, here or elsewhere, of one of them being triggered by the etymology of that word.

                                                                        1. 2

                                                                          Exactly and that’s because on a global platform, filled with people from all around the world, with absolutely different cultures and languages, people (including myself) are not going to ruin someone else’s day just because they used a word which was/sound/looks/is derogatory in my own language on something totally unrelated.

                                                                          If there was something totally unacceptable, one could look into why such a word was chosen or if it was intended at all, inform them privately of the thing (if, say, they are interested in being known in those countries) and move on. Not starting a debate about something pretty much every single culture in the world has had.

                                                                          This seems to come from the wave of social justice which is plaguing certain countries, where one is offended for someone else and wants to be their saviour, while making everyone more miserable in the process, probably to achieve a “victory” for humanity or just themselves.

                                                                          1. 1

                                                                            It’s not specifically American at all. Human trafficking rings (a polite euphemism for ‘selling people into slavery’) are still being fought by law enforcement around the world today.

                                                                            I’m sure there are plenty of Slavic lobsters here, but I haven’t read a single comment, here or elsewhere, of one of them being triggered by the etymology of that word.

                                                                            Reasonable people aren’t triggered by thousand-year-old etymology.

                                                                            https://en.wikipedia.org/wiki/Human_trafficking#Revenue indicates $150 billion/year in revenue derived from forced labor, globally. Consider that your users might actually include someone who has been enslaved and would rather not be reminded of it at work.

                                                                            1. 2

                                                                              Yet your example is hypothetical. There might actually be someone. People shouldn’t be triggered by homonyms at all.

                                                                              Maybe the next hill to die on is killing or terminating child processes. Many women need to have late abortions and be reminded at work of infanticide.

                                                                              This will never stop unless maintainers put their foot down and say that messing around with APIs because a vocal minority bullies them is not ok.

                                                                              The case would be stronger if the proponents of this had personal experience to share, or other evidence to back it up. Even then, we’re talking about software, not people. Better would be to donate to Amnesty or do something else than change every word and rewrite all the dictionaries.

                                                                              1. 1

                                                                                People shouldn’t be triggered by homonyms at all.

                                                                                If only we got a choice about what trauma the world inflicted on us.

                                                                                Maybe the next hill to die on is killing or terminating child processes

                                                                                Maybe. But since nobody has, to my knowledge, brought it up - you are raising a ridiculous straw-man (straw-person, ha).

                                                                                This will never stop unless maintainers put their foot down and say that messing around with APIs because a vocal minority bullies them is not ok.

                                                                                Of the things that might stop it, that seems like one of the least likely to work.

                                                                                As noted elsewhere in the thread, “Feel free to submit a complete PR, including all documentation updates” is more than enough to get them to leave you alone. In the unlikely event that someone cares enough to actually do the work, I’d suggest you’ve just gained a hardworking co-contributor.

                                                                          2. 1

                                                                            Please do not conflate my questions or position with abusive behaviour towards others. My post was not about that. I haven’t asked you to do anything, nor have I endorsed abusive behaviour by others. If my questions or position causes so much grief and anger maybe it’s worth exploring why that is the case?

                                                                            Please relax, there’s no need for this aggravating tone here.

                                                                            1. 1

                                                                              Mmm what?

                                                                1. 1

                                                                  Lack of any feedback from the repo maintainers would be such a let down for me.

                                                                  1. 2

                                                                    The last time this project had significant activity was around 20 days ago. The pull request is aged at 19 days - I think it’s just unlucky timing. Perhaps the maintainer went on for holiday with his family? Perhaps he just finished a lot of work around the project and needs a rest from it? Perhaps he just doesn’t feel like looking at it now - it’s volunteer time.

                                                                    Point is, don’t assume your work is unappreciated because you didn’t get a reply - open source is hard work but once you put things into the public time works for you, someone will see it sooner or later :)

                                                                    1. 2

                                                                      Rubygems.org is maintained by the RubyTogether non-profit, which pays devs hourly for maintenance. The money is nice but it likely means it will take a bit to get approved for work. I’m guessing this also isn’t high priority – Rails 5.1 is working just fine.

                                                                  1. 1

                                                                    Really nice work and writeup!

                                                                    1. -3

                                                                      “Considered Harmful” Essays Considered Harmful (I think “considered dangerous” falls in the same category)

                                                                      It’s not difficult to use C correctly. Don’t blame your vulnerabilities on C when the real culprit is your own sloth.

                                                                      I’ll concede that C (and it’s API) has quite a few foot guns, but I’ve learned how to avoid them pretty effectively, and I should be able to expect the same from kernel devs. The whole “rewrite everything in <insert promising new lang here>” mentality doesn’t work for large projects (like kernels). To rewrite the Linux kernel in Rust would take months (even if you had all hands on deck). And, who’s to say that Rust wouldn’t change incompatibly three times in the middle?

                                                                      1. 21

                                                                        It’s not difficult to use C correctly.

                                                                        [citation needed]

                                                                        There is no evidence to suggest that large codebases written in C can maintain memory safety in the face of that. The counter evidence, that writing code in C/C++ tends to produce large volumes of vulnerabilities, for reasons that are explained by language choice, is plentify. To whit, every major OS (Windows, Linux, macOS), every major browser (Chrome, Firefox, Edge, Safari), every major anti-virus program, every major image parsing library, I can keep going for a while.

                                                                        Denialism about the dangers of memory unsafety is not productive, we need to move on to discussing how we address this.

                                                                        1. 0

                                                                          There is no evidence to suggest that large codebases written in C can maintain memory safety in the face of that.

                                                                          Using C correctly means not making large codebases. C isn’t a language for programming in the large.

                                                                          1. -1

                                                                            there is no evidence that large codebases in any language produces anything better.

                                                                            1. 7

                                                                              Yes there is. The default, failure mode of safe languages doing common things is not potential code injection. The default for C language is. Given same bug count, using C will lead to more severe problems. The field results confirm that from fuzzing to CVE’s.

                                                                              1. 4

                                                                                Yes there is. The default, failure mode of safe languages doing common things is not potential code injection.

                                                                                I don’t think this is wrong, exactly, but there’s a 100 exploits related to python pickle, etc. as counterexamples. And java serialize, etc.

                                                                                1. 3

                                                                                  Do the memory-safe parts have the memory errors of C (a) at all or (b) as much? And do libraries in concurrency safe languages show same or less races as equivalent in multithreaded C?

                                                                                  You’re going to find vulnerabilities in all of them. My side are saying C amplifies that number by default or others greatly reduce it by default. That’s all we’re saying. I think the evidence is already supporting that.

                                                                                  1. 1

                                                                                    amplify requires some comparative numbers.

                                                                                    1. 2

                                                                                      The numbers on using C are that the common operations lead to piles of vulnerabilities with code injection. This happens a lot on average. It happens less with veterans but still happens. That’s irrefutable. The numbers on safe languages show the problems mostly lead to compiler failures or DOS’s from runtime checks. The burden of proof is on your side given your side’s stuff is getting smashed the hardest all the time whether the app is small or big.

                                                                                      What numbers do you have showing C is safer for average developer than Ada, Rust and so on? And I’m especially interested in fuzzing results of software to see how many potentially lead to code injection among new, half-ass, or just time-constrained programmers in C vs the same in safe, systems languages.

                                                                                      1. 1

                                                                                        you don’t even have good examples of large scale systems built using some other language that are substantially safer. Until you do, it’s just folklore.

                                                                                2. 0

                                                                                  I see a real shortage of example of large-scale systems constructed in any language that are secure and bug free but I am happy to look at references. Like what do we have comparable to Qmail written in something better that has fewer bugs? I know that C has numerous limitations, but in CS we tend to embrace projects that claim a win by hiding a problem by e.g. using pragmas to do the things that are the most buggy as if pushing the problem into the corner made it go away.

                                                                                  And the code injection bugs I see are all example of bad engineering - not of bad programming.

                                                                                  1. 3

                                                                                    There’s bugs and there’s serious bugs that the language causes. The latter are what hackers hit the most. The latter are what we’re talking about, not just bugs in general. The size of the program also doesnt matter since the safe language is immune to the latter by design. Scaling code up just increases odds of severe vulnerabilities in the unsafe, control language.

                                                                                    Java and .NET apps are what to look at if you want big ones. Very few CVE’s posted on the apps of the kind you see in C apps. The ones that are posted are usually in C/C++ runtimes or support libraries of such languages. That just illustrates the problem more. The languages whose runtimes arent C have fewer of those since they’re immune or contain them by design.

                                                                                    1. 1

                                                                                      My impression is that a) the reasons that those c/c++ runtimes show up so much is that these language delegate the most dangerous code such as parsing of raw input or packets or complex interaction with the OS to the C/C++ runtimes where it is possible to do that work and b) the same errors show up in different form in different languages. The massive prevalence of scripting exploits is not due to C but to lazy interface construction where, for example, user inputs are treated as parts of database scripts etc etc. I do not think that “do all the hard stuff in pragmas or C libraries” actually does limit vulnerabilities.

                                                                                      1. 1

                                                                                        “where it is possible to do that work”

                                                                                        The first part is true. That part isn’t. They think lower-level language is better for speed, bit handling, or OS interface. The second part implies you need C to do that work. There’s systems languages which can do that work with more safety than C. So, it’s “possible to do that work” in them without C’s drawbacks. Many low-level programs and OS’s were written in PL/0, PL/S, Ada, Modula-2, Oberon, Modula-3, Clay, and so on. They’re safe by default turning it off only where you need to. C doesn’t do that since it’s designers didn’t care when they were hacking on a PDP-11 for personal use.

                                                                                        “b) the same errors show up in different form in different languages. The massive prevalence of scripting exploits is not due to C but to lazy interface construction where, for example, user inputs are treated as parts of database scripts etc etc.”

                                                                                        Aside from something language-specific, the logic errors that happen in scripting languages can happen in C, too. You get those errors plus C’s errors plus the catastrophic effect that comes with them being in C. Let’s say you wrote the interpreter in Ada or Rust with safety-checks on. Most of the errors in the interpreter won’t lead to hacks. The extensions would have same property if building on base language like how extensions to C-based programs are often in C having same problems. Platforms like Java that built libraries on C are hit heavily in those C dependencies.

                                                                                        Additionally, the extensions could leverage aspects of these languages, such as type or module systems, designed for knocking out integration errors. Finally, if it’s Ada 2012 and SPARK, they can eliminate runtime checks in performance-critical code by using the provers to show they’re not needed if specific pre-conditions pass early on. Unlike Frama-C, they get a good baseline on code they hurried and highest assurance of what they proved.

                                                                                        1. 1

                                                                                          Data would help. These arguments by what seems sensible to different people don’t go anywhere.

                                                                            2. 16

                                                                              To rewrite the Linux kernel in Rust would take months (even if you had all hands on deck).

                                                                              Months? It would take at least 10 years, regardless of headcount.

                                                                              I’ve learned how to avoid them pretty effectively, and I should be able to expect the same from kernel devs.

                                                                              I’m impressed with your abilities, but then something nags me about the order-of-magnitude mistake in your rewrite estimate. Hmm.

                                                                              1. 13

                                                                                It’s not difficult to use C correctly. Don’t blame your vulnerabilities on C when the real culprit is your own sloth. I’ll concede that C (and it’s API) has quite a few foot guns, but I’ve learned how to avoid them pretty effectively, and I should be able to expect the same from kernel devs. The whole “rewrite everything in ” mentality doesn’t work for large projects (like kernels). To rewrite the Linux kernel in Rust would take months (even if you had all hands on deck). And, who’s to say that Rust wouldn’t change incompatibly three times in the middle?

                                                                                I suggest you read the linked article first. The title is clickbait but the content is solid. No one even mentioned Rust or anything else… The guy talks on their effort to reduce the foot guns in the kernel code…

                                                                                Here is a quote for the lazy:

                                                                                Kees Cook gave a presentation on some of the dangers that come with programs written in C. In particular, of course, the Linux kernel is mostly written in C, which means that the security of our systems rests on a somewhat dangerous foundation. But there are things that can be done to help firm things up by “Making C Less Dangerous” as the title of his talk suggested.

                                                                                1. 4

                                                                                  I suggest you read the linked article first.

                                                                                  Ok, you got me, I only skimmed the article and I didn’t see any mention of rewrite until the comments (it was literally the first response to the second comment). Although I do hear that mentality about other large projects (such as Firefox) as well. I guess I should’ve said “Clickbait considered harmful” ;-)

                                                                                  I’ve read some more of the article and he seems to know what he’s talking about but I would like to see the original talk.

                                                                                  As far as reducing foot guns, I guess Linux did start out as just one guy so I can understand a lot of foot shooting, but it’s been years and I would’ve thought that things like VLAs would’ve been avoided in the kernel. Then agian, I’ve never worked on a project as large as Linux so i guess I’m not the best judge of such things.

                                                                                  1. 4

                                                                                    Ok, you got me, I only skimmed the article and I didn’t see any mention of rewrite until the comments (it was literally the first response to the second comment). Although I do hear that mentality about other large projects (such as Firefox) as well.

                                                                                    Agreed. It’s annoying as hell, and the loud-mouths never do the work.

                                                                                    I guess I should’ve said “Clickbait considered harmful” ;-)

                                                                                    Funny because the talk is titled ‘Making C Less Dangerous’ - the lwn reporter is actually responsible for the horrible title that misrepresents the content and invites rewrite talks. I think this is the first time I’m using the lobste.rs ‘suggest’ a new title option to rename the link to ‘Making C Less Dangerous’ disrespecting the reporters chosen title. This is an abstract of the talk so keep the title close to the content.

                                                                                2. 7

                                                                                  Literally 20+ years of unending computer security exploits disagree with you.

                                                                                1. 4

                                                                                  YAGNI (You Are Not Gonna Need It)

                                                                                  Quibble: it’s actually You Ain’t Gonna Need It

                                                                                  1. 12

                                                                                    The error draft proposals look great – this is a real need.

                                                                                    I would like to see Go’s standard types get useful higher-level list operations like Ruby’s Enumerable, map, reduce, filter, etc but I’m not sure Go needs to expose that for all types. Maybe special casing the standard types gets us 80% of the benefit of generics without further complexity to user code.

                                                                                    1. 2

                                                                                      yield and Enumerable are the crux to building idiomatic Ruby APIs. I love them so much.

                                                                                      1. 25

                                                                                        I give away Sidekiq and sell Sidekiq Enterprise. If you use Ruby/Rails, it’s the standard. https://sidekiq.org

                                                                                        1. 3

                                                                                          it’s the standard

                                                                                          For a very good reason. Wonderful piece of software. Thanks @mperham!

                                                                                          1. 2

                                                                                            Woah thats awesome. I have been using sidekiq lots. Great bit of software.

                                                                                            1. 1

                                                                                              Hey @mperham!

                                                                                              Thanks for all of your code. I’ve used and loved several of your projects.

                                                                                              Out of curiosity what is the current status of Faktory? It sounded like an interesting project, but the rate of development looks like it kind of cratered last Dec.

                                                                                              1. 2

                                                                                                It’s under active development but summer has been slow due to family issues. Latest:

                                                                                                https://mastodon.xyz/@mperham/100583959557092421

                                                                                            1. 23

                                                                                              “It is difficult to get a [web developer] to understand something, when [their] salary depends on [them] not understanding it.”

                                                                                              ― Upton Sinclair

                                                                                              1. 4

                                                                                                My back looks like a pin cushion from all the arrows I received over the years fighting for web that would be more ethical and void of mostly useless crap. Some battles won, too many lost. I lost one just yesterday, but it didn’t occur to me that it was because of my money-induced blindness.

                                                                                                I actually like this quote and have used it myself before, but while I met many web developers over the years who didn’t care about bullshit described in the article, almost all of them didn’t simply because they were either ignorant of available technologies, didn’t care much about quality of anything they did and most often both.

                                                                                                1. 1

                                                                                                  Some battles won, too many lost.

                                                                                                  What were some of the wins?

                                                                                                  1. 4

                                                                                                    Example of a small recent one would be Klevio website (as it currently exists, less so after today). I am not linking to it because I don’t want referrals from Lobsters to show up in website’s logs, but is trivial to find.

                                                                                                    Almost everything on this website works with Javascript turned off. It uses Javascript to augment experience, but does not needlessly rely on external libraries. Should work reasonably well even on poor connections. Does not track you and still has a privacy policy handling that tries to be closer to the spirit of GDPR then to what you may get away with.

                                                                                                    It would certainly be easier for me and faster to develop (cheaper for company) if I just leaned on existing tools, build yet another SPA and have not spent more than a week arguing with lawyers about what is required.

                                                                                                    Alas, because unsurprisingly most people do not opt-in to analytics, I am now working on a different confirmation dialog, more in line with what others are doing. It will still be better than most, but certainly more coercive than current.

                                                                                                    And this is in a company that is, based on my experience, far more conscientious about people’s privacy than others I worked for.

                                                                                                    1. 1

                                                                                                      It would certainly be easier for me and faster to develop (cheaper for company) if I just leaned on existing tools, build yet another SPA and have not spent more than a week arguing with lawyers about what is required.

                                                                                                      Is this really true? Not to downplay your craft but I always thought tinkering with HTML/CSS until things look right would be way easier than learning a separate library.

                                                                                                      I checked out that website and it’s pretty refreshing that stuff actually works. If you want a little constructive feedback, the information density is very low especially on a desktop computer with a widescreen monitor. I have to scroll down 7 screens to get all the information, which could have fit on a single screen. Same with the “about us” page. I notice the site is responsive, giving a hamburger when you narrow your window, so maybe the “non-mobile” interface could be more optimized for desktop use.

                                                                                                      1. 1

                                                                                                        I don’t think it is in every case, but in this one I think it would be since everything was handwritten without picking up existing solutions for things like galleries. If you mean the SPA part, then I guess it becomes more moot. It would probably be about the same doing the first implementation, but this one, which is basically a bunch of static files, certainly has a higher cost of maintenance because we (I) didn’t get around to finishing it so page “components” still have to be manually copied to new files and updated everywhere when their content changes. The plan was to automate most of this, but we haven’t spent the time on it yet.

                                                                                                        I agree with everything in the second paragraph. Regretfully that is one of those battles lost.

                                                                                                        1. 1

                                                                                                          so what do your managers feel is the benefit of having such low information density? how do these decisions get made?

                                                                                                          1. 1

                                                                                                            If I remember correctly it was because it supposedly looks modern, clean and in-line with company’s brand. It has been a while so my memory is fuzzy on this.

                                                                                                2. 2

                                                                                                  I’ve heard this a few times already, but I’ve never quite understood what the implication is. What precisely are web developers not understanding? I get the default examples (eg. oil companies funding environmental research), but just can’t see the analogy in this case.

                                                                                                  1. 22

                                                                                                    You’re on week three of your new job at a big city ad and design firm. Getting that first paycheck was nice, but the credit card bill from the moving expenses is coming up, that first month of big city rent wiped out your savings, and you don’t really have a local personal network to find new jobs. The customer wants a fourth “tag” for analytics tracking. Do you:

                                                                                                    1. Put it in
                                                                                                    2. Engage in a debate about engineering ethics with your boss and his boss (who drives a white Range Rover and always seems to have the sniffles after lunch) culminating with someone screaming and you storming out, never to return?
                                                                                                    1. 8

                                                                                                      Web devs know that auto play videos and newsletter pop ups are annoying but annoying people is profitable

                                                                                                  1. 4

                                                                                                    May I request a better title in the future? Something like “jsoniter - faster than Go’s json”.

                                                                                                    1. 2

                                                                                                      You can directly suggest to change the title, if you think it would be better.