1. 5

    To address some of the author’s complaints — when judging from the lens of other languages and ecosystems — and note that I refer to Haskell, because it is the author’s current favorite programing language …

    Compilation times are a problem, especially for libraries where macros are involved

    • I hope Dotty will solve this; that said the compiler works pretty well, compared with a language like Haskell — maybe it’s because the dependencies are distributed as binaries, but I always hate compiling Haskell projects, whereas Scala is bearable
    • In my experience Scala compile times are OK, however if you experience issues, the best thing you can do is to split your project in sub-projects, thus forcing a clear separation between modules — if a library like Shapeless becomes a problem, then the parts that depend on Shapeless should be in their own module
    • that Twitter is building their own compiler is pretty cool, however note that companies like Google, Facebook and Twitter operate at a scale that makes any compiler problematic, especially when they have monorepos
    • if anything that Twitter is working on this is great, it means we might get something really good out of it

    Tooling for Scala is in my opinion, great and I have much experience with many other ecosystems, including Java, Node.js/JavaScript, Python, Ruby, Perl, .NET and Haskell

    • IntelliJ IDEA is the best IDE available on the market and the Scala plugin is made by Jetbrains and has their quality standards
    • other ecosystems can only dream of such support 😉 just to give an example, all Haskell IDEs (plugins for IntelliJ IDEA, VS Code, Emacs, etc) are unusable, the only reasonable option for Haskell is ghci and ghcid (which is basically sbt console and sbt ~compile respectively), everything else sucks
    • Scala’s sbt is one of the best build tools available — you get incremental compilation, continuous testing, dependency management, a REPL and tons of plugins for most tasks you have in one package; its syntax can surely be better and some of the plugins would need better maintenance, however overall I miss sbt whenever I’m building projects for anything non-Scala
    • It is regrettable that Ensime was deprecated and if Metals isn’t used, that’s a shame too — note however that other languages, like F# and C#, have official tooling and LSP server integration in their organization; I do hope they’ll reuse the work from Metals
    • On the other hand, everything that comes out of Scala Center is free work for us and if you don’t like what they do, then you can always roll your sleeves and provide a competing alternative

    It is indeed hard to hire Scala developers, but have you ever tried hiring Java developers that have some knowledge of multi-threading and asynchrony concerns?

    Hiring good developers is not easy at all. Also one mistake that people make is to hire Java developers and then retrain them for Scala. I think this is a mistake …

    Hire Node.js / TypeScript developers instead. JavaScript developers tend to be familiar with the style of functional programming practiced in Scala — too bad they write quite shitty code due to their dynamic typing roots — but you can filter those out by selecting for TypeScript experience. I now believe that Node.js + TypeScript devs are better candidates. And that’s not the only pool you can search in.

    The real problem are the concepts involved, which aren’t at all easy to grok:

    • higher order functions
    • immutability, persistent collections
    • not working with null
    • signaling errors via types instead of exceptions
    • map and flatMap everywhere
    • implicits
    • type classes
    • higher kinded types
    • macros
    • if you’re super unlucky: working with untyped actors, especially ones that have shared state

    The more you go down this path, the WTFs per minute increase exponentially. As you read this list, you should realize this isn’t about the language at all, but about its functional programming culture. It’s in fact much harder to hire for languages like Haskell. The more you diverge from the mainstream practices, the harder it is to hire. That’s why Go, like PHP before it, is a raging success

    Java hasn’t caught up in any meaningful sense.

    And that’s in fact a problem, because the pool of Java developers familiar with basic FP concepts is very small, which is one reason for why we’ve got an inability to hire people. I’m now more inclined to hire from pools where devs are familiar with basic FP concepts.

    I want Java to catch up, just a little would go a long way. It doesn’t and anybody that watched Java’s evolution should know that the language has been historically super conservative, all meaningful improvements going into the runtime and I don’t see that changing. Many people have been screaming for C#’s features since 2001 at least.

    Scala 3 is going to be a great release. The reasons why I like Scala 3:

    • cleans up macros — which is great, because macros are the main reason for why Scala is hard to learn by beginners
    • simplifies implicits
    • makes typeclasses natural, introduces derivation — this should address the author’s main concerns with libraries like Shapeless
    • adds untagged union types — just like what you have in TypeScript
    • might express nullables via unions — we already use Option, but this is a tagged type; having the ability to express nulls via the type system is great, because it reduces the accidents
    • adds enums (nicer syntax for ADTs)
    • GADTs are now supported

    This is a list of features that simplify the language and if there’s breakage involved, then it’s worth it.

    Also Scala 2.14 binaries are going to be binary compatible with Scala 3. This is not the same situation as with Python 3.

    In your career it’s important to not get bitter — tools are tools. If a tool stops working for you and you can’t invest in that ecosystem anymore, then simply move on.

    And criticism should be welcome, but make it constructive. This article contains criticism that’s not constructive. Nonconstructive criticism isn’t criticism you make in the hope of changing things and it leads to anger and resentment, both for you and your readers.

    1. 1

      The lead dev asks for money/recognition through social networks? What a bunch of beggars! He needs money? Me too! Does this person have a Patreon? Who cares! This guy owes me to use his software, he loves coding for free, the sucker.

      I think it’s great when people help out or donate to OSS projects, but I’ve been getting a little nervous some of the attitudes towards project funding/sponsorship I’ve seen lately. Corporate sponsorship of projects gives those sponsors a fair amount of power over those projects. Personally I’d rather use software that was produced by someone who’s able to make it without a for-profit company subsidizing development. I think it also makes forking a project more difficult as you won’t be able to carry that revenue stream with you to the fork and is an extra hurdle to competition with the original software .

      I’m not saying it’s wrong to accept money for work on an open source project, just that there’s a risk of harmful commercialization when we push for monetary support of OSS.

      1. 8

        I’m speaking as an open source author with contributions that were never sponsored, everything being entirely in my free time …

        What you want is not sustainable due to these facts:

        1. people need to put money on the table, so they need jobs
        2. working in their free time cuts into their personal life, which is especially hard when you have a family; and let me tell you, OSS work may be fun, but it is NOT relaxing; in fact when you have users, it’s more stressful than your day job
        3. not having a personal life is a recipe for burnout

        You may ask yourself, given that I confessed to doing this in my free time, how I cope with it? The answer is that I don’t. And I don’t know for how long I’ll manage to keep it up.

        Of course, highly popular projects can afford a good churn out rate, however such projects are also sponsored and have long timers available to train new contributors. For medium projects (e.g. most programming libraries) long timers are very hard to replace due to having specialized knowledge that’s hard to pass on to the next guy. And sometimes the problem domain is a difficult one, therefore the buss factor isn’t great.

        The cold, hard truth is that projects eventually die without eventually receiving sponsorship in some form.

        So if you want healthy projects, either donate money or contribute. Because otherwise, frankly, the wishes of non-contributors aren’t very relevant.

        1. 3

          I’m not convinced that people have to choose between a personal life and OSS work. People’s capacity to work on OSS on top of a job varies, maybe instead of working 8 hours a day on OSS projects some of us only have a couple hours here and there and the amount of work you can do in that time will be different from person to person. But personally I’d rather use a piece of software with a slower release cycle that’s produced and maintainable by hobbyists than one that has full time developers but is beholden to a sponsoring organization.

          And realistically I’m skeptical that it makes economic sense for anyone involved. If the cost to a company of using OSS is to pay one or more developers of that software anything close to their market value they’d be better off actually hiring someone to do it, then either keeping it inhouse to have a competitive advantage, or release under their copyright/license for the PR. For developers, donations for OSS work is probably the least you’ll ever be paid for your time.

          1. 2

            Out of curiosity, have you done meaningful open source work? Have you led any projects that have users?

            Note that I am not talking of occasional bug fixes or personal code that you end up storing on GitHub and that nobody uses.

            I’m asking because I believe that no OSS software author with serious contributions would ever claim that working in your free time is doable long term.

            And if you’re new to this, just give it 2-3 more years, we can talk then :-)

            1. 3

              I’m not sure what your criteria for “meaningful” is but I have run projects with a userbase in my free time.

              You’ve referred to your experience a couple times now and questioned mine, that’s fine, but I feel like you haven’t really touched on the main points I raised: OSS development by corporate sponsorship poses a risk of commercialization that we should be wary of.

              1. 1

                The point wasn’t to highlight my experience or anything, but I also talked to a lot of other contributors and the pattern is the same … burnout. It’s a thing that eventually happens to a majority of OSS contributors that use their free time.

                The other pattern that keeps happening is the self-entitlement complex of users that don’t contribute anything.

                Wariness sounds good in principle, can’t hurt to be wary right? But it is highly discouraging and dare I say, is in line with the self-entitlement. Notice how you’re suggesting that people should just sacrifice their free time, instead of selling out, what’s the problem?

                You not being convinced that people have to choose between a personal life and OSS work is absolutely irrelevant, because it is not you that has to choose. Talk to the maintainers of the projects that you’re using and that are actively maintained (instead of dying), ask them what costs they have. I do that very often.

                The point was, if you haven’t maintained a popular project to have an idea of the personal costs that can incur, and unless you come up with an alternative plan to make OSS sustainable (e.g. crowd funding, etc), then your oppinion is actively harmful to an ecosystem that already depends on too few people.

                The reality is that most popular OSS projects that people depend on are understaffed. The bus-factor is huge. The dangers of the authors selling out to big corporations is the last thing you should worry about ;-)

                1. 1

                  burnout. It’s a thing that eventually happens to a majority of OSS contributors that use their free time.

                  Do you have anything beyond anecdotal evidence to support this claim? Or that corporate sponsorship prevents burnout?

                  But it is highly discouraging and dare I say, is in line with the self-entitlement.

                  It’s discouraging and self entitled that I prefer to use software that isn’t funded by for profit organizations? I’m sorry if that’s how you feel but I don’t think it’s a very good argument for corporate sponsorship of OSS.

                  If I was a developer that only worked on proprietary software, would I be justified in calling out people who prefer to use open source software as being “highly discouraging and self entitled”?

                  Notice how you’re suggesting that people should just sacrifice their free time, instead of selling out, what’s the problem?

                  I think that’s a poor representation of my position here. At no point have I said anyone is obligated to work for free, I haven’t shamed anyone for “selling out” as you put it. I’ve simply said that I would prefer to use software that doesn’t require sponsorship from a for profit organization to exist.

                  Again, I think the OSS/proprietary analogy works well. I don’t think people who work on closed source commercial software have “sold out”, I don’t think they’re doing something wrong. But I prefer to use open source software because I think there are advantages on both philosophical and practical grounds. Same thing here.

                  The point was, if you haven’t maintained a popular project … then your oppinion is actively harmful

                  I’m sorry but this reads a lot like “you don’t have the same experiences as me so you’re wrong”. I haven’t couched my argument in my experience, so trying to say I don’t have X, Y and Z qualification to speak on the matter doesn’t really do anything for your case here.

                  1. 1

                    Asking for “evidence” in this is completely ridiculous, it’s not like you’re expecting some sort of study. Would a randomized controlled trial be enough? Let’s not do sealioning.

                    As for my argument being about experience, yes, that’s exactly what I’m saying, it’s good that we understand each other.

                    Have a good day.

                    1. 1

                      Asking for “evidence” in this is completely ridiculous, it’s not like you’re expecting some sort of study. Would a randomized controlled trial be enough?

                      Some kind of systematic evaluation of data would seem reasonable. A systematic observational analysis would seem like a reasonable place to start, I haven’t said anything about controls or randomization.

                      Surely you see the issue with leaning entirely on anecdotal evidence right? Like if I provide an anecdote about a successful project that didn’t involve corporate support have I made a compelling case for my position in your mind? If not, how is what you’re doing any different?

                      As for my argument being about experience, yes, that’s exactly what I’m saying, it’s good that we understand each other.

                      Well I guess it is good that we understand each other, although I’ve make it clear why I don’t think you asking me or anyone else to simply take your word on the matter is reasonable. And again, even if you think you’ve conclusively ruled out the possibility of volunteerism software development at scale you don’t seem to have addressed at any point the issues with corporate sponsorship of OSS which was kind of the central point I was making.

      1. 5

        I’ve written in one of the comments that if projects don’t receive sponsorship in some form, money or contributions from people on a payroll, such projects die. But here’s the other edge of that knife …

        Projects get built for 2 reasons:

        1. self interest, to solve your own problems
        2. to have users, either to sell a complementary or maybe because we like the thrills or we have some higher ideals

        When you have users, you cannot complain that those users don’t contribute. That’s not how this works. This is why I don’t like the usage of “Free and Open Source Software (FOSS)” to describe such work:

        • either you believe in Free Software, in which case the contributions have a sociopolitical factor to them
        • or if not then “Open Source” is already that neutral term to describe what you’re doing
        • projects can be both, you can say about Firefox that it is FOSS; but not people, i.e. you can’t say that you’re working on FOSS; your motivation perfectly describes the camp you’re in and you’d better have that motivation clear

        This is relevant because:

        • Open Source is driven by self interest; if projects die, it’s sad, but if you’re not paid to work on that project, you need to learn to let go at some point, it’s OK, you don’t owe anybody anything
        • Free Software is driven by ideals, but ideals don’t survive without a business model; so if your project dies, maybe it’s because the market doesn’t want it, so time to refocus

        As for getting money or contributors, personally I found that a good strategy is to just ask. OK, I know people that could barely make a living from contributions, but just ask is a simple and effective strategy that’s apparently not common knowledge.

        And there are always plenty of newbies lurking in the background willing to contribute, all they need is a little guidance which starts with a simple call for help. Well maintained projects are those providing guidance on what low hanging fruits exist for newbies ;-)

        The article also describes the self-entitlement complex of some users. Note for the wise: this happens with free services in general, the most annoying people, the most costly for support, being the ones that don’t pay.

        As a software author being too nice is a recipe for burnout. Don’t be an asshole of course, you’re not Linus Tolvards, but do enforce a professional conduct on all communication channels under your control. Remember that the community follows the leaders and their behavior. And also learn to say no to requests which you cannot honor.

        1. 6

          If you’re looking for an excuse to learn Haskell, there’s also Hakyll, a static website generator built with Haskell and it’s pretty nice: https://jaspervdj.be/hakyll/

          1. 3

            Hakyll sure does seem to be the go-to static site generator for Haskellers or people interested in Haskell, but I’ve found it to be somewhat complex, at times needlessly so.

            I’d recommend checking out Slick (successor to SitePipe), built on top of the Shake build system. There’s also yst but I haven’t had a close look at it yet.

          1. 15

            Most of the static site generators don’t seem to generate “sites”. They instead generate “blogs”, with the concept of posts and pages very deep-rooted in the implementation.

            I mention because I recently came across statik which is a static site generator written in Python which really lets you generate sites. You get to define the “models” which you’d like your site to have (if you define Post and Page models, you have something like pelican). Imagine Django, but compile-time (for the lack of a better analogy).

            I might write a blog (heh) post on this later, but I would definitely suggest checking it out if you’re interested in static sites.

            1. 6

              I maintain 3 websites, two with Jekyll (https://monix.io and https://typelevel.org/cats-effect/) and one with Middleman (https://alexn.org).

              Both Jekyll and Middleman are perfectly capable for static websites. The blogging part is just a nice built-in feature.

              1. 4

                I’ve been using Nikola (edit: for my landing page), because at the time it was the only one that had incremental builds. You have to follow their guide to reconfigure it for a non-blog setup: https://getnikola.com/creating-a-site-not-a-blog-with-nikola.html

                VuePress has my interest now, especially once Netlify support is implemented.

                Edit: I also have Sphinx instances: one as a public wiki and the other as a private notes repo.

                1. 4

                  The handful or so that I have worked with all support defining models, Sculpin and Tapestry (i’m the author) call them content types, Jigsaw calls them collections. All three can be used to generate a static site, but for convenience (and likely because the usual users are minimalist bloggers) they come “blog aware” which simply means they have a blog collection configured out of the box.

                  I have used all three and a few others such as metalsmith (which also supports collections via plugin) for the purpose of generating small static sites with a handful of pages for clients as well as reasonable sized multi author blogs.

                  TL;DR, yes some SSGs come shipped with blog content types (models) pre-configured but that doesn’t make them only good for generating blogs.

                  1. 2

                    This is interesting. I wish it didn’t use YAML though.

                    For the website, I ended up making a custom generator, and focus on blogs in most generators was one of the biggest reasons, even though not the only one.

                    1. 2

                      Most of the static site generators don’t seem to generate “sites”. They instead generate “blogs”, with the concept of > posts and pages very deep-rooted in the implementation.

                      Bravo for saying this. I’ve faced the same problem, with static generators forcing me to give an author / date setting for each page. This might make sense for blogs, but doesn’t for the simple site I want to build.

                      And most of them force you to do things a certain way; it is so painful, which is no wonder people just go back to Wordpress.

                      Amy Hoy wrote a great article on this : https://stackingthebricks.com/how-blogs-broke-the-web/

                    1. 8

                      Note that the “idempotence” definition that people use to describe side effectful actions is not the mathematical definition.

                      In math a function is idempotent if:

                      f(f(x)) = f(x)

                      This becomes very relevant in functional programming, as in FP the definition of idempotence is the math definition. Mentioning this in order to avoid confusion.

                      As always developers are overloading math terms in a way as to confuse everyone :-)

                      1. 1

                        it’s not confusing in this case. the analogy works perfectly.

                        1. 1

                          I don’t find the resemblance very intuitive. I was familiar with the meaning in TFA and got very confused when acquainted with the math definition.

                          Also in FP and in computer science in general the more correct term for executing effectful actions only once and then caching the result for subsequent calls is “memoization”.

                          See: https://en.m.wikipedia.org/wiki/Memoization

                          1. 1

                            correct term for executing effectful actions only once and then caching the result for subsequent calls is “memoization”

                            Memoizing actions which have observable side effects seems more like the sort of situation where you shouldn’t be using memoization. As I understand it the term usually implies an optimization that doesn’t change a program’s semantics, and that’s only the case when the function being memoized is referentially transparent.

                      1. 2

                        Publishers know what they are doing. Nobody cares about people wanting to block Javascript.

                        You can disable Javascript of course and at this point the web is still usable without it. However publishers will increasingly turn to protections against JS blockers, you can thank the increasing aggressiveness and popularity of ad blocking extensions for that.

                        1. 4

                          You can disable JavaScript but not with a usable UI, so practically most people cannot.

                          Also, JavaScript should be disabled by default, enabled on a per site basis and marked as “Not Secure” anyway.
                          Browsers should make SRI mandatory for JavaScript and they should take special attention to suspect HTTP headers used with scripts.

                          1. 1

                            Interestingly sites like ebay and amazon do work fine without javascript. Not quite as comfortable but no quirks there either. Ebay has gotten worse over the years I admit….

                          2. 2

                            Their is a fairly good compromise. I use uMatrix which blocks 3rd party scripts by default and gives you a ui to enable them as needed. Quite often it doesn’t break anything and when it does it’s usually super easy to work out that a script from a cdn is important but a script from twitter or google analytics is not.