1. 45

  2. 6

    Although I’m told there are good reasons for it, the fact that Scala 3 is developed and maintained in a separate repo than Scala 2 continues to concern me. That ecosystem has clearly been good for research and contributions from post-docs, but I worry it isn’t as good at avoiding regressions.

    The natural consequence of the divergence has been increased contribution overhead to the compiler (two PRs over one), which leads to some patches only ending up in one version. Also, since some of Scala 3 was developed from scratch, there are still a lot of Scala 2 features not in Scala 3 (I think the entire optimizer is gone, the REPL functionality is different, there are still compiler options missing). Going through the git history is also not particularly fun: you sometimes end up in massive patches that have commit messages like “copy files over from Scala 2”.

    1. 6

      Zig article -> 44 upvotes, Scala 3 -> 14 upvotes.

      I am not sure what Scala is used for anymore. F# / Rust / Kotlin and probably OCaml ate away much of the user base over time. If you are an ex Scala coder, what language did you switch to?

      1. 11

        There’s 18 Apache projects written in Scala

        • Apache CarbonData
        • Apache Clerezza
        • Apache Crunch (in the Attic)
        • Apache cTAKES
        • Apache Daffodil
        • Apache ESME (in the Attic)
        • Apache Flink
        • Apache Hudi
        • Apache Kafka
        • Apache Polygene (in the Attic)
        • Apache PredictionIO (in the Attic)
        • Apache Samza
        • Apache ServiceMix
        • Apache Spark
        • Apache Zeppelin

        There’s some big names there (Spark, Kafka, Flink, Samza), especially in data movement. Also Netflix has atlas (time-series DB), Microsoft has hyperspace. Seems like most Scala code is associated to Spark in one way or another.

        1. 2

          Huh, I thought Kafka, Flink and Samza were written in Java. Shows what I know. Neat link!

          1. 2

            This overstates the case for Scala a bit. Check the founding years of these projects. Kafka and Spark, which are two of the most popular projects in this list, were created in 2011 and 2014, at the height of Scala popularity. Both projects were written in Scala, but had to put significant effort into engineering a first-class pure Java API. Kafka team even rewrote the clients in Java eventually. GitHub repo analysis has the Kafka codebase as 70% Java and 23% Scala.

            It’s true that Spark does use Scala a bit more. GitHub there has Scala as 70% of codebase, with Python 13% and Java 8%. But Spark might just be the “perfect” use case for a language like Scala, being as focused as it is on concurrency, immutability, parallel computing, higher-order programming, etc.

            I also closely tracked development of Apache Storm (created 2011), and it started as a Clojure project, but was eventually rewritten (from scratch) in Java. There are lots of issues with infrastructure software not sticking with vanilla Java (or other “systems” languages like C, Go, etc.). Apache Cassandra and Elasticsearch stuck with pure Java, and had fewer such issues. Durability, simplicity, and ecosystem matter more than programming language features.

        2. 8

          It’s still pretty big in data engineering. Apache Spark was written in Scala.

          1. 6

            The company I work for uses Scala for data engineering. I don’t think that team has any plans to move away from it. I suspect that the use of Scala is a product of the time: the company is about ten years old; Scala was chosen very early on. It was popular back then.

          2. 7

            Elixir and loving it for almost 6 years now. I miss a couple of Scala’s features; in particular, implicits were nice for things like DI, execution contexts, etc. I don’t miss all the syntax and I certainly don’t miss all the Haskell that creeps in in a team setting.

            1. 6

              If you are an ex Scala coder, what language did you switch to?

              Python, now Go. At one point I figured I could ship entire features while waiting for my mid-sized Scala project to compile.

              I hope they addressed that problem.

              1. 3

                YAML in mid 2015 (I took a detour in infrastructure) but Clojure since 2019 now that I’m a dev again.

                FWIW I liked Scala as a “better Java” when I started using it around mid 2012, until early 2015 when I left that gig.

                I remember that I found it very difficult to navigate Scala’s matrix-style documentation; and that I hated implicits, and the operator precedence. I loved case classes, and I think I liked object classes (not sure that’s the right terminology). And I liked that vals were immutable.

                Compile times didn’t bother me that much, perhaps because I worked on early-stage greenfield projects with one or two other devs. (So didn’t have lots of code.)

                I liked programming with actors (we used Akka) but we found it difficult to monitor our services. Some devs were concerned about loss of type safety when using Akka actors.

                1. 2

                  Akka actors are type-safe now for several years now.

                  1. 2

                    Off topic: there’s a lot of conversation/blog posts to be had about devs finding themselves in infrastructure. I’m there at the moment, and it’s a very different world.

                  2. 3

                    I’m using Scala for my hobby projects. It does all I need. I like multiple argument lists and ability to convert last argument to a block. Implicits are cool, though they need to be controlled, because sometimes they’re confusing; for example, I don’t really understand how uPickle library works inside, even though I know how to use it. Compilation times are not that bad as some people say; maybe they were bad in some early Scala versions, but they’re not as bad as e.g. C++. It works with Java libraries (though sometimes it’s awkward to use Java-style libs in Scala, but it’s the same story as using C from C++ – it’s a matter of creating some wrappers here and there).

                    1. 3

                      I wrote several big scala projects a decade ago (naggati, scrooge, kestrel) but stopped pretty much as soon as I stopped being paid to. Now I always reach for typescript, rust, or python for a new project. Of the three, rust seems the most obviously directly influenced by (the good parts of) scala, and would probably be the most natural migration.

                      Others covered some of the biggest pain points, like incredibly long compile times and odd syntax. I’ll add:

                      Java interoperability hurt. They couldn’t get rid of null, so you often needed to watch out and check for null, even if you were using Option. Same for all other java warts, like boxes and type erasure.

                      They never said “no” to features. Even by 2011 the language was far too big to keep in your head, and different coders would write using different subsets, so it was often hard to understand other people’s code. (C++ has a similar problem.) Operator overloading, combined with the ability to make up operators from nothing (like <+-+>) meant that some libraries would encourage code that was literally unreadable. Implicits were useful to an extent, but required constant vigilance or you would be squinting at code at 3am going “where the heck did this function get imported from?”

                      1. 2

                        2 days later, Scala 3 -> 44 upvotes ;)

                        1. 1

                          I’m currently in Scala hiatus after nearly eight years as my primary stack with a splash of Rust, Ruby, Groovy, and a whole lot of shell scripting across several products at three companies. This included an IBM product that peaked near $100M/yr in sales. The major component I managed was JVM-only and 3/4 of it was Scala.

                          For the last few months, I’m working in Python doing some PySpark and some Tensorflow and PyTorch computer vision stuff. While I concede that the Python ecosystem has certainly matured in the 15 years since I did anything material with it, my preference would be to rewrite everything I’m writing presently in Scala if the key libraries were available and their (re)implementations were mature.

                        2. 4

                          There’s a lot of stuff in here that seems interesting – (optionally) removing Null from the type hierarchy is perhaps the coolest, although the enum changes are also pretty neat. I still wouldn’t choose to use it, but it seems like a lot of the friction between FP and OOP is being reduced.

                          1. 5

                            it seems like a lot of the friction between FP and OOP is being reduced.

                            I suspect this is in line with Odersky’s past assertions regarding the duality of FP and OOP. I’ve heard him interviewed on the topic and he essentially boils it down to a “two sides of the same coin” sort of position.

                            1. 5

                              Javier Casas has a post about it from a category theory perspective.

                              1. 3

                                My previous beefs with Scala had to do more with the aesthetics of the language – it was totally Java++, which nobody wanted to hear. But it seems like they are paring down from “anything goes” to a more opinionated subset of functionality, which is great. Again, it wouldn’t be the first tool I reach for, but I’m not a programmer any longer, really, so take that assertion with the appropriate grains of salt.