Did the article say they have no code review? It looks possible to run into this problem even with that in place.

Scalaz can really complicate code and some Scala teams discourage or prohibit its use. I admit that I don’t have a lot of experience with it, but perceive it is powerful and useful, albeit a bit arcane, and believe that it increases the learning curve of Scala greatly when used.

Scala permits the use of type parameters, kinda like Java generics. In the article example, Foo[F[+_] : Monad, A, B] means that a Foo has three types that it cares about that are defined at compile/runtime: F, which holds anything (that’s _) and is a Monad, and A and B.

When a Foo is instantiated, its constructor takes a few parameters:

• an execute parameter takes a function that takes a Foo.Request[B], a Foo.Request that cares about B, and produces an F[B], or an F that cares about B
• a joins parameters takes a function that takes a Foo.Request[A], a Foo.Request that cares about A, and produces another function that takes a B and produces a list of Foo.Request[A] objects.
• lastly, an implicit parameter J that is a Foo.Join[A,B], or a Foo.Join that cares about A and B.

A and B are consistent and the same class throughout the entire lifetime of the particular Foo object.

That’s all the farther I feel like typing out at 0222 hours ;-)

Some Scala people really like arrow notation and all kinds of single letters for type notation, in order to convey that they are type parameters and not actual types. I consider myself perhaps among a small number of heretics that favor readable code, and use full words for type parameters, commonly In, Out, and SomethingElse, if a third, middle state is required.

For instance, a piece of a program I’ve been working on looks something like this:

trait Chainable[In, Out] {
  def andThen[SomethingElse](next: Chainable[Out, SomethingElse]): Chainable[In, SomethingElse]
  def apply(thing: In): Out
}

If val barifyFoo: Chainable[Foo, Bar] and val bazifyBar: Chainable[Bar, Baz], then I can do val bazifyFoo = barifyFoo andThen bazifyBar and call val bazified = bazifyFoo(foo) with the same eloquence as something like val bazified = bazifyBar(barifyFoo(foo)) and let the implementation of andThen perhaps work out any shortcuts or concurrency.

This might otherwise be written as trait Chainable[A, B] { def andThen[C](next: Chainable[B, C]): Chainable[A, C] def apply(thing: A): B } Which of the above is more readable for this simple trait building a chain of associative operations? Which is more easily explained to a wider range of developers, many of whom are unfamiliar with such type… magic? I’ll tell you through two years of experience teaching Scala that being explicit yields much more readable code at the expense of a few more keystrokes.

To add to @colindean’s explanation:

The implicit Foo.Join[A, B] is the idiomatic way to constrain the relationship between the types A and B, or that we know how to perform some kind of operation that’s supported for some pairs of types but not others. This can get complex, but offers power that you don’t get in Haskell. A better alternative is probably a full dependently typed language with native support for type functions e.g. Idris.

({type l[+a]=WriterT[F, Log[A, B], a]})#l is a pattern for expressing a partially applied type (unfortunately scala doesn’t have type-level currying, unlike Haskell). There’s a compiler plugin that would let you write it more clearly as WriterT[F, Log[A, B], ?]. TraceW is a similar type alias. The type itself is a standard monad transformer, WriterT: it’s saying we’re going to work with (functions to) pairs of a log and a value inside some context F[(Log[A, B], ?)], and we want to compose them by composing F in its native fashion and combining the logs in their native fashion. There is certainly something to learn here, but it’s an extremely general, reusable pattern, compared to e.g. annotation/AOP-based logging/tracing generators that are (IME) often equally complicated to understand, but much less general.

The name :++>> is indefensible, but looking at the function what it does is very simple: it takes a function from a value to a log entry and adds that entry to the log, keeping the value the same. So execute returns a log containing just List(request -> request.response(repr, self.joins(request)(repr))).

Looking at it this does seem unnecessarily complicated. I think it could be defined as just:

def execute(request: Request[A]): WriterT[F, Log[A, B], B] =
  WriterT(self.execute(request) map {repr => (List(request -> request.response(repr, self.joins(request)(repr)), repr)})

If I knew what joins did (it’s not in the given code) it might be possible to simplify further. We could expand the one-liner (e.g. name the log value rather than immediately putting it in a pair with repr) if that would clarify, though IMO people underestimate the cost of verbosity and it’s easy enough to mentally unpack these things.

[Comment removed by author]

I don’t think this article is about Go. The point here is the Scala language (and a popular library) made it possible for the code to be opaque to his own teammates (who were also Scala programmers!).

Choosing to use Go going forward is one possible conclusion. If the article said that they chose to just use vanilla Java 8 instead, I think the takeaway about the dangers of clever code would be the same.

I remember when I first saw the potential issues of scaling Scala at Gravity back in 2009/10ish. It was close to the end of the day when we had a major issue reported in production that was affecting our large customers. Several of us started investigating and were able to track the source of the issue. The only problem was we had no idea what the code was doing at first. We came across a strange symbol we hadn’t seen in our projects before. The spaceship operator |> . Someone said out loud “what the hell is that?”. There was some other implicit magic going on that wasn’t immediately apparent. A CMD+B to traverse into the method yielded nothing in our IDE as it couldn’t find the symbol (IDE’s have improved here since). A quick googling of “|>” yielded nothing as well. We were stumped and didn’t have sources pulled down.

The developer who wrote the code was unreachable on vacation so we had to figure it out. We noticed a new library was being included called scalaz, hours later we had tracked down the mystery symbol and grok’d what was going on, made the fix and life was good again. That blip turned a fix that should have taken minutes into a fix that took hours. That was the point I started seeing the split in our engineering team.

As an aside, it reminds me of how many Java programmers have trouble with Guice because it makes it very difficult to understand how code actually fits together. Yet another dependency injection framework (Dagger) tries to fix this by generating codepaths you can manually inspect.

I think he mentions this with the difficulties with scala, but that there is more to learn with the jvm and how to tune it than with the golang runtime (which is fairly simple aside from GOMAXPROCS)

Whenever people say they’re resistant to a new language because it lacks IDE support, I want to fucking scream.

Why so? I dont mind hacking C++ and scheme on emacs for leisure, but when you’re picking the tools to create a project, people want (need?) to be productive, and proper tools (read IDEs, usable debuggers, autocomplete, …) is a must-have, not a nice-to-have, for that. I don’t get the hate towards people who want proper tools :)

Implicits are also painful to debug, SBT is a nightmare, and the language feels like Magic: the Gathering where if you leave for a year and come back to it, there are all these new rules that have changed the game entirely. Then there are the slow build times.

The language has slowed down a lot in response to this kind of criticism. Five years ago was very different from three years ago, but I’d say it’s very much the same language it was two years ago.

SBT is awful. I don’t know why people use it. Just ignore it and use maven.

Implicit debugging is relatively well-supported in IntelliJ these days, or with -Xlog-implicits on the command line.

Build times are better than they were, though that’s always relative.