1. 1

    One of the more useful comparisons I’ve read.

    I’ve been using Chef to deploy a reasonably complicated system recently and have found it to be a bit of a struggle. While Chef describes itself as a orchestration tool, I’ve found that this isn’t really what it is good at.

    It seems good at solving the “I have a software stack, and I want to be able to install it without caring about the underlying OS”, but not so good at “I have a number of different virtual machines, each performing a different task and I want to get them talking to each other, scale up and down and debug when things go wrong”, which in mind is the difficult bit of orchestration.

    It appears that Chef started as a tool for bootstrapping arbitrary boxes, hence all the abstraction around packages/cron/templating etc, and then tacked on the cloud functions when the cloud started taking off.

    What we’ve found is that we had to write all the “difficult” code ourselves, while only getting a minimal benefit out of the complicated abstractions Chef provides. Which brings me onto my question: do people have any recommendations for a cloud orchestration system, that:

    • Integrates well with cloud providers
    • Has a good framework for scaling systems up and down, and reclustering nodes? E.g I have a 4 node Cassandra cluster and want to double it in size.
    • Provides the useful file templating and package setup functions that Chef has to get initial installs working, but without the abstraction bloat of roles/environments/recipes/attributes/databags etc…
    • Provides tools for monitoring/debugging this system when things go wrong
    1. 2

      Great article (and illustrations), however I’m still not sure about how adding this level of abstraction really gives me an advantage when writing applications. Does anyone know of a good resource for explaining this?

      Comparing say Python to the given example:

      with open(raw_input()) as f:
          print f.read()


      getLine >>= readFile >>= putStrLn

      Sure, Haskell is more succinct, but Python doesn’t feel far behind. Is the benefit with Functors et al that one can elegantly include the error handling by using the Maybe construct?

      1. 4

        @moses did a good job explaining some of the benefits. These “abstractions” are really generalisations; they work over lots of things.

        So, benefits?

        First of all, your Haskell example is pure and referentially transparent. The compiler can now make those calls non-blocking because it knows about all effects (and GHC does). Awesome stuff but there’s lots of stuff out there that can talk about the other benefits of purity.

        But anyway:

        printFile :: IO ()
        printFile = getLine >>= readFile >>= putStrLn

        We’re working with the Bind/Monad class when we say >>=. In this case we’re working with IO but what’s interesting is that we can put Monads around other Monads. Let’s say we want IO calls that instead of throwing exceptions, returned values:

        -- safeReadFile will return a String in an error (yeah, urgh)
        safeReadFile :: ErrorT String IO String

        We’re going to assume getLine and putStrLn can never error (of course, a bad assumption) just for illustration:

        safePrintFile :: ErrorT String IO ()
        safePrintFile = lift getLine >>= safeReadFile >>= lift putStrLn

        We’re “lifting” our non-error IO functions into IO functions that always succeed. So together, safePrintFile returns the first error that is found, otherwise a successful value.

        We’re very easily mixing different “effects” together by stacking them, while staying pure and referentially transparent. Isn’t that cool!? This would be a lot more tricky in Python.

        Just one of the benefits of recognising generalisations and using them.

        1. 2

          I’m no functional programming maven, so I’m sure @puffnfresh or one of the other people on lobsters who knows functional programming well could give you a better answer, but as a python convert, what I found was that it was really nice to have a generalized version of some of my favorite things in python. For example, in python we use list comprehensions, dict comprehensions, and with as a poor man’s do loop (or in scala, for comprehension). Python has these built in, but with the monad abstraction, you can suddenly do for comprehensions on many more datatypes. Not just Maybes, or Lists, or Dicts, but we can make up our own monads. Just built into scala, some monads are: Either, Future, Traversable, Option. Some monads that I’ve seen people come up with outside of the scala standard library include Throwable, and ManagedResources. This means that instead of having to rely on the compiler to give you sugar like with, people are empowered to create it themselves.

          More specifically, the reason why people like things like the IO monad is because it banishes IO, which is inherently impure, as far away from the program as possible. You can think of the IO monad as gloves we can use to handle toxic waste. We prefer to keep things referentially transparent because state makes programs hard to reason about.

          The other nice thing is that code can get more legible when you just declare what you want to do with your data.

          edit tl;dr: To be clear, “with” is something that the haskellers would consider a functional abstraction, but they would pride themselves on being able to use it without it being a special language feature.