1. 3

I wrote this trying to implement a lazy stream in Python after being inspired by seeing the implementation in Scala. Appreciate any feedback or commentary.

  1.  

  2. 2

    Cool article! I think it really shows how close you can get, despite the syntactic overhead of Python’s lambda.

    The only thing I may add is a word of warning that streams a la Scala are a very bad idea in general.

    They basically combine

    • a generator that can create an infinite amount of elements
    • a memoized, singly-linked list-like data-structure
    • with deep integration into the collections API.

    This means it comes with

    • the added possibility of non-termination in every collection operation on more general types like Seq,
    • a massive danger of memory leaks if the head isn’t dropped in each and every place it is required,
    • a horrid amount of implementation complexity, not only because you have to play defense against every implementation supertypes “helpfully” provide, but also because it’s really hard to avoid evaluating thunks unnecessarily.

    Scala’s Stream should have been deprecated and removed years ago.

    1. 1

      Thanks!

      Interesting take on why streams are a bad idea. I haven’t used them in practice in Scala, but in other languages like Python generators are very common (in Python 3 range, filter, map, etc. became lazy by default), and I think in node streams are used quite often. So I’m curious in Scala if streams are to be avoided, then whats the alternative? What about Haskell?

      I do realize the danger of memory leaks is something to be aware of. Avoiding thunk evaluation was also something I didn’t try since the first call on filter would always be evaluated. So I can see that the full implementation might get complicated.

      1. 2

        I haven’t used them in practice in Scala, but in other languages like Python generators are very common […]

        There is nothing wrong with generators in isolation – Scala even ships with a lot generator functionality on the Iterator object.

        The problem is that Scala married functions that can generate an infinite amount of values with a memoizing data structure.

        The Stream companion object comes with pretty much the same “generator” feature set that Iterator provides, except that while everything Iterator produces is ephemeral, everything that Stream produces is retained if no care is taken to drop the head at the right places (which is non-trivial).

        So I’m curious in Scala if streams are to be avoided, then what’s the alternative?

        I’d stay away from the eagerly evaluated collections operations and try to use views; they have been too buggy before 2.13 to be usable, but things may have improved after the rewrite in 2.13.

        The exact brand of laziness Scala’s Stream offers is hardly ever useful – I haven’t found a single use-case during the time I worked with/on the language.