1. 18

  2. 8

    We use mypy extensively at work. The performance critical parts of our code are in C, and then the rest of the system is in Python.

    We’re in the process of migrating the system from Python 2 to Python 3 with full type hinting and compile-time checking with mypy.

    The type system is elegant and, thanks to its deep integration into Python’s standard library, can perform some semantic checks too (e.g. if you pass a static string to open for the mode the type checker will infer whether the stream is of binary or text type rather than a union of the two).

    The type system supports flow typing, union types, generics, classic subtyping…it’s pretty great. Not quite as good as Ceylon’s type system, but no one’s is. :)

    The cases it still doesn’t handle well:

    • function decorators often cause type erasure
    • there are some issues with typechecking the value side of dicts
    • recursive type support is iffy
    • subtyping doesn’t always ensure you’ve called the necessary init functions via super (or any other way), so mypy will pass a program that dies at runtime due to incompletely initialized objects
    • the things mypy does to check variable assignment/liveness don’t line up completely with Python’s runtime rules, which can cause it to miss potential issues. For example, it doesn’t always catch variables being used before their assigned.

    The big decision when this whole phase of the project started was to ask ourselves if we should simply move to a more statically typed language like Go or Ocaml or C++. We decided to stick with Python and mypy because of the sheer amount of library support out there for the kinds of things we want to do and the amount of experience we have in Python in-house. Our performance sensitive code is already in C, so it definitely made sense in our case.

    1. 1

      Python 3 appears to me too have a more sophisticated type system than Go.

      1. 1

        Unless I’m misunderstanding how python’s static typing works, Go’s interfaces are far ahead of Python’s attempt.

        Go seems like a formalization of duck typing, while python encourages duck typing but doesn’t offer an easy way to check if something actually walks and quacks correctly.

        1. 4

          Well, Python itself doesn’t have static typing at all. The syntax allows for type hints, but they’re not enforced by the language. Mypy is a separate utility that takes type-hinted Python source and reports type errors, but there’s nothing stopping you from running mis-typed code.

          Mypy does handle “interfaces” via abstract base classes (the abc module), and Python (via a bit of metaclass chicanery) does enforce part of that defined interface at runtime.

          Mypy’s type system is quite powerful and can detect typing errors that Go’s type system can’t. For example, Go doesn’t have union typing, so you have to do type assertions and can check for types that would never actually end up in that spot in the code, whereas Mypy’s union- and flow-typing can statically verify that you’ve got the types you expect when you’re doing type assertions.

          Ultimately, though, Go is statically typed and Mypy isn’t. There is no such thing as a “valid Go program that is mis-typed according to Go’s type system” but there are infinitely many valid Python programs that are mis-typed according to Mypy. Mypy is perhaps best thought of as a mostly-type-aware linter rather than a “compiler” or whatever (it’s definitely not a compiler, since it doesn’t produce code).

          1. 3

            abc’s and go interfaces are not quite the same thing.

            abc’s require explicitly subclassing them, a-la most OOP lang interfaces.

            Go’s interfaces are implicitly satisfied if a type has the same method name and types. This closer matches duck typing, which python is supposed to be more based around.

            1. 2

              this comes closer to go’s interfaces: https://www.python.org/dev/peps/pep-0544/

              1. 1

                Excellent, this is almost exactly what I wanted!

    2. 5

      I really wanted to like mypy (so much so that I contributed the dataclass support!). We used it for about 6 months at work, but dropped it because it was too slow, spelling out the types can be cumbersome and the little inference it provides is weak and flaky (often you’ll have to explicitly spell out the type of an instance variable despite it being assigned the value from a typed binding in __init__, for example). Our unit tests provided more safety and caught more issues (which is to be expected). It never made refactoring feel safe – which is one of the things I look for in a static type system – in part due to the nature of gradual typing.

      The only reason I don’t drop it from molten (and likely never will) is because I don’t want to break users’ code.


      1. 1

        could you give pytype a try and see if it fits your use case better? (disclaimer - i work on it). you might end up having the same list of gripes, but one of our differences from mypy is that we do a lot more type inference without your needing to supply types yourself.

        on the down side, if your reference to dataclasses means you’re already on 3.7, we don’t have full support for a lot of the newer python3 features (it’s a work in progress, but pytype was developed in the context of google’s large 2.7 codebase, so python3 support was not a priority till last year)

      2. 2

        I once had a student who wrote a nice benchmarking tool. He used mypy for that and was a fan afterwards.

        1. 2

          I’ve been writing Python for almost 20 years now, and using mypy+ale in Vim was the most notable change in my workflow. I like how it allows for gradual typing and how the type definitions are embedded in the language (ie. as a module). I tried it on an experimental project with lots of early-stage refactoring, and the benefits were very clear. The main drawback I’ve noticed, though, is that you can’t refer to classes/symbols defined later in the source code, so you have to put their type name as a string, instead. Apparently, this is being worked on.

          1. 1

            String annotations by default are coming as a future import in 3.7:

            from __future__ import annotations

            Apart from solving the forward-reference problem, this speeds up imports.

            ALE looks awesome! Thanks for mentioning it.

          2. 2

            I wish something like this could be stitched together with hask.

            1. 1

              So something like Idris?

            2. 1

              mypy is great. It’s definitely saved me from a lot of silly mistakes.