1. 16

While reading yesterday’s link to the FastMail blog post, “Email is your electronic memory”, I followed the link in the post to the JMAP protocol (discussed three years ago here, on HN, and most recently LWN).

In a nutshell, JMAP is an IETF draft protocol for “synchronising JSON-based data objects efficiently”, implemented initially as more efficient protocol for synchronizing email between servers and MUAs in lieu of IMAP.

As a mail-client protocol, in addition to being an efficient RESTful-like RPC (JSON + HTTPS), it has other features to recommend it, including mail submission, support for GMail-like labels (i.e., membership in multiple folders), and extensions to support contact and calendar/todo management in lieu of CardDav and CalDav. (N.B. JMAP is not an MTA-to-MTA replacment. It’s a client protocol only.)

JMAP is not without controversy as a mail-client protocol:

  • It’s a structured protocol that almost requires libraries to and tools to interact with. (Yes, JSON is text based so you can handcraft the requests and parse the replies. See examples.) For developers, structure is very good. But for sysadmins wanting to test a server quickly, it’s hard to beat nc or openssl s_client.

  • It doesn’t handle binary objects (e.g., attachments in email). They’re referenced using blob ids that have to be pulled separately to avoid having to encode them in base64 (or similar).

The project has come along quite a bit since last discussed. There are server implementations, an IMAP proxy, client libraries, and a sample web front end available for testing. Perhaps it’s time for another review and discussion.

  1.  

  2. 5

    Unlike SMTP I never really bothered to look how IMAP works and simply assumed it’s quite ok, since it was much better (from my end-user experience) than pop. A few years back I read My email made me miserable. So I decided to build my own email app from scratch and was really surprised.

    I also remember friends from emclient talking about really crappy implementations of many servers (like giving you dates in really weird formats and many other quirks).

    I guess some new protocol would be really nice then.

    As for other proposed functionalities like contacts, calendars, todos… it’s not that easy either. Yes, more efficient syncing would be nice, but there are more pressing problems. Have you ever tried syncing contacts across platforms and get consistent results? Every platform has slightly different data models for these things and not always reliably compatible. I’m not sure new protocol itself will fix this.

    1. 4

      They should have used canonical S-expressions instead of JSON: they are simpler to parse & emit; they are better-suited to handing encryption; and they readily handle binary data.

      It’s a matter of taste, but I also think that they’re a lot more attractive:

      (request
       (using ietf.org/rfc/smap-core ietf.org/rfc/smap-mail)
       (method-calls
        (method1 ((arg1 arg1data) (arg2 (arg2data))) "#1")
        (method2 ((arg1 arg1data)) "#2")
        (method3 () "#3")))
      

      vs.:

      {
        "using": [ "ietf.org/rfc/jmap-core", "ietf.org/rfc/jmap-mail" ],
        "methodCalls": [
          ["method1", {"arg1": "arg1data", "arg2": "arg2data"}, "#1"],
          ["method2", {"arg1": "arg1data"}, "#2"],
          ["method3", {}, "#3"]
        ]
      }
      
      1. 2

        Any particular reason not to go for ... (method1 (arg1data arg2data) "#1") ...?

        Then, the attractiveness thing sort of gets lost when you go for actual canonical S-expressions with binary data, doesn’t it? No more looking at the raw expressions in a text editor.

        And is JSON really that hard to parse?

        1. 1

          Any particular reason not to go for ... (method1 (arg1data arg2data) "#1") ...?

          I was just following the original style directly. Certainly a more common way to write that in a Lisp would be the way you indicated.

          Then, the attractiveness thing sort of gets lost when you go for actual canonical S-expressions with binary data, doesn’t it? No more looking at the raw expressions in a text editor.

          I dunno, this looks pretty good to me:

          (cert
               (issuer (hash sha1 |TLCgPLFlGTzgUbcaYLW8kGTEnUk=|))
               (subject (hash sha1 |Ve1L/7MqiJcj+LSa/l10fl3tuTQ=|))
               …
               (not-before "1998-03-01_12:42:17")
               (not-after "2012-01-01_00:00:00"))
          

          The only bits which are binary are the hashes, and the rest of the expression is fine. The whole thing can be edited in a text editor, if necessary.

          And is JSON really that hard to parse?

          No, not really — but it’s still more complex than S-expressions.

      2. 2
        1. 2

          What existing standard is JMAP competing with? The only protocol I’m aware of that already does what JMAP proposes is Exchange ActiveSync, which is proprietary and Microsoft charges a licensing fee for even clients to implement it.