Part of Joe Duffy’s series on Midori), a research operating system from Microsoft.
This whole project is impressive and amazing, so I won’t pretend to know enough about it to really make any kind of informed statement so the following is really more about thinking about things rather than critiquing the design choices.
I was surprised that they ended up sticking with something with async and await keywords. In a the various other languages that have this, it makes sense because they have a lot of backwards compat to deal with. In this, they were building from the ground up so it seems like they could do basically anything they wanted. I expected them to land on something more like Go or Erlang where the possibility of a context switch is implicit. I’m not sure which is better or not. The keyword based solution has the benefit of being explicit about when a context switch might happen but it has the downside of this information having to bubble up, which can be annoying if you end up changing something really deep down to now be async. On top of that, an infinite loop can kill your program if it doesn’t perform an await. In Go and Erlang, the runtime can decide to context switch whenever it wants, so loops won’t block anything.
What I think is a bummer about async/await is that they are explicit implementations of a particular monad. With a little bit more generality, one could implement a richer system for expressing monads and get a lot more benefits out of the language.
A lot of effort is put into zero-copy but I wonder how expensive the indirection is in the end. The blog post said that you end up getting a proxy object, so that will probably kill any cache locality benefits. However, the upside of the default being zero-copy is that you can explicitly decide to copy things when it’s important.
I think there is a lot of work that can still happen here. Cancellation is just really hard. Erlang gives an abstraction that lets you kill an Erlang process but even then you have to write the code to deal with cleanup, which sucks. In cooperative multithreading it’s much harder, though, IME.
I’m an FP snob, so the usage of objects in here felt a bit weird to me. There were objects, and message passing, and it seems like you should only need one of those. I think the message passing setup was awesome though, each process exposes a message interface. That’s just what I want from my OS. But what’s it mean to have message passing and then objects inside of the process that get exposed via a proxy object? I’m just not sure that is a good abstraction. I’m very used to solving problems with actors and I think that is generally what makes sense in these cases.
I’m working on an async library for Ocaml right now, for fun, and several of the ideas in here matched what I’m thinking which makes me feel like i’m on the righ track. I’m really enjoying these Midori posts. In a sense, I’m glad that all we get to see about it is this little glimps through a blog post, keeps my imagination going about how the world could be a better place :)
Hm. The Markdown parser seems buggy. The generated link for [title](http://foo)) is http://foo, not http://foo) as expected, and the closing paren is rendered as text outside the link.
is that a bug? I’d expect that to be how it would work.
I love these articles! I do wonder why MS didn’t open source it though. Only two reasons come to mind.
1) They plan on selling it.
2) Old school mentality says never open source anything.
I don’t think it’s #1 or they wouldn’t like Joe put out these exposés on the system’s internals. I don’t think it’s #2 either, though, because MS has in recent years become very cool with the idea of open sourcing projects. So what am I missing?
They may not own all the IP, and it might be too expensive to get the rights to open source it from the owners. That’s usually the case in these sorts of situations.
They’ve been cool with open-sourcing projects they’re actively working on, but so far as I know they haven’t released the source for older dead projects. The view is probably that open-sourcing a project takes a nonzero amount of effort to make it ready for a community, and Midori isn’t worth any effort.
That or Midori isn’t dead yet. That would be nice.