1. 31
  1. 29

    It seems to be converging on exactly where we were a few years ago with regular unix processes. Oh, but see, unikernels are hardware isolated. So are unix processes. The MMU provides complete hardware isolation between processes. The fact that processes can interact and interfere with each other is enabled and mediated by the kernel. Somebody wrote code to let it happen because there was a need for it to happen.

    It does look a lot like just a ring shift. Lots of ring 3 processes mediated by a ring 0 monitor has now become lots of ring 0 processes mediated by a ring -1 monitor.

    1. 3

      Network performance is a big motivator for putting your application, IP stack, and drivers all in one place. A hypervisor and unikernel is one approach, the other common one is to run all of those things as a single user space process that controls the network card. In either case you probably need a fixed 1:1 thread to core mapping and some place to put all of the misc management code so it starts looking pretty similar.

    2. 14

      The “unikernels are entirely undebuggable” thing is…kinda just a tooling issue and far from inherent. Unikernels are fundamentally more debuggable than the current mess of a huge kernel written in C that runs processes, both because of the fact that unikernels run in a single address space and because they tend to use languages with sensible type systems.

      That nobody has written good stubs that allow for things like:

      pausing unikernel execution

      examining/modifying variables

      dumping/prettyprinting state of things like the unikernel’s tcp stack

      is absolutely a barrier to using unikernels in production today but is no inherent barrier.

      Take netstat as an example. Mirageos doesn’t have a netstat now – but finding the state of all open/listening/connecting TCP connections under mirageos is criminally easy, they’re in the data structures here, and the hardest bit is finding a clean way to interface this to the outside. It’s definitely a tooling issue, but not at all an unfixable one.

      1. 5

        It’s definitely a tooling issue, but not at all an unfixable one.

        How different does this look from an existing operating system by the end, though? OS’s didn’t have netstat at first either. Heck, back in the day they were unikernels, I had to reboot to switch between a game and a word processor. We got where we are now for a reason, what’s changed that makes moving back a good idea?

        1. 3

          MMU mechanisms and languages with good type systems/memory safety both provide isolation between bits of code – the difference is that the MMU can provide isolation between hostile and unknown bits of code as well (at the cost of making things more complicated if you want communication over the isolation that the MMU provides).

          The unikernel idea is pretty much just motivated by the case of:

          my OS kernel doesn’t need the bulletproof isolation that the MMU provides between kernel/userland because I’m the one running it and all the code on this kernel/userland system is mine

          – so why use MMU to provide isolation when just a language with a good type system will do? Leave the isolation between hostile/unknown bits of code to something that is much better at that task, like a hypervisor / microkernel.

          Also, losing the MMU barrier between kernel/userland burdens you with less performance issues – things like userspace/kernel-bypass networking and exceptionless syscalls are useful because systems software abstractions and interfaces that were fine in the 80s just can’t cut it with the performance of modern hardware.

          tl;dr: unikernels are nice because in many use cases the (useful and usually required!) userland/kernel isolation properties can be provided just with a language with a good type system and memory safety and don’t require a full-on MMU between userland and kernel.

          tl;dr tl;dr: unikernels are about redrawing isolation boundaries with more appropriate and specific tools, not about moving back to “no isolation”.

        2. 3

          What if the inevitable consequence of a debuggable unikernel is a Lisp/Smalltalk/Erlang runtime in disguise?

          1. 3

            Maybe something like Erlang on Xen?

        3. 13

          This rant from Joynet is not very charitable. If we stomp on every nascent technology, where would we be today? While unikernels still are in their fledgling stages and lacking tooling to, dismissing them holistically is short cited.

          1. 15

            I think he has two good points.

            1. Can we accomplish this with today’s tech? Is MirageOS great because it’s a unikernel, or because it’s a small simple system written in ocaml? Is it possible to write a small simple system in ocaml that’s not a unikernel and achieve the same benefits?

            2. Will the natural evolution of this new tech trend back towards where we are today? Will these simple and correct systems remain simple and correct, or will they become increasingly intricate and complex over time?

            It’s worth asking what’s really new and different here, and why that’s better, and whether it will remain better when it’s no longer new.

            1. 6

              I would love to see an in depth article or paper that focuses on the two points you dug out of the article in a somewhat more neutral fashion. I do agree with you that it is worth spending the time and effort to understand what tangible benefit a person/team/company can gain from a unikernel based approach. Personally, the jury is still out on the usefulness of unikernels in the long run, but Joynet of all companies should not be going and throwing a new tech stack under the bus.

              1. 3

                Will these simple and correct systems remain simple and correct,

                You can argue that Unix started out simple and correct. Hell, you could argue that Unix started out simpler than these systems.

                or will they become increasingly intricate and complex over time?

                Absolutely. These systems will become increasingly intricate and complex over time, as all systems tend to do.

                However, despite a complexity vs time graph with both a higher Y-intercept and positive slope, the coefficients are likely to be smaller. That is, you’ll ultimately arrive at a system that is less complex than what we’ve got now.

                I’m reminded of this now classic slide.

              2. 2

                I can’t tell if you’re mispelling it on purpose, but please: it’s Joyent.

              3. 5

                I was really put off by the tone of the article.

                1. 6

                  he’s right that you don’t get dtrace and so on, but on the other hand, the need for dtrace is largely driven by the profusion of complexity of the modern operating system. Saying “hey, instead of having eighteen layers of abstraction, how about we just try eleven?” may not get us all the way to nirvana, but it’s a better option than doing nothing at all.

                  1. 7

                    A unikernel is usually more layers of abstraction. Now instead of your userspace program inside the operating system, you run your unikernel inside qemu inside the operating system: https://youtu.be/ZBcMy-_xuYk?t=22m13s

                    The need for dtrace, etc is because systems are weird and hard. If you are using TCP, filesystems, etc, you’ll be using that logic in your application. A unikernel merely bakes it into one package rather than providing a clean layer of abstraction for you to reason about and debug.

                    1. 6

                      I would say it’s usually less, because the “usual stack” in this context is (metal -> hypervisor os -> vm -> application), and what unikernels permit is instead (metal -> hypervisor os -> unikernel). I agree that if you just slathered unikernels on top of existing os or vm stacks, you’d be adding more crap.

                      As far as dtrace et al, it’s my belief as a 35 year Unix veteran, and I think you’d agree, that our systems are weird and hard at least in part because there are so many abstraction layers in between a given packet and a given response packet, and many of those abstraction layers have been written at different times, by differently skilled developers, in persnickety languages, using inadequate tooling. And that in fact dtrace, et al., are immune-system-style responses to that problem; that the systems fall over so unceasingly, so dramatically, and so grotesquely that people literally spend their entire lives creating complex DSLs to try to figure out where the tire fire fell over this time.

                      But let’s pretend for a minute – and I admit, this glorious future is not here now, not yet – that we boot up a vm into a kernel which exhibits the M:N qualities and actor model of Erlang, the type system and provability of Idris, and the capabilities model of (e.g.) Pony; and that the hypervisor is also written in this or a similar language, and provides a clean API and (literally-)proven drivers.

                      Given that universe, what use would any normal human have for dtrace or any other deep surgical tool?

                      So I see the unikernel idea not as a replacement for processes, or as yet another abstraction layer like docker, but as a way to start making up for the obvious flaws in docker (and lxc, jails, etc.). It may be a long road, may never get there, but it doesn’t hurt to start trying.

                      1. 4

                        I agree that if you just slathered unikernels on top of existing os or vm stacks, you’d be adding more crap.

                        Note that “posix compatible” is often a selling point. Combining libos and app in the same address space doesn’t do much to reduce abstraction. (Of course, there are systems which forego compat too.)

                        1. 1

                          true, but that’s because the early providers want early adopters, and the way to get that is to be close to their current needs (or, like some popular programming languages, to suggest it’ll be in 1.0). The more adventurous attempts, like Ling/L and Mirage, forswear that pretty comprehensively. I wager the paradigm will eventually settle at a place where many of the useful posix features are either provided or simulated, and most of the crappy ones with underspecified behavior fall by the wayside.

                      2. 1

                        A unikernel merely bakes it into one package rather than providing a clean layer of abstraction for you to reason about and debug.

                        The clean layer of abstraction is the type system of whatever the unikernel’s written in.

                      3. 3

                        but on the other hand, the need for dtrace is largely driven by the profusion of complexity of the modern operating system.

                        But that isn’t quite true. dtrace is effectively a standardized way to do printf debugging, but more powerful, and printf debugging is valuable basically anywhere.

                        But let’s say you’re right, systems are too complex, which I mostly agree with, in what way are Unikernels actually solving that?

                        For most problems people hit, the bug is in their own code, not the OS or surrounding system, so removing all the surrounding layers doesn’t necesarily fix that and making it difficult to debug doesn’t make anything better. There is also this whole movement to move Unikernels into containers which I don’t really get, but at least you have all the debuggability of an OS there, I guess, but that is making the whole system more complicated.

                        1. 1

                          I would say that anything that reduces the mental load is important. Check THIS out: https://medium.com/@MikeRyan52/angular-2-first-app-post-mortem-b2b2b3618828 and now imagine that you have to understand all that nonsense, plus html, css, nginx, (say) node.js, (cough) systemd, …

                        1. 2

                          Unix systems failed to really respect the “principle of least privilege”; for example, both the idea of user-level right management and global filesystem visibility create a lot of ambient authority, making them unfit for secure – or, I believe, robust – system design. Many techniques are employed today to provide better compartimentalization. The approach that Joyent is pushing is an incremental layer over Solaris Zones, that is a thin-ish version of a monolithic Unix system with cross-instance communication at network level. This benefits, as the post claims, from the existing system management tools, but also, as the blog seems to omit, a non-neglectible amount of cruft and a large trusted base.

                          Unikernel are all over the place in terms of design, but at least MirageOS tries to solve those same system issues using more language-level compartementalization¹. This may or may not prove effective in the long run, but it’s a good idea that has many merits and should I think rather be encouraged than shot down. Unikernels allow rapid experimentation on this front, and this is good – we are finally turning the page on “Systems Software Research is Irrelevant”.

                          I find it frustrating that the author would compare the trusted computing base of the linux kernel (or Solaris for that matter) to the one of an hypervisor, and call the security argument for unikernels “security theater”. This is biased, unproductive name-calling, and, I think, plain wrong. There is a lot of interesting things in this post, but it seems unable to get over its (natural and understandable) competitor’s rivalry bias.

                          ¹: It is important to note that application boundaries could lie either inside a given unikernel, with separation provided by language-based mechanisms, or outside it. Both are interesting options with a rich design space – for example, inside-unikernel communication allows to share a GC across instances, which could have either positive or negative side-effects. Again, the post stumbles over these details by merely mentioning that separate unikernel instance do not share their memory resources.

                          1. 1

                            Upvoted even though I disagree with lots of it (needing to program in OCaml is bad?). Wish it were more even-handed, but worth considering both sides of the coin.

                            1. 3

                              I did not interpret him as saying needing to program in Ocaml is bad, but rather that you need everything in Ocaml which can be prohibitively painful in some cases.

                              1. 1

                                you can deploy a surprising number of existing open source software in unikernel form without a single drop of OCaml.