-
I appreciate the vagueness of this question, but: I can’t help but notice that virtually all non-C++ versions of capnproto do not implement the RPC interface. Is this because the RPC interface is especially difficult, because it’s not widely used, because it’s not trivial to invoke via JNI/ctypes/etc., or what?
-
I think a lot of people are happy just using the serialization protocol, as it does stand on its own. Cap'n Proto’s object-capability layer is somewhat more complex than other RPC protocols, but that’s because it includes support for killer features like promise pipelining. The Python and Node implementations demonstrate that it’s not too hard to wrap the C++ implementation. As sandstorm.io gets more popular, there should be a growing interest in RPC implementations in other languages.
-
-
It would be great if Cap'n Proto came out of beta and had support for a few more languages.
-
-
-
I’m currently working on a pure Java implementation, and I’m in fact writing the tests in Scala, which seems to work quite well.
For Scala code that uses capnproto-java, the main non-idiomatic thing I’ve hit is that unions don’t get to take advantage of sum types and pattern matching. I’m by no means a Scala expert, though, so I’d be interested to hear about any other potential impedance mismatches.
-
How does building up messages work? For idiomatic Scala I’d expect an immutable COW style, which gives you an awkward choice; either instantiating an object instantiates the serialized version (which means serializing copies of all the partial instances) or you only create the serialized version once everything’s been built (which means an extra copy stage). Maybe some kind of effect style is the right way to go, building up a monad that knows how to create the serialized instance and then running it to actually serialize? I can’t imagine you could do that in Java though.
Reading is easier as I’d expect to read immutable messages in Scala.
-
For an example of building messages in Java, see the
writeAddressBook()function here. In Scala, type inference would allow this to be a bit less verbose.You’re right that implementing copy-on-write would probably be tricky. Cap'n Proto works best if you’re writing to a message just once.
I do suspect that there are some clever things you could do in Scala to make things more ergonomic. It’d be fun to investigate the possibilities at some point.
-
-
-
-
-
-
here’s the how it works - https://github.com/kentonv/sandstorm#how-it-works
it’s an lxc wrapper like docker. or perhaps an alternative to lxc using cgroups? but with more of an api to write against and an emphasis on “apps” (which seem to be visible in a browser?)
-
Sandstorm’s containerization directly uses the
unsharesystem call (and others); there’s no dependency on lxc.Yes, apps are currently accessible through a web interface.
Apps will be able to define their own APIs via Cap'n Proto intefaces. This will enable apps to work together in ways more interesting than those available to traditional web apps.
See also the discussion on Hacker News : https://news.ycombinator.com/item?id=7460828.
-
-
dwrensha
edited
4 years ago
|
link
| on:
The Game of Distributed Systems Programming. Which Level Are You?
See also Cap'n Proto RPC (http://kentonv.github.io/capnproto/rpc.html), which implements an elegant distributed object system. Cap'n Proto deals what this post calls “level 1” problems by providing promise pipelining; RPC actions are asynchronous and get reified as events and wrapped as promises, allowing the Cap'n Proto runtime to optimize away unnecessary network communication, and giving the programmer a simple, honest abstraction.
(Full disclosure: I’m a contributor on the Rust implementation of Cap'n Proto.)
-
Since I want better control over epoll, I’m interesting in packing more than RPC. I originally went down the path of trying to use protobufs, then msgpack, then cap'n proto for a C app I’m building (not C++). I ended up running into tons of issues with stable C library support with all of them. Currently I’m using XDR (using rpcgen as a base) and it seems to be the best so far out of what I’ve tried. Another one that recently caught my eye is MQTT, but I’m not familiar with it yet.
If you’re writing in C++ (or Rust perhaps?), you’ll probably have a different experience.
-
One thing that’s interesting to me about this is the possibility of using OCaml for more serious network services work. That’s not to say that OCaml isn’t used for serious work already, just that something like Go, with builtin multicore and concurrency support is more likely to be adopted at this point in time.
So, anyway, with OCaml being a great language to begin with, having loads of benefits, and a (seemingly) growing community, how will it compete with Rust / Go? Can it compete with Rust and Go?
It can compete with Go (both are high-level, managed languages), however Rust has its sights on targets that are ill-suited to OCaml or Go.
Agreed.
With network based services (the target of my comment, my bad if it wasn’t clear), I/O is typically the biggest bottle neck, so, fairly efficient machine code compilation (which both OCaml, and Go have), should put them in the same “ballpark”, I’d guess for things like web applications and other similar workloads. This of course assumes a similar I/O model, which I believe can be the case. Rust uses libuv, right?
I’m just wondering if we’ll see an increasing number of “How my startup went from 50 Node.js servers to 2 OCaml Servers” anytime soon, cause if so, sign me up for hopping on the bandwagon first! :-)
No, Rust has abandoned its libuv-based runtime. The code for the old runtime library now lives here. It has largely gone unmaintained, though there is some ongoing effort to restore at least some parts of it.
Ah! Thanks for the pointer!
Does it still utilize event driven i/o, or have they moved on from that idea as well?allow me to apologize. I should have answered my own question by actually reading through all of the pointed to document
Some history.
Long ago, Rust was a very different language. It had a big runtime, and had language support for things like channels. This is because it was thought that language support was needed to make such things safe.
As the language matured, more and more things could be expressed safely by the type system. And so language features got pulled out.
One thing that did remain, however, was N:M threading. It was pointed out that a systems language without 1:1 threads isn’t much of a systems language, and so, support for 1:1 was added. Like any good programmer, if you have two things that differed just by interface, you make an abstraction. So we did. You could sort of start Rust in N:M mode, or 1:1 mode.
… but it turns out that interface isn’t the only thing. To make this work, we had to slow down N:M threading, to the point where they weren’t really that lightweight, all things considered. Plus, things like segmented stacks really affect your ability to FFI into C, as switching stacks is a cost you now have to pay.
So, the choice was made: 1:1 threads made the most sense for the stdlib, as you should always have the ability to use your OS’s features in a systems language. On top of that, it would let us kill our runtime, which has a huge benefit, and sped up our FFI.
Because Rust is such a low-level language, this kind of thing is really ultimately a library concern, too. There’s nothing stopping green threading from being added to Rust later, even as a library. So we doubled down on making 1:1 as great as we can, and green threads will come back whenever someone motivated enough to do so takes it on. Which may even be the team at some point, we’ll see.
It’s basically the same with evented IO: IO is a library concern, which means it doesn’t really block the language development at all. So we’ve focused on non-blocking IO at first. OSes complicate things: IOCP is a very different API than epoll or kqueue. There’s already a great Rust library, mio, which people are using for this kind of thing. It’ll grow just like any other Rust library.
Woah! Thanks so much for the great reply!
Any time. I think history is really, really important, and often explains a lot of things. I’ve been considering trying to write some sort of “History of Rust” thing, but never enough time…
I suspect it will not compete with Go for the popularity of Go appears to be nearly completely disconnected from the quality of the language (people like it because it is simple, not expressive). Ocaml, minus the O, has no built in subtyping in the sense most people are acustomed to, which is a strength but a big hurdle for many people.
But it might be able to compete on performance, which I am looking forward to seeing more on.
Simplicity is a strength, to be fair.
Le sigh – I have high hopes for those who like things like Scala but want something closer to the metal. But, you’re right. And, as Rob Pike has (allegedly – I don’t have a real citation) said, “[Googlers] … They’re not capable of understanding a brilliant language but we want to use them to build good software. …”
I mean, I guess, if Googlers can’t understand a brilliant language, then who can? /sarcasm.