Interesting project, but this marketing BS is not a good link for Lobsters. Here are the very basic questions I had about Nanos:
Does it descend from some pre-existing system?
What language is it implemented in?
How is it licensed?
Who supports its development, and what is their business model?
To answer all but the last of these, I had to click around and find the github repo. The FAQ on this marketing site is pretty vacuous. “Syscalls: N/A” lol.
Anyway, I think I’ll stick with https://mirage.io/ for my unikernel needs. But thanks for sharing!
Anyway, I think I’ll stick with https://mirage.io/ for my unikernel needs.
I only just looked into this stuff but… all of MirageOS’s peers are dead. So I think I welcome anyone new in the space. We need progress and competition!!!
Good point, but it perhaps says something about the unikernel concept itself and how it’s played out in practice. Overall, I agree, and I’m happy to see Other People’s Money being spent on a technical subject I personally find interesting. I sincerely wish the nanos folks all kinds of success.
At the same time, I must admit to having some doubts about the wisdom of implementing a unikernel with a completely fresh code base in a famously unsafe language, but hey, I guess we’ll see how it plays out.
The money point is important to point out. It’s not something you can just whip together in a weekend and not something I could see happening without lots of $$ to employ full time engineers on. Even then the difference between hello world and production usability is a huge gulf to cross - most teams unfortunately die before they can cross it. That in my most humble opinion is the biggest problem in the ecosystem. Government grants can only take you so far.
I hear your complaint against c. We’ve discussed doing other complementary projects in rust but the pkg/dep system is a major turn off and none of our engineers speak rust either other than hello world projects. Likewise $$ again is a prime concern for working in that ecosystem. When you have tiny little saas companies that raise tens of millions and employ tens to hundreds of software engineers you have to ask what is the true cost of a new language or a new operating system?
All very cogent points, and be it far from me to armchair-quarterback anyone’s business strategy. Application support is probably the first priority for a real-world unikernel, and that drags in legacy APIs and other ecosystem factors that we can’t really control. Latent security problems only become critical when you actually have some customers.
I’ve had a corner of an eye on this space for a little while, but I wouldn’t claim any expertise. To me, it seems like Mirage has carved out a bit of a niche and, like many open source projects, is sort of puttering along just fine with little in the way of sponsorship. But its adoption is of course limited to the (relatively tiny) Ocaml ecosystem. Then there’s the rump kernel approach, which largely piggybacks on NetBSD’s flexibility and mature-ish code base. Looks like there’s some consultancies promoting that. Probably works great for some applications. Doesn’t appear to need VC, at a glance.
The only unikernel I have had any real first-hand experience with is HalVM. I can assure our readers that no realistic amount of government grants or eager first customers could have saved it from drowning in its own space leaks. Haskell itself is largely subsidized by the academic tenure system, and is thus well insulated from its own practical failings, but that’s another story entirely.
Agree with most of this, however, Antti spent something like well over a decade fleshing out rump - and that was on top of netbsd which was forked 27 years ago so I wouldn’t agree that the ecosystem doesn’t need financial resources. The EU is trying to pump some money into the ecosystem thankfully. There indeed are a few consultancies but they tend to come and go (again lack of resources) - I keep a running list.
Also, this is not intended to be a marketing site - it’s a community site. I was curious if the lobsters crowd would cast it as such but since I have seen plenty of other sites like ziglang with ‘donate now’ buttons figured one link on a community site wouldn’t be ‘marketing’.
As for the questions:
No. From the ground up it’s written in scratch - github.com/nanovms/nanos .
A donate button doesn’t make a marketing site, but there’s almost no substantial content — barely a synopsis — and it is indeed mostly trying to convince you to use it. That’s marketing.
No, of course it’s fine; the point is someone said “weird that this page is missing these basic things” and the author replied “the site is a WIP, it only went up yesterday” to explain why these basic things are missing. Both are talking about the site, not the software itself, because they’re talking about the submission, not the product, which is a distinction so many people in this thread can seemingly not make.
Lobsters is about discussing submissions, right? If you feel the need to defend your submission by saying “it’s a WIP, it only went up yesterday” when someone points out key information is missing, I don’t think you should have posted it yet. What’s the rush?
If it is not appropriate I apologize, although, I feel lobste.rs should post a guidelines document as I see a lot of links posted that have commercial links back and I can’t think of a single OSS project that is commercially supported that doesn’t have a link back.
Just now I found the following on lobste.rs - all that had more than one link going to a commercial post:
Like I said, the donate button isn’t what matters, and the examples you’ve given serve my point: they are chock full of technical content and detail, which is what we want to see here. This submission has almost no substance.
There’s sincerely no aggression; you’re asking about what’s appropriate and I’m doing my best to help you understand why (from my point of view) I feel this isn’t an appropriate submission. I think my opinion is at least somewhat representative of community norms based on votes. I’m not attacking you, I’m sharing my context. You don’t have to agree with my assessment.
The page is light on detail and mostly serves to advertise. That’s what it comes down to. I don’t think this submission adds anything. The direct repo link would always be better, or a technical detail post.
And yes, you posted the repo a little while ago, and that’s okay. If there’s a new release, link to the patch notes instead. Reposting the same link isn’t forbidden either, especially if there’s been big changes.
I think a lot of people would find these comments aggressive - maybe you can disclose your real name and who you work for. :) (That’s ok, you don’t have to.)
You are most definitely correct that it does to advertise the community aspect of the operating system. I’m sorry if you are looking for a corporate only POV. I won’t advertise it here but you can easily find it.
I’m glad you don’t find patch notes offensive - I’d find any patch notes offensive that are offensive. Nanos.org is a brand new site so sorry no new changes.
I actually found your comments to be on the aggressive side, while others tried to explain their views on why your post is flagged. And the length of their posts implies it’s a sincere efforts.
maybe you can disclose your real name and who you work for.
This is in bad faith. Additionally, my real name and employer are trivial to locate. I really think you should reevaluate your angle here.
I’m sorry if you are looking for a corporate only POV.
I have to believe you’re being intentionally obtuse in light of my suggesting “the direct repo link” or “a technical detail post”. I’ve really tried only to be kind in my comments here and represent my views to get to a shared understanding with you, but you seem to consistently engage in hostility and points raised while further stirring things up.
There seems to be a misunderstanding here. I think absolutely nobody is objecting to submitting Nanos, that would be ridiculous. People are objecting (although I disagree) to submitting the URL https://nanos.org/, instead of (more relevant) URL https://github.com/nanovms/nanos.
The nanos.org website, is strictly for providing a place for the community to place a voice outside of the company. One of the reasons of having a community site was to move a lot of knowledge that was in engineers heads to the community in the hopes that it would not be lost.
The very fact that it is a .org and not a .com, should be painfully clear but in 2020 I realize that is just not something that works with everyone.
Just as kernel.org is an .org, yet retains logos from Fastly, Packet, Redhat, Google and many others, nanos.org is an open source site with source code found elsewhere.
Submitting a site structured like kernel.org would indeed meet some annoyance, because there is nothing to read on this page, and not clear which of the links you considered most worth following. You could pick one previously unposted of the many pages you have mentioned with technical substance and say in the comments that «We are currently working on building a community site on nanos.org and making the linked material available (or updating it) was a part of that work». That would look in a different way.
Note that a page that looks like a pure advertisement for a community is still pure advertisement, even if of a slightly different kind compared to commercial advertisement.
I guess you could imagine the following imaginary (or is it?) use case for Lobste.rs: automatically download the linked articles and separately comments automatically and read them offline. If changing what you link to and mentioning the original link in the first comment would increase the value for such a use case and not clearly decrease the value for the more typical use, it is likely that such a replacement would also improve the reception here.
I still don’t understand the frustration here. Is the main page not the most direct way to announce a public free open source tool that did not have it before?
If you had never heard of rust before aside from a few coworkers chatting about it and rust-lang.org didn’t exist but then it pops out of existence you wouldn’t find it appropriate to post?
rust-lang.org goes in somewhat more details than the current state of nanos.org, and I guess a more detailed page about what choices Rust makes would be a better choice than the main page.
Many of us want to click the link and find a page containing a text about something technical that directly convinces us to care. Currently nanos.org tells me «Nanos is a unikernel» and nothing else (sure, almost every unikernel will mention that avoiding context switches will improve performance in read()-bottlenecked tests), then there are some links. I can easily understand people preferring to get a submission where the main link goes to a substantial text, not to a place where you need to find out which link to click to get the text. (I personally did not flag this submission, though)
I 100% agree there should be more detailed documentation and there most definitely will be, it’s a work in progress.
I suppose all the frustration can be summed up with “come back when there is X bytes of technical documentation and link to that rather than the front page”. Noted.
https://nanos.org/thebook isn’t technical at all? After one day? With links to pull requests of code? and binary serialization formats for a non extX filesystem?
This community has become so fucking toxic! It’s really sad that I now no longer enjoy reading the comments. I use to come here for the fun technical discussions. But once again the internet has ruined another great website. jcs should have kept everything invite only and small. No wonder he gave up and walked away…
I think what you’re witnessing is the site adjusting to a course correction toward its original purpose. A few people were making a push toward getting more ‘culture’ related content on the front page, and that’s not what this site has ever been about. Some folks have been pushing back.
Fortunately, in the last few months I’ve seen more interesting material hitting the front page than in recent years, and the comments are still well worth exploring (in those threads). Anything resembling “toxicity” comes in ‘culture’ tagged articles.
We have nginx and many many other applications available as packages. (think apt-get).
The focus on go is that we have a lot of random infrastructure tooling written in go and so is probably the best supported. Both ops.city and nanos.org are go unikernels running nanos. ops has been up since more than a year now?
We have a project to do automated performance regression analysis and reporting so it’s our hope that this becomes extremely transparent/reproducible in the future.
Full disclosure: there’s (very little) code of mine in nanos, but I haven’t worked on it/used it in about 1.5 years now – I wanted to but there just weren’t enough hours in a day for it :(.
Back when I used it, IwIP wasn’t super-fast, but it was pretty easy to work with and pretty flexible. I haven’t ran any serious benchmarks (i.e. on dedicated networking/serving hardware), but I did run some basic tests on some of my development machines and it wasn’t dog-slow. I don’t expect it would be useless for production workloads, albeit there’s likely quite some room for improvement.
Thanks for your contributions! Perf was not top of mind at that point in time and is actually one reason why nanos.org didn’t exist back then but a few of the ft’s have made some serious progress. When brian added ftrace it really revealed a lot that was hard to see before.
FWIW, I think lwIP really is the best choice here, and as I mentioned above, I don’t doubt that it’s adequate for production workloads. There’s always room for improving any networking stack, but lwIP itself is the result of a lot of super solid work. Coming up with something better than it is, in and of itself, a project of considerable magnitude, comparable to developing a unikernel itself!
I think it gets a bad reputation because it’s used in tons of bad firmware, often running on top of bad hardware. I haven’t done it myself, but I know of at least one case where poor performance was traced not to lwIP, but a buggy Ethernet MAC implementation in the FPGA below it.
wasn’t super-fast, but it was pretty easy to work with and pretty flexible.
This is fantastic reminder that often we need to balance usability (and ease of use in general) with other things. Kudos for being honest and open about trade-offs.
Indeed! See my other reply for some additional details – like I said, my involvement with nanos has been minimal (a few weeks?) and it was more than an year ago, so it’s not really “my” trade-off to comment about (and I know things have improved considerably – see the parent of that post of mine). But in general – and, for what it’s worth, in this case, too – I think this is a very valid trade-off to make. lwIP offers adequate performance for many (most?) of the things for which you’d use a unikernel running on virtualized hardware, and it’s pretty solid and easy to integrate. It takes years for a network stack of any performance level to reach lwIP’s maturity, doing that in the early stages of building a new kernel would be both unmanageable and pretty much useless.
I don’t really know how to classify a network stack as fast, but I’ve worked with LwIP and we’ve pushed it to more than gigabit speeds in synthetic benchmarks (although we weren’t benchmarking LwIP but rather some other code around it).
I don’t get why some folks are annoyed about you posting a link to the site, it gives me a good idea about what Nanos and the ops tool do, at a glance.
Really? I have literally no idea what Nanos is or does except “runs applications” on a unikernel. Would you mind sharing what it is Nanos actually does?
Mind sharing your good idea? What does Nanos do, and what does the ops tool do? I couldn’t tell, at a glance. I didn’t find “run code faster than the speed of light” to be informative, the FAQ didn’t say much, and all 600-ish words of “the book” tell me it’s a unikernal that’s opinionated about security but doesn’t say what it’s good for or why I might want to use it.
I could tell from “getting started” that they wanted me to pipe some script from curl to sh, but I didn’t want to do that, especially before I understood what those things do and why I might want to use them.
Nanos is not Linux (it is an independent implementation, so for example, as noted above, it uses lwIP as a network stack, not the Linux network stack), but it can run Linux binaries, by emulating Linux’s ABI and syscall interface. It claims to run Linux binaries better than Linux in some sense, for example by being faster. Nanos probably can’t ever be better in Linux binary compatibility than Linux, but it claims to be good enough.
But it claims it can’t run on bare hardware, right, so I’d need to run it on top of KVM or Xen, which means I’d need to run Linux also? Even just to fire up a VM on VMWare workstation and try it out?
And “the book” claims it has no syscalls…
How is it faster than running Linux binaries than Linux is, if I need to fire up KVM or Xen in order to bootstrap it? Why wouldn’t I just statically link my binaries and directly run them on the host, sandboxed?
I’ve spent 5 minutes looking at the github site, which is a little more helpful than the site OP linked, but I’m still not sure what my use case is for this yet, even though it does sound interesting.
Let’s say you were running Linux on AWS virtualized (very common scenario). Since you weren’t running on bare hardware anyway, there is no disadvantage to switching to Nanos on that scenario, from that reason.
It has no syscalls in a sense that there is no transition from user mode to kernel mode. Your code runs in kernel mode, and Linux syscalls are emulated as function calls.
I haven’t done benchmarks myself, but it is very believable Nanos runs some Linux binaries faster than Linux, since it does less than Linux. As I noted above, there is no syscall transition, which is often a significant overhead. There is also no users and no processes: you are the only user, and your code is the only process. This does greatly simplify what the “kernel” (quoted, since technically your code is kernel) needs to do.
We actually do keep CPL for no other reason that if you have certain pages that are say only readable you don’t want an attacker to make them writeable (and that still gives us a speed boost). Context switching can imply a few different scenarios depending on context (heh!). Kernel -> User, kthread to kthread, user -> user, etc. We’ve found that a lot of the heavy switching is actually a result of the fact that modern GPOS like linux have hundreds of user processes upon boot even if the intention is to only run a single application (which is very common in server-side deployments).
First off, I very much respect the OSv engineers, however, the corporate side of that has moved on to https://www.scylladb.com/ which makes governance a bit of an issue (for us). There are a handful of architectural differences. We have strived to be as binary compatible towards linux as possible (elf loading, using whatever libc the app is linked to, etc.) We also approached the niche from a security POV, versus the performance view so that’s where we’ve focused. It should be noted that all of these are very much operating systems just not general purpose operating systems and so each one has a lot of engineer hours that go into it. So each system excels in various areas where others might be deficient. This is compounded by the fact that most unikernel enthusiasts are motivated by different goals/needs (NFV, edge, performance, security, etc.).
That looks to me like it’s running on a linux host, and I think that is contributing to my confusion, if the real idea is that I should be replacing Linux with this on some public cloud hypervisor. Which leaves me needing to go understand the ops tool, I think, in order to assess whether this might fit into my workflow or offer any benefit.
Thanks for jumping in and explaining… I hadn’t spotted where in the stack this fits, and that’s helpful.
The ops tool is complementary. You can envision it as something comparable to terraform/chef/puppet (but not really). The idea is that most end-users provision their web application software (eg: all websites) to various public cloud providers which we support every single popular one. There’s a lot of api involved that is not proper to put into the core kernel nanos and if you dive into nanos further you’ll find things like the filesystem manifest which for real applications can get rather large (think jvm or python applications). Ops also helps in this regard by generating that on demand.
ops can be ran on linux and mac today and we have someone working on windows as we speak.
It very much does have syscalls. That section is empty because it needs to be filled out. There are some syscalls we support 100%. Some are stubbed out (on purpose). Some are implemented differently.
You are right it most definitely is not written for bare metal. It is written for most production web application environments which run predominately on the public cloud (aws, gcloud, etc.) which use 2 layers of linux. One for the hypervisor layer and one for guest. This replaces the guest layer.
Thanks. That clarifies it some. I think I need to understand some details about the ops tool in order to evaluate further. If you’re working on the site, that might be a good thing to expand on.
One of the whole reasons for the site is to provide more documentation that would be hard to grok from the code alone.
Nanos is meant to be a server-side only system of running linux applications with a focus on performance and security. For the install script - it points to https://github.com/nanovms/ops/blob/master/install.sh which is viewable/editable on github. You don’t have to use it and can opt to build from source instead.
I think I understood that the intention of the site is to provide info that would be hard to absorb from just looking at a git repository. The feedback I was offering is that the “what” and “why” information that might make me want to try it out isn’t robust enough for me to understand yet, and the “getting started” information that is on the site leaves me needing to go pick through a git repository and read the source code for an install script/orchestration tool in order to feel comfortable that I understand what it claims it will do before I run it.
I fully agree that we need/want more documentation/information - that’s the whole purpose of the site. We have decent documentation for OPS https://nanovms.gitbook.io/ops/ that is fully PR’able on github but that provides no justice for Nanos itself which is way more technical and since OPS isn’t the only horse in town we wanted a central place to detail nanos much like a rust-lang.org .
Interesting project, but this marketing BS is not a good link for Lobsters. Here are the very basic questions I had about Nanos:
To answer all but the last of these, I had to click around and find the github repo. The FAQ on this marketing site is pretty vacuous. “Syscalls: N/A” lol.
Anyway, I think I’ll stick with https://mirage.io/ for my unikernel needs. But thanks for sharing!
I only just looked into this stuff but… all of MirageOS’s peers are dead. So I think I welcome anyone new in the space. We need progress and competition!!!
Good point, but it perhaps says something about the unikernel concept itself and how it’s played out in practice. Overall, I agree, and I’m happy to see Other People’s Money being spent on a technical subject I personally find interesting. I sincerely wish the nanos folks all kinds of success.
At the same time, I must admit to having some doubts about the wisdom of implementing a unikernel with a completely fresh code base in a famously unsafe language, but hey, I guess we’ll see how it plays out.
The money point is important to point out. It’s not something you can just whip together in a weekend and not something I could see happening without lots of $$ to employ full time engineers on. Even then the difference between hello world and production usability is a huge gulf to cross - most teams unfortunately die before they can cross it. That in my most humble opinion is the biggest problem in the ecosystem. Government grants can only take you so far.
I hear your complaint against c. We’ve discussed doing other complementary projects in rust but the pkg/dep system is a major turn off and none of our engineers speak rust either other than hello world projects. Likewise $$ again is a prime concern for working in that ecosystem. When you have tiny little saas companies that raise tens of millions and employ tens to hundreds of software engineers you have to ask what is the true cost of a new language or a new operating system?
All very cogent points, and be it far from me to armchair-quarterback anyone’s business strategy. Application support is probably the first priority for a real-world unikernel, and that drags in legacy APIs and other ecosystem factors that we can’t really control. Latent security problems only become critical when you actually have some customers.
I’ve had a corner of an eye on this space for a little while, but I wouldn’t claim any expertise. To me, it seems like Mirage has carved out a bit of a niche and, like many open source projects, is sort of puttering along just fine with little in the way of sponsorship. But its adoption is of course limited to the (relatively tiny) Ocaml ecosystem. Then there’s the rump kernel approach, which largely piggybacks on NetBSD’s flexibility and mature-ish code base. Looks like there’s some consultancies promoting that. Probably works great for some applications. Doesn’t appear to need VC, at a glance.
The only unikernel I have had any real first-hand experience with is HalVM. I can assure our readers that no realistic amount of government grants or eager first customers could have saved it from drowning in its own space leaks. Haskell itself is largely subsidized by the academic tenure system, and is thus well insulated from its own practical failings, but that’s another story entirely.
Agree with most of this, however, Antti spent something like well over a decade fleshing out rump - and that was on top of netbsd which was forked 27 years ago so I wouldn’t agree that the ecosystem doesn’t need financial resources. The EU is trying to pump some money into the ecosystem thankfully. There indeed are a few consultancies but they tend to come and go (again lack of resources) - I keep a running list.
Oof. I only just noticed it’s in C. Okay, I’ll root for it, but hopefully they’re making up for that handicap.
This site is a WIP - literally went up yesterday.
Also, this is not intended to be a marketing site - it’s a community site. I was curious if the lobsters crowd would cast it as such but since I have seen plenty of other sites like ziglang with ‘donate now’ buttons figured one link on a community site wouldn’t be ‘marketing’.
As for the questions:
If it’s a WIP, why did you post it here?
A donate button doesn’t make a marketing site, but there’s almost no substantial content — barely a synopsis — and it is indeed mostly trying to convince you to use it. That’s marketing.
Are WIPs not allowed to be posted to Lobsters? Zig is a WIP, it’s at 0.6.0. Can we not link to a Zig release page?
No, of course it’s fine; the point is someone said “weird that this page is missing these basic things” and the author replied “the site is a WIP, it only went up yesterday” to explain why these basic things are missing. Both are talking about the site, not the software itself, because they’re talking about the submission, not the product, which is a distinction so many people in this thread can seemingly not make.
Lobsters is about discussing submissions, right? If you feel the need to defend your submission by saying “it’s a WIP, it only went up yesterday” when someone points out key information is missing, I don’t think you should have posted it yet. What’s the rush?
If it is not appropriate I apologize, although, I feel lobste.rs should post a guidelines document as I see a lot of links posted that have commercial links back and I can’t think of a single OSS project that is commercially supported that doesn’t have a link back.
Just now I found the following on lobste.rs - all that had more than one link going to a commercial post:
https://blog.cloudflare.com/unimog-cloudflares-edge-load-balancer/ https://android-developers.googleblog.com/2020/09/android11-final-release.html https://blog.ipfs.io/2020-09-08-nix-ipfs-milestone-1/
Like I said, the donate button isn’t what matters, and the examples you’ve given serve my point: they are chock full of technical content and detail, which is what we want to see here. This submission has almost no substance.
A new operating system and a new file system aren’t technical enough?
This “submission” is backed up by multiple repositories of open source code:
https://github.com/nanovms/nanos
Where is this aggression coming from?
There’s sincerely no aggression; you’re asking about what’s appropriate and I’m doing my best to help you understand why (from my point of view) I feel this isn’t an appropriate submission. I think my opinion is at least somewhat representative of community norms based on votes. I’m not attacking you, I’m sharing my context. You don’t have to agree with my assessment.
The page is light on detail and mostly serves to advertise. That’s what it comes down to. I don’t think this submission adds anything. The direct repo link would always be better, or a technical detail post.
And yes, you posted the repo a little while ago, and that’s okay. If there’s a new release, link to the patch notes instead. Reposting the same link isn’t forbidden either, especially if there’s been big changes.
I think a lot of people would find these comments aggressive - maybe you can disclose your real name and who you work for. :) (That’s ok, you don’t have to.)
You are most definitely correct that it does to advertise the community aspect of the operating system. I’m sorry if you are looking for a corporate only POV. I won’t advertise it here but you can easily find it.
I’m glad you don’t find patch notes offensive - I’d find any patch notes offensive that are offensive. Nanos.org is a brand new site so sorry no new changes.
I actually found your comments to be on the aggressive side, while others tried to explain their views on why your post is flagged. And the length of their posts implies it’s a sincere efforts.
This is in bad faith. Additionally, my real name and employer are trivial to locate. I really think you should reevaluate your angle here.
I have to believe you’re being intentionally obtuse in light of my suggesting “the direct repo link” or “a technical detail post”. I’ve really tried only to be kind in my comments here and represent my views to get to a shared understanding with you, but you seem to consistently engage in hostility and points raised while further stirring things up.
There seems to be a misunderstanding here. I think absolutely nobody is objecting to submitting Nanos, that would be ridiculous. People are objecting (although I disagree) to submitting the URL https://nanos.org/, instead of (more relevant) URL https://github.com/nanovms/nanos.
I hope this makes things clear.
The nanos.org website, is strictly for providing a place for the community to place a voice outside of the company. One of the reasons of having a community site was to move a lot of knowledge that was in engineers heads to the community in the hopes that it would not be lost.
The very fact that it is a .org and not a .com, should be painfully clear but in 2020 I realize that is just not something that works with everyone.
Just as kernel.org is an .org, yet retains logos from Fastly, Packet, Redhat, Google and many others, nanos.org is an open source site with source code found elsewhere.
Is this such a problem?
Submitting a site structured like kernel.org would indeed meet some annoyance, because there is nothing to read on this page, and not clear which of the links you considered most worth following. You could pick one previously unposted of the many pages you have mentioned with technical substance and say in the comments that «We are currently working on building a community site on nanos.org and making the linked material available (or updating it) was a part of that work». That would look in a different way.
Note that a page that looks like a pure advertisement for a community is still pure advertisement, even if of a slightly different kind compared to commercial advertisement.
I guess you could imagine the following imaginary (or is it?) use case for Lobste.rs: automatically download the linked articles and separately comments automatically and read them offline. If changing what you link to and mentioning the original link in the first comment would increase the value for such a use case and not clearly decrease the value for the more typical use, it is likely that such a replacement would also improve the reception here.
Are you kidding me?
I guess I was wrong in posting a free/open source community based website to lobste.rs - I’ll refrain from that in the future. So much hostility.
Strictly speaking, you were submitting a page, submissions are not really read as sites — there are surely many good pages on your site to submit.
I still don’t understand the frustration here. Is the main page not the most direct way to announce a public free open source tool that did not have it before?
If you had never heard of rust before aside from a few coworkers chatting about it and rust-lang.org didn’t exist but then it pops out of existence you wouldn’t find it appropriate to post?
rust-lang.org goes in somewhat more details than the current state of nanos.org, and I guess a more detailed page about what choices Rust makes would be a better choice than the main page.
Many of us want to click the link and find a page containing a text about something technical that directly convinces us to care. Currently nanos.org tells me «Nanos is a unikernel» and nothing else (sure, almost every unikernel will mention that avoiding context switches will improve performance in read()-bottlenecked tests), then there are some links. I can easily understand people preferring to get a submission where the main link goes to a substantial text, not to a place where you need to find out which link to click to get the text. (I personally did not flag this submission, though)
I 100% agree there should be more detailed documentation and there most definitely will be, it’s a work in progress.
I suppose all the frustration can be summed up with “come back when there is X bytes of technical documentation and link to that rather than the front page”. Noted.
Literally nothing about the link you used for the submission is technical at all. It’s a marketing splash page.
Literally? Nothing?
https://nanos.org/thebook isn’t technical at all? After one day? With links to pull requests of code? and binary serialization formats for a non extX filesystem?
Please.
Then why didn’t you link to the book? Remember, I said the link you used for the submission.
It’s a link from the main site…
This community has become so fucking toxic! It’s really sad that I now no longer enjoy reading the comments. I use to come here for the fun technical discussions. But once again the internet has ruined another great website. jcs should have kept everything invite only and small. No wonder he gave up and walked away…
I think what you’re witnessing is the site adjusting to a course correction toward its original purpose. A few people were making a push toward getting more ‘culture’ related content on the front page, and that’s not what this site has ever been about. Some folks have been pushing back.
Fortunately, in the last few months I’ve seen more interesting material hitting the front page than in recent years, and the comments are still well worth exploring (in those threads). Anything resembling “toxicity” comes in ‘culture’ tagged articles.
If you’re testing static content, is there a reason to use something other than nginx?
We have nginx and many many other applications available as packages. (think apt-get).
The focus on go is that we have a lot of random infrastructure tooling written in go and so is probably the best supported. Both ops.city and nanos.org are go unikernels running nanos. ops has been up since more than a year now?
We have a project to do automated performance regression analysis and reporting so it’s our hope that this becomes extremely transparent/reproducible in the future.
I was surprised to see it uses lwIP as a network stack. Contrary to my preconception, maybe lwIP is fast enough?
Full disclosure: there’s (very little) code of mine in nanos, but I haven’t worked on it/used it in about 1.5 years now – I wanted to but there just weren’t enough hours in a day for it :(.
Back when I used it, IwIP wasn’t super-fast, but it was pretty easy to work with and pretty flexible. I haven’t ran any serious benchmarks (i.e. on dedicated networking/serving hardware), but I did run some basic tests on some of my development machines and it wasn’t dog-slow. I don’t expect it would be useless for production workloads, albeit there’s likely quite some room for improvement.
Thanks for your contributions! Perf was not top of mind at that point in time and is actually one reason why nanos.org didn’t exist back then but a few of the ft’s have made some serious progress. When brian added ftrace it really revealed a lot that was hard to see before.
FWIW, I think lwIP really is the best choice here, and as I mentioned above, I don’t doubt that it’s adequate for production workloads. There’s always room for improving any networking stack, but lwIP itself is the result of a lot of super solid work. Coming up with something better than it is, in and of itself, a project of considerable magnitude, comparable to developing a unikernel itself!
I think it gets a bad reputation because it’s used in tons of bad firmware, often running on top of bad hardware. I haven’t done it myself, but I know of at least one case where poor performance was traced not to lwIP, but a buggy Ethernet MAC implementation in the FPGA below it.
This is fantastic reminder that often we need to balance usability (and ease of use in general) with other things. Kudos for being honest and open about trade-offs.
Indeed! See my other reply for some additional details – like I said, my involvement with nanos has been minimal (a few weeks?) and it was more than an year ago, so it’s not really “my” trade-off to comment about (and I know things have improved considerably – see the parent of that post of mine). But in general – and, for what it’s worth, in this case, too – I think this is a very valid trade-off to make. lwIP offers adequate performance for many (most?) of the things for which you’d use a unikernel running on virtualized hardware, and it’s pretty solid and easy to integrate. It takes years for a network stack of any performance level to reach lwIP’s maturity, doing that in the early stages of building a new kernel would be both unmanageable and pretty much useless.
lwIP was chosen for speed of adoption not necessarily for anything else - I fully expect us to evolve to something else in the future
I don’t really know how to classify a network stack as fast, but I’ve worked with LwIP and we’ve pushed it to more than gigabit speeds in synthetic benchmarks (although we weren’t benchmarking LwIP but rather some other code around it).
Nanos looks very interesting, thanks for sharing!
I don’t get why some folks are annoyed about you posting a link to the site, it gives me a good idea about what Nanos and the
opstool do, at a glance.Really? I have literally no idea what Nanos is or does except “runs applications” on a unikernel. Would you mind sharing what it is Nanos actually does?
Nanos is a Linux syscall compatible unikernel. It seems to be a less mature OSv, another Linux syscall compatible unikernel.
Thank you, the OSv site was really helpful.
I always forget about OSv but it does look pretty awesome. Do you also know about UniK?
Mind sharing your good idea? What does Nanos do, and what does the ops tool do? I couldn’t tell, at a glance. I didn’t find “run code faster than the speed of light” to be informative, the FAQ didn’t say much, and all 600-ish words of “the book” tell me it’s a unikernal that’s opinionated about security but doesn’t say what it’s good for or why I might want to use it.
I could tell from “getting started” that they wanted me to pipe some script from curl to sh, but I didn’t want to do that, especially before I understood what those things do and why I might want to use them.
Nanos is not Linux (it is an independent implementation, so for example, as noted above, it uses lwIP as a network stack, not the Linux network stack), but it can run Linux binaries, by emulating Linux’s ABI and syscall interface. It claims to run Linux binaries better than Linux in some sense, for example by being faster. Nanos probably can’t ever be better in Linux binary compatibility than Linux, but it claims to be good enough.
But it claims it can’t run on bare hardware, right, so I’d need to run it on top of KVM or Xen, which means I’d need to run Linux also? Even just to fire up a VM on VMWare workstation and try it out?
And “the book” claims it has no syscalls…
How is it faster than running Linux binaries than Linux is, if I need to fire up KVM or Xen in order to bootstrap it? Why wouldn’t I just statically link my binaries and directly run them on the host, sandboxed?
I’ve spent 5 minutes looking at the github site, which is a little more helpful than the site OP linked, but I’m still not sure what my use case is for this yet, even though it does sound interesting.
Let’s say you were running Linux on AWS virtualized (very common scenario). Since you weren’t running on bare hardware anyway, there is no disadvantage to switching to Nanos on that scenario, from that reason.
It has no syscalls in a sense that there is no transition from user mode to kernel mode. Your code runs in kernel mode, and Linux syscalls are emulated as function calls.
I haven’t done benchmarks myself, but it is very believable Nanos runs some Linux binaries faster than Linux, since it does less than Linux. As I noted above, there is no syscall transition, which is often a significant overhead. There is also no users and no processes: you are the only user, and your code is the only process. This does greatly simplify what the “kernel” (quoted, since technically your code is kernel) needs to do.
We actually do keep CPL for no other reason that if you have certain pages that are say only readable you don’t want an attacker to make them writeable (and that still gives us a speed boost). Context switching can imply a few different scenarios depending on context (heh!). Kernel -> User, kthread to kthread, user -> user, etc. We’ve found that a lot of the heavy switching is actually a result of the fact that modern GPOS like linux have hundreds of user processes upon boot even if the intention is to only run a single application (which is very common in server-side deployments).
Thanks for the correction.
Since you are here, I want to ask why should I run Nanos instead of OSv. Since two seem to be in a similar niche, a good comparison would be welcome.
First off, I very much respect the OSv engineers, however, the corporate side of that has moved on to https://www.scylladb.com/ which makes governance a bit of an issue (for us). There are a handful of architectural differences. We have strived to be as binary compatible towards linux as possible (elf loading, using whatever libc the app is linked to, etc.) We also approached the niche from a security POV, versus the performance view so that’s where we’ve focused. It should be noted that all of these are very much operating systems just not general purpose operating systems and so each one has a lot of engineer hours that go into it. So each system excels in various areas where others might be deficient. This is compounded by the fact that most unikernel enthusiasts are motivated by different goals/needs (NFV, edge, performance, security, etc.).
I think where I’m getting lost is looking at the proposed getting started workflow captured in this image:
https://nanos.org/static/img/terminal-2.png
That looks to me like it’s running on a linux host, and I think that is contributing to my confusion, if the real idea is that I should be replacing Linux with this on some public cloud hypervisor. Which leaves me needing to go understand the
opstool, I think, in order to assess whether this might fit into my workflow or offer any benefit.Thanks for jumping in and explaining… I hadn’t spotted where in the stack this fits, and that’s helpful.
The image indeed shows running on Linux host. As I understand, it is for convenience, and it launches KVM behind the scene.
The real idea is indeed replacing Linux with Nanos on public cloud. This chapter on how to create AWS image should make things clearer.
The ops tool is complementary. You can envision it as something comparable to terraform/chef/puppet (but not really). The idea is that most end-users provision their web application software (eg: all websites) to various public cloud providers which we support every single popular one. There’s a lot of api involved that is not proper to put into the core kernel nanos and if you dive into nanos further you’ll find things like the filesystem manifest which for real applications can get rather large (think jvm or python applications). Ops also helps in this regard by generating that on demand.
ops can be ran on linux and mac today and we have someone working on windows as we speak.
It very much does have syscalls. That section is empty because it needs to be filled out. There are some syscalls we support 100%. Some are stubbed out (on purpose). Some are implemented differently.
You are right it most definitely is not written for bare metal. It is written for most production web application environments which run predominately on the public cloud (aws, gcloud, etc.) which use 2 layers of linux. One for the hypervisor layer and one for guest. This replaces the guest layer.
Thanks. That clarifies it some. I think I need to understand some details about the ops tool in order to evaluate further. If you’re working on the site, that might be a good thing to expand on.
For sure - thank you for the suggestion!
One of the whole reasons for the site is to provide more documentation that would be hard to grok from the code alone.
Nanos is meant to be a server-side only system of running linux applications with a focus on performance and security. For the install script - it points to https://github.com/nanovms/ops/blob/master/install.sh which is viewable/editable on github. You don’t have to use it and can opt to build from source instead.
I think I understood that the intention of the site is to provide info that would be hard to absorb from just looking at a git repository. The feedback I was offering is that the “what” and “why” information that might make me want to try it out isn’t robust enough for me to understand yet, and the “getting started” information that is on the site leaves me needing to go pick through a git repository and read the source code for an install script/orchestration tool in order to feel comfortable that I understand what it claims it will do before I run it.
I fully agree that we need/want more documentation/information - that’s the whole purpose of the site. We have decent documentation for OPS https://nanovms.gitbook.io/ops/ that is fully PR’able on github but that provides no justice for Nanos itself which is way more technical and since OPS isn’t the only horse in town we wanted a central place to detail nanos much like a rust-lang.org .
Nice overview, thanks
That’s literally impossible.
it’s a superlative (inserted by the designer) ;)
Why so narrow-minded? https://en.wikipedia.org/wiki/Quantum_dot_cellular_automaton (for the humor-deficient among us, I’d like to clarify: that’s “humor”)