Threads for bcantrill

  1. 1

    Nice illustration of how even seemingly simple and low-level things are quite intricate.

    1. 1

      I think that that’s our unofficial slogan at Oxide…

    1. 3

      I’d love to have been a fly on the wall when Oxide engineers decided to make their sleds read their initrd using XModem

      1. 7

        It’s less interesting than you might think XMODEM isn’t a part of the product, it’s merely what we’re using to get images loaded into RAM over UART to be able to quickly iterate on host OS development (i.e., obviating the need to drop a payload into SPI NOR) – so it was really a question strictly of expedience. If you’re curious about what is in the product, I went into some of the details of what we’re building (and why) in my recent talk on engineering towards holistic systems.

        1. 3

          Sounds very familiar. For software work on the early CHERI prototypes, the easiest way of loading a kernel was via the JTAG interface (which actually injected instructions to materialise a constant value and then write it into memory at a specific address into the pipeline) and then use JTAG to set the PC value to the kernel entry point. No loader, no filesystem, no system firmware even, just a debugging interface that splatted data into memory and poked the register file.

          I sometimes miss this on more production systems. Being able to just take a kernel image and run it with no supporting infrastructure was great for iteration.

      1. 2

        The Arduino folks have had lots of problems with FTDI USB->serial adapters on different platforms, and solutions for them. The official drivers are pretty bad. Another big problem is that a TON of FTDI chips are just counterfeits and don’t work very well. I’m assuming Oxide doesn’t have this issue in their lab, but maybe their customers will have it.

        1. 4

          Totally agreed on FTDI driver quality and the counterfeit problem. Fortunately, our customers never need to connect via serial port: all of the management is via the dedicated management network.

        1. 10

          A level up the stack, I have slightly different perspective:

          The pc platform is nothing short of miraculous. The existence of an open, broadly adopted standard that everyone can use to get something up and running is a boon, and it does not seem to have been replicated. From what I hear, arm is still a fragmentary shitshow, for instance. Sure, bioses have some fragmentation, but very little in practice, and efi is even more cohesive.

          It is true, of course, that all abstractions have a cost. Joel’s law of software abstraction is universal. I think integration is a good idea, in general. But this is a bizarre target. I have never felt hampered in my ability to design systems by the firmware<->os bringup protocol. And I can’t speak to firmware authors, but I can’t imagine that supporting these interfaces is a tremendous burden for them either. Acpi, pci, usb, etc. are good enough, on the whole and for the most part—but even those would make better targets. If we are trying to co-design hardware and software—why are we sticking with decades-old isas? I want encodings tuned to empirically minimise code size. Cheap userspace interrupts, and read barriers built into my loads. Fine-grained bounds checks built into all my memory accesses.

          A CPU business is not an easy thing to build—azul managed for a bit, but no longer sells computers, for instance—and I don’t mean to imply that, if one is interested in cohesion, one must build a CPU. But this seems like a very trivial target to take aim at, and the loss of portability does not seem worth its weight.

          1. 8

            This is a bit of a surprising take; when you say that “you have never felt hampered” in your ability to design systems, what kind of systems are we talking about here? In particular: are we talking about server-side computers? And when you say you “can’t imagine that supporting these interfaces is a tremendous burden”, is that based on any experience with UEFI? Even UEFI’s proponents (such as there are any, as even the inventor of UEFI believes that it should be replaced!) would acknowledge that UEFI places tremendous constraints on firmware implementation.

            1. 7

              From what I hear, arm is still a fragmentary shitshow, for instance. Sure, bioses have some fragmentation, but very little in practice, and efi is even more cohesive.

              It isn’t that bad now (at least for systems that aim for the same space as x86 hardware), but mostly as a result of adopting UEFI.

              A better comparison point would be IBM and Sun workstations (or Apple Macs) from the ’90s and early 2000s. These all used OpenFirmware, which was vastly better than UEFI. It had a lighweight Forth interpreter that allows the same ROM code for devices to execute on PowerPC or SPARC hardware. A lot of embedded devices use Flattened Device Trees (FDTs) for SoC device enumeration. FDT is a serialisation of the Device Tree format from OpenFirmware.

              1. 2

                These all used OpenFirmware, which was vastly better than UEFI.

                Everyone I talked to at IBM despised writing drivers in Forth. To them, Petitboot was a major improvement.

              2. 1

                I suppose it depends how much value you put on being able to run supervisors not really designed for the hardware/firmware onto new hardware. So much of the PC is made weird by having to run older OSes on the newest HW, instead of being developed in near-lockstep.

              1. 1

                Does anything recent exist that can be considered a holistic system?

                1. 12

                  For personal computing, I would say that the Apple M1 is definitely a holistic system – and there are many instances of personal devices.

                  1. 6

                    I agree about Apple Silicon Macs. Asahi Linux’s description of the Apple Silicon boot process is interesting. My favorite part:

                    One consequence of the boot picker being implemented as a macOS application behind the scenes is that is has full accessibility support (including VoiceOver), which is rather unique.

                  2. 4

                    Precursor comes close in the mobile space: https://www.crowdsupply.com/sutajio-kosagi/precursor

                  1. 7

                    It’s great to see this article here! I asked Bart write this back in the day, as part of an ACM Queue issue that I was putting together on performance. I think the issue[0] ended up really strong, and I think that Bart’s article has aged well – and I would encourage folks to read the other articles in that issue, which I think have also broadly aged well.

                    [0] https://queue.acm.org/issuedetail.cfm?issue=1117389

                    1. 2

                      I should probably admit I learned of it after glimpsing it posted somewhere on the work chat by Bart himself 🙈

                      It’s a great article, it goes a bit beyond generic advice about making sure not to trash the cache and has some actual examples as well as some v cool historical tidbits

                    1. 2

                      Is this just a transcript of the previously circulated video, or it adds more to the story?

                      1. 9

                        It is broadly a transcript (with included visuals), but several folks have indicated that that’s it’s easier to consume this way.[0] That said, there is some new content in the “Epilogue” – and especially in its link to the (new) Hubris FAQ[1], which merits its own read.

                        [0] https://twitter.com/joranwei/status/1468054541530591238

                        [1] https://github.com/oxidecomputer/hubris/blob/master/FAQ.mkdn

                        1. 1

                          Oh yes please! I am pretty interested in this topic so I actually watched the video but having all this material in a form that’s easy to examine carefully, re-read, ponder about, archive and take notes on is a kingly gift! I realize it’s still just a single blip on the engagement radar but it’s the useful kind of blip :-P.

                      1. 30

                        Man, I’m really glad to see Oxide shipping something.

                        That said…flagged. Lobsters is not for slick product pages and signups. It’s not shipping for another year.

                        1. 4

                          I would love to see a more technical breakdown of the hardware and software stack. For instance @riking was able to suss out that the hardware runs Illumos with the bhyve hypervisor, more details on all the components would be super interesting.

                          1. 4

                            Yeah, sorry – a lot more technical information to come, I promise. In the meantime, I went into some details in an episode of the The Data Center Podcast[0] that we hadn’t gone into elsewhere. But more is definitely coming – and it will all be open source before we ship!

                            [0] https://www.datacenterknowledge.com/hardware/why-your-servers-suck-and-how-oxide-computer-plans-make-better

                        1. 38

                          “The Gang Builds a Mainframe”

                          1. 28

                            Ha ha! I don’t think the mainframe is really a good analogue for what we’re doing (commodity silicon, all open source SW and open source FW, etc.) – but that nonetheless is really very funny.

                            1. 7

                              It makes you wonder what makes a mainframe a mainframe. Is it architecture? Reliability? Single-image scale-up?

                              1. 26

                                I had always assumed it was the extreme litigiousness of the manufacturer!

                                1. 3

                                  Channel-based IO with highly programmable controllers and an inability to understand that some lines have more than 80 characters.

                                  1. 1

                                    I think the overwhelming focus of modern z/OS on “everyone just has a recursive hierarchy of VMs” would also be a really central concept, as would the ability to cleanly enforce/support that in hardware. (I know you can technically do that on modern ARM and amd64 CPUs, but the virtualization architecture isn’t quite set up the same way, IMVHO.)

                                    1. 2

                                      I remember reading a story from back in the days when “Virtual Machine” specifically meant IBM VM. They wanted to see how deeply they could nest things, and so the system operator recursively IPL’d more and more machines and watched as the command prompt changed as it got deeper (the character used for the command prompt would indicate how deeply nested you were).

                                      Then as they shut down the nested VMs, they accidentally shut down one machine too many…

                                      1. 2

                                        Then as they shut down the nested VMs, they accidentally shut down one machine too many…

                                        This sounds like the plot of a sci-fi short story.

                                        1. 3

                                          …and overhead, without any fuss, the stars were going out.

                                  2. 1

                                    I’d go with reliability + scale-up. I’ve heard there’s support for like, fully redundant CPUs and RAM. That is very unique compared to our commodity/cloud world.

                                    1. 1

                                      If you’re interested in that sort of thing, you might like to read up on HP’s (née Tandem’s) NonStop line. Basically at least two of everything.

                                    2. 1

                                      Architecture. I’ve never actually touched a mainframe computer, so grain of salt here, but I once heard the difference described this way:

                                      Nearly all modern computers from the $5 Raspberry Pi Zero on up to the beefiest x86 and ARM enterprise-grade servers you can buy today are classified as microcomputers. A microcomputer is built around one or more CPUs manufactured as an integrated circuit. This CPU has a static bus that connects the CPU to all other components.

                                      A mainframe, however, is built around the bus. This allows not only for the hardware itself to be somewhat configurable per-job (pick your number of CPUs, amount of RAM, etc), but mainframes were built to handle batch data processing jobs and have always handily beat mini- and microcomputers in terms of raw I/O speed and storage capability. A whole lot of the things we take for granted today were born on the mainframe: virtualization, timesharing, fully redundant hardware, and so on. The bus-oriented design also means they have always scaled well.

                                1. 19

                                  @bcantrill: Is the lack of fine-grained allocator control hurting you in embedded contexts? I know that was a major pain point for some and a big motivating factor for Zig, but maybe Oxide doesn’t actually need any of that?

                                  1. 24

                                    Well, at the moment we are really doing everything possible to avoid dynamic allocation full-stop – and in fact, what is interesting about the system that we’re developing (in my opinion) is the creativity in assuring that traditionally dynamic activities (like task creation) are done entirely statically. So if/when/where we do have dynamic memory allocation, it is likely to be exceedingly simple – and focused on space efficiency rather than time.

                                    1. 5

                                      Having done embedded for a long time in the past. not having dynamic allocation (with all the gimmick to get the size “right”) was the best long term decision to guarantee long uptime. It is very different when you have the luxury of your application being started from the OS fresh each time it needs to run,

                                      1. 11

                                        Adding on: it’s not “only” a matter of uptime, it also helps tremendously with testing, reviewing and analysis. When everything is allocated statically, you know whether you have enough memory or not right from the start. There’s no imaginable cornercase under which malloc might fail to allocate some memory because, well, nobody’s malloc-ing anything.

                                        The fact that you don’t get OOM crashes (and, thus, your uptime doesn’t get thrashed) is just one of the nice things about it. You also get more predictable timing, you have a more solid basis for all sorts of hardware-related decisions and so on.

                                        It’s not a matter of embedded developers being opposed to being dragged out of the stone age of computing, there are some valid technical reasons behind our preference for flint stone arrowheads :).

                                  1. 2

                                    What gives it that security?

                                    Single Process

                                    Oh yes, process isolation is definitely not a security feature (/s). And of course no word about binary exploit. What about an attacker that can jmp into the whole kernel code ?

                                    No Users

                                    Oh no, users privilege separation is definitely not a security feature (/s). If we remove them we can’t bruteforce their password. We immediately log in as “god” instead (better than root, you’re the kernel).

                                    [the shell,] It’s an antiqutated concept that only lends it’s hands towards those who want to do your company harm.

                                    So you are adressing this speech to company-owners that have no idea of how UNIX-style computer works? I noticed that when you offered to expose kernel memory address rather than the system shell. But no shell means harder for an attacker to exploit an existing breach (or an admin to debug), true.

                                    Reduced Attack Surface […] Less code == less exlloits

                                    You made a point here. Kudos.

                                    1. 2

                                      P.S.: I still like the idea of unikernels.

                                      1. 2

                                        Not sure about the first point but to elaborate there is a single process and no facilities to create another process (no fork/exec). Also, unlike other unikernels we actually do have the concept of cpl0 and cpl3 and have opted to utilize vdso style ‘speedups’ where it makes sense. Also, we are explicitly addressing ‘binary exploits’ (eg: shellcode).

                                        As for users - you can’t login to these - at all. There is no ssh/telnet. Half the user syscalls are merely stubs. You can’t su/sudo. The entire concept of a shell doesn’t make sense in an unikernel since it already means you intend to run multiple programs.

                                        While this can execute linux programs I would be hard-pressed to call it a unix.

                                        If you are curious about more of the security properties I wrote up a short blogpost on it that is more technical.

                                        https://nanovms.com/dev/tutorials/assessing-unikernel-security

                                        1. 2

                                          From that blog entry (which I certainly wouldn’t accuse of being “technical”):

                                          This is reminiscent of the “unikernels are un-debuggable” points brought up earlier. This is funny too considering one of the authors started working on an [sic] unikernel specific debugger…. For real.

                                          Is that a reference to my blog entry on unikernels being unfit for production? On the one hand, I can’t imagine it is – I am certainly not engaged in any unikernel debugger, and my piece was clearly single-authored – but on the other, I don’t know what else this could be a reference to. Could you clarify?

                                          1. 1

                                            hi Bryan :)

                                            It wasn’t directed at you. One of the NccGroup authors of the paper in question ended up writing a unikernel debugger because at the time there was no gdb stub to hook up to whatever implementation he was looking into.

                                            1. 1

                                              Is it not easier to debug when the source is fully available ? * hum hum * ;)

                                          2. 1

                                            Nice to know there is some privilege structure that remains. The article linked talks for itself.

                                            But I get the idea : noone will ever log (or break) into the kernel and an attacker will have to think again (I still don’t consider SSH with a strong key and no password a large threat).

                                            It looks like a reversed approach: instead of dropping privileges as the program goes (changing for a different user than root, chroots, pledge, capabilities, read-only filesystems…), do not expose the vunerable resources on the first place: filesystem privileges can be implemented by a fileserver in which the unikernel logs in, or that only expose (or have at all) the data the application is allowed to access.

                                            But there are many cases where there is a need to parse complex strings (risky) theb do something, then send the result back.

                                            To mitigate the risk, it is possible to split the job into multiple processes : one that parse, one that does and send back, so that the risky work of parsing the user input gets done in a separate memory space and process as the one that have full access to the protected data.

                                            How to do this with unikernels? Many tiny unikernels with networking to communicate rather than a pipe()fork()? Then there is a need for strong firewalling between the two or implementing authentication and crypto for every pipe replaced. This adds complexity and TCP overhead and require more configuration so that the two point at each other.

                                            Yet it permits a much more distributed system where it is possible to have a lot of distributed parsers and a few workers or the opposite.

                                            1. 1

                                              Threads! Maybe there is some way to limit privileges per thread (I’ll read the doc). Then while it is still the same memory space, one thread that have no network access, one thread that have no storage access. Woops, am I baking thread-ish processes? Not the good approach for unikernels…

                                              I’ll think again…

                                          3. 2

                                            In my part of the industry most software is deployed into a vm where it is the only thing running. Having access to that one thing is functionally equivalent to getting root on the box.

                                            1. 1

                                              In my part of the industry, it’s all vms running debian running apache running php running a framework running a website. In my part of the industry clients get their wordpress plugin hacked in multiple ways every day. I’ll probably quit my part of the industry.

                                              At works, things are managed from the inside of the system. Each have an agent that does the monitoring (a la puppet).

                                              But for every hack I’ve seen (not that many, it’s not on me to spot them…) there were basically no attempt to get root.

                                              1. 2

                                                If you’ve already gained control of the website, what point is there to getting root?

                                                1. 1

                                                  Yes, exactly! Not that much point!

                                                  Well still, attacking other machines, take over the infrastructure some more or attacking the hypervisor (VM escape)… But I bet the main target is taking over “whatever the client is doing” (taking over the application running as non-root).

                                          1. 4

                                            The book describes a disfunctional team. I have never understood why people speak so highly of it (I guess books about heroes spell much better, and also Kiddler was not in a position to know otherwise).

                                            1. 6

                                              Have you actually read it? Honestly, I don’t think anyone would read the book and come away with the conclusion that the team was dysfunctional – and no one on the actual team seems to have come away with that conclusion either. I think it’s also a mistake to claim that the book is about “heroes”; yes, it’s an epic tale involving heroic effort, but Kidder stops short of lionizing the protagonists – he paints them in full, flaws and all.

                                              1. 3

                                                I really enjoyed the book, but it’s definitely a tale of abuse and bad management.

                                                1. 1

                                                  How do you figure?

                                                  1. 3

                                                    From the Wikipedia page of the book:

                                                    “Tom West practices the “Mushroom Theory of Management” – “keeping them in the dark, feeding them shit, and watch them grow.” That is, isolating the design team from outside influences and, instead, using the fear of the unknown to motivate the team.”

                                                    1. 5

                                                      I don’t think that the Wikipedia page is really doing the book justice. The “Mushroom Theory of Management” is something that is mentioned in passing, not a mantra of West’s. None of this is to lionize West – he’s a complicated figure, and Kidder paints a suitable nuanced picture – but (both from the book and from anecdotes of those who served on the Eagle team), West used trust much more than fear, especially when one considers the milieu of computer engineering in the 1970s!

                                                1. 7

                                                  One thing that is clear to me: the author hasn’t actually written much (or perhaps any) Rust. This is clear to me because I think one of the traps that the merely Rust-curious fall into is a disproportional fear and loathing of the borrow checker. This is disproportional because it ignores many of the delightful aspects of Rust – for example, that algebraic types in a non-GC’d language represent a revolution in error handling. (I also happen to love the macro system, Cargo, the built-in testing framework, and a bunch of other smaller things.) Yes, the lack of things like non-lexical lifetimes can make for some wrestling with the borrow checker, but once one is far enough into Rust to encounter these things, they are also far enough in to appreciate the value it brings to systems programming.

                                                  To sum, the author shouldn’t weigh in on Rust (or any language, really) so definitively without having written any – or at least make clear that his perspective is informed by reading blog entries, not actual experience…

                                                  1. 1

                                                    One thing that is clear to me: the author hasn’t actually written much (or perhaps any) Rust. This is clear to me because …

                                                    To sum, the author shouldn’t weigh in on Rust (or any language, really) so definitively without having written any – or at least make clear that his perspective is informed by reading blog entries, not actual experience…

                                                    I believe it wasn’t your intent, but your commentary reads a bit like “Only true Rustaceans should be allowed to talk about Rust”.

                                                    1. 3

                                                      Everyone should be allowed to talk about Rust. There is no authority that deserves to have the power to decide which people can or cannot talk about Rust.

                                                      That said, it’s also fine to say that the author’s opinion about Rust is untrustworthy because it bears the hallmarks of someone who has read about Rust but not actually used it themselves in any meaningful way. I myself agree that it’s possible to write lots of useful rust code without running into situations where the borrow checker trips you up, and that some of Rust’s best innovations are the “small” things like the algebraic types, macros, Cargo, etc. that are now available in a non-GC systems language.

                                                      1. 1

                                                        it bears the hallmarks of someone who has read about Rust but not actually used it themselves in any meaningful way

                                                        I still use rustlang but share the same opinion as the author. Did I write enough of it to be trustworthy? :)

                                                        Rust’s best innovations are the “small” things like the algebraic types, macros, Cargo, etc. that are now available in a non-GC systems language

                                                        Nothing on that list was rustlang’s innovation.

                                                  1. 6

                                                    This news caused the public release for XSA-267 / CVE-2018-3665 (Speculative register leakage from lazy FPU context switching) to be moved to today.

                                                    1. 16

                                                      These embargoed and NDA’d vulnerabilities need to die. The system is broken.

                                                      edit: Looks like cperciva of FreeBSD wrote a working exploit and then emailed Intel and demanded they end embargo ASAP https://twitter.com/cperciva/status/1007010583244230656?s=21

                                                      1. 8

                                                        Prgmr.com is on the pre-disclosure list for Xen. When a vulnerability is discovered, and the discoverer uses the responsible disclosure process, and the process works, we’re given time to patch our hosts before the vulnerability is disclosed to the public. On balance I believe participating in the responsible disclosure process is better for my customers.

                                                        Pre-disclosure gives us time to build new packages, run through our testing process, and let our users know we’ll be performing maintenance. Last year we found a showstopping bug during a pre-disclosure period: it takes time and effort to verify a patch can go to production. With full disclosure, we would have the do so reactively, with significantly more time pressure. That would lead to more mistakes and lower quality fixes.

                                                        1. 2

                                                          This is a bad response to the issue. The bad guys probably already have knowledge of it and can use it. A few players deemed important should not get advanced notification.

                                                          1. 15

                                                            Prgmr.com qualifies for being on the Xen pre-disclosure list by a) being a vendor of a Xen-based system b) willing and able to maintain confidentiality and c) asking. We’re one of 6 dozen organizations on that list–the criteria for membership is technical and needs-based.

                                                            If you discover a vulnerability you are not obligated to use responsible disclosure. If you run Xen you are not obligated to participate in the pre-disclosure list. The process consists of voluntary coordination to discover, report, and resolve security issues. It is for the people and organizations with a shared goal: removing security defects from computer systems.

                                                            By maintaining confidentiality we are given the ability, and usually the means to have security issues resolved before they are announced. Our customers benefit via reduced exposure to these bugs. The act of keeping information temporarily confidential provides that reduced exposure.

                                                            You have described a voluntary process with articulable benefits as “needing to die,” along with my response being “bad.” As far as I can tell from your comments you claim “the system is broken” because some people “should not get advanced notice.” I’ve described what I do with that knowledge, and why it benefits my users. I’m thankful the security community tells me when my users are vulnerable and works with me to make them safer.

                                                            Can you improve this process for us? Have I misunderstood you?

                                                            1. 11

                                                              Some bad guys might already have knowledge of it. Once it’s been disclosed, many bad guys definitely have knowledge of it, and they can deploy exploits far, far faster than maintainers, administrators and users can deploy fixes.

                                                              1. 8

                                                                You’re treating “the bad guys” like they’re all one thing. In actuality, there’s a string of bad guys from people who will use a free, attack tool to people who will pay a few grand for one to people who can customize a kit if it’s just a sploit to people who can build a sploit from a description to rare people who had it already. There’s also a range in intent of attackers from DOS to data integrity to leaking secrets. The folks who had it already often just leak secrets in stealthy way instead of do actual damage. The also use the secrets in a limited way compared to average, black hat. They’re always weighing use vs detection of their access.

                                                                The process probably shuts down quite a range of attackers even if it makes no difference for the best ones who act the sneakiest.

                                                                1. 4

                                                                  The process probably shuts down quite a range of attackers even if it makes no difference for the best ones who act the sneakiest.

                                                                  I believe the process is so effective at shutting down “quite a range of attackers” that it works despite: a) accidental leaks [need for improvement of process] b) intentional leaks [abuse] c) black hats on the pre-disclosure list reverse engineering an exploit from a patch. [fraud] In aggregate, the benefit from following the process exceeds the gain a black hat would have from subverting it.

                                                            2. 9

                                                              Well, it’s complicated. (Disclosure: we were under the embargo.)

                                                              When a microprocessor has a vulnerability of this nature, those who write operating systems (or worse, provide them to others!) need time to implement and test a fix. I think Intel was actually doing an admirable job, honestly – and we were fighting for them to broaden their disclosure to other operating systems that didn’t have clear corporate or foundation backing (e.g., OpenBSD, Dragonfly, NetBSD, etc). That discussion was ongoing when OpenBSD caught wind of this – presumably because someone who was embargoed felt that OpenBSD deserved to know – and then fixed it in the worst possible way. (Namely, by snarkily indicating that it was to address a CPU vulnerability.) This was then compounded by Theo’s caustic presentation at BSDCan, which was honestly irresponsible: he clearly didn’t pull eager FPU out of thin air (“post-Spectre rumors”), and should have considered himself part of the embargo in spirit if not in letter.

                                                              For myself, I will continue to advocate that Intel broaden their disclosure to include more operating systems – but if those endeavoring to write those systems refuse to honor the necessary secrecy that responsible disclosure demands (and yes, this means “embargoed and NDA’d vulnerabilities”), they will make such inclusion impossible.

                                                              1. 18

                                                                We could also argue Theo’s talk was helpful in that the CVE was finally made public.

                                                                Colin Percival tweeted in his thread overview about the vulnerability that he learned enough from Theo’s talk to write an exploit in 5 hours.

                                                                If Theo and and the OpenBSD developers pieced enough together from rumors to make a presentation that Colin could turn into an exploit in hours, how long have others (i.e., bad guys) who also heard rumors had working exploits?

                                                                Theo alone knows whether he picked-up eager FPU from developers under NDA. Even if he did, there’s zero possibility outside of the law he lives under (or contracts he might’ve signed) that he’s part of the embargo. As to the “spirit” of the embargo, his decision to discuss what he knew might hurt him or OpenBSD in the future. That was his call to make. He made it.

                                                                Lastly, I was at Theo’s talk. Caustic is not how I would describe it, nor would I categorize it as irresponsible. Theo was frustrated that OpenBSD developers who had contributed meaningfully to Spectre and Meltdown mitigation had been excluded. He vented some of that frustration in the talk. I’ve heard more (and harsher) venting about Linux in a 30 minute podcast than all the venting in Theo’s talk.

                                                                On the whole Theo’s talk was interesting and informative, with a sideshow of drama. And it may have been what was needed to get the vulnerability disclosed and more systems patched.


                                                                Disclosure: I’m an OpenBSD user, occasional port submitter, BSDCan speaker and workshop tutor, FreeNAS user and recommender, and have enjoyed many podcasts, some of which may have included venting.

                                                                1. 4

                                                                  If Theo and and the OpenBSD developers pieced enough together from rumors to make a presentation that Colin could turn into an exploit in hours, how long have others (i.e., bad guys) who also heard rumors had working exploits?

                                                                  It was clear to me the day Spectre / Meltdown were disclosed that there would be future additional vulnerabilities of the same class based on that discovery. I think there is circumstantial evidence suggesting the discovery was productive for the people who knew about it in the second half of 2017 before it was publicly disclosed. One can safely assume black hats have had the ability to find and use novel variations in this class of vulnerability for at least six months.

                                                                  If Theo did pick up eager FPU from a developer under embargo that demonstrates just how costly it is to break embargo. Five hours, third hand.

                                                                  1. 4

                                                                    If Theo did pick up eager FPU from a developer under embargo that demonstrates just how costly it is to break embargo. Five hours, third hand.

                                                                    I have absolutely no idea what point you’re trying to make. Certainly, everyone under the embargo knew that this would be easy to exploit; in that regard, Theo showed people what they already knew. The only new information here is that Theo is every bit as irresponsible as his detractors have claimed – and those detractors would (of course) point out that that information is not new at all…

                                                                    1. 1

                                                                      With respect, how is Theo irresponsible for reducing the time the users of his OS are vulnerable?

                                                                      Like, the embargo thing sounds a lot to the ill-informed like some kind of super-secret clubhouse.

                                                                  2. 4

                                                                    Theo definitely wasn’t part of the embargo, but it’s also unquestionable that Theo was relying on information that came (ultimately) from someone who was under the embargo. OpenBSD either obtained that information via espionage or via someone trying to help OpenBSD out; either way, what Theo did was emphatically irresponsible. Of course, it was ultimately his call – but he is not the only user of OpenBSD, and is unfortunate that he has effectively elected to isolate the community to serve his own narcissism.

                                                                    As for the conjecture that Theo served any helpful role here: sorry, that’s false. (Again, I was under the embargo.) The CVE was absolutely going public; all Theo did was marginally accelerate the timeline, which in turn has resulted in systems not being as prepared as they otherwise could be. At the same time, his irresponsible behavior has made it much more difficult for those of us who were advocating for broader inclusion – and unfortunately it will be the OpenBSD community that suffers the ramifications of any future limited disclosure.

                                                                    1. 6

                                                                      Espionage? You’re suggesting one of:

                                                                      1. Someone stole the exploit information, leaked it to the OpenBSD team, a team known for proactively securing their code, on the off-chance Theo would then further leak it (likely with mitigation code), causing the embargoed details to be released sooner than expected,

                                                                      2. OpenBSD developers stole the exploit information, then leaked it (while committing mitigation code), causing the embargoed details to be released sooner than expected.

                                                                      The first doesn’t seem plausible. The second isn’t worthy of you or any of the developers on the OpenBSD team.

                                                                      I’m sure you’ve read Colin’s thread. He contacted folks under embargo after he wrote his exploit code based on Theo’s presentation. The release timeline moved forward. OSs that had no knowledge of the vulnerability now have patches in place. Perhaps those users view “helpful” in a different light.


                                                                      Edit: Still boggling over the espionage comment. Had to flesh that out more.

                                                                      1. 8

                                                                        Theo has replied:

                                                                        In some forums, Bryan Cantrill is crafting a fiction.

                                                                        He is saying the FPU problem (and other problems) were received as a leak.

                                                                        He is not being truthful, inventing a storyline, and has not asked me for the facts.

                                                                        This was discovered by guessing Intel made a mistake.

                                                                        We are doing the best for OpenBSD. Our commit is best effort for our user community when Intel didn’t reply to mails asking for us to be included. But we were not included, there was no reply. End of story. That leaves us to figure things out ourselves.

                                                                        Bryan is just upset we guessed right. It is called science.

                                                                        He’s also offered to discuss the details with Bryan by phone.

                                                                        1. 4

                                                                          Intel still has 7 more mistakes in the Embargo Execution Pipeline™️ according to a report^Wspeculation by Heise on May 3rd.

                                                                          https://www.heise.de/ct/artikel/Exclusive-Spectre-NG-Multiple-new-Intel-CPU-flaws-revealed-several-serious-4040648.html

                                                                          Let the games begin! 🍿

                                                                          1. 1

                                                                            What’s (far) more likely: that Theo coincidentally guessed now, or that he received a hint from someone else? Add Theo’s history, and his case is even weaker.

                                                                            1. 13

                                                                              While everyone is talking about Theo, the smart guys figuring this stuff out are Philip Guenther and Mike Larkin. Meet them over beer and discuss topics like ACPI, VMM, and Meltdown with them and you won’t doubt anymore that they can figure this stuff out.

                                                                              1. 6

                                                                                In another reply you claim your approach is applied Bayesian reasoning, so let’s go with that.

                                                                                Which is more likely:

                                                                                1. A group of people skilled in the art, who read the relevant literature, have contributed meaningful patches to their own OS kernel and helped others with theirs, knowing that others besides themselves suspected there were other similar issues, took all that skill, experience and knowledge, and found the issue,

                                                                                or

                                                                                1. Theo lied.

                                                                                Show me the observed distribution you based your assessment on. Show me all the times Theo lied about how he came to know something.

                                                                                Absent meaningful data, I’ll go with team of smart people knowing their business.

                                                                                1. 4

                                                                                  Absent meaningful data

                                                                                  Your “meaningful data” is 11 minutes and 5 seconds into Theo’s BSDCan talk: “We heard a rumor that this is broken.” That is not guessing and that is not science – that is (somehow) coming into undisclosed information, putting some reasonable inferences around it and then irresponsibly sharing those inferences. But at the root is the undisclosed information. And to be clear, I am not accusing Theo of lying; I am accusing him of acting irresponsibly with respect to the information that came into his possession.

                                                                                  1. 3

                                                                                    Here is at least one developer’s comment on the matter. He points to the heise.de article about Spectre-NG as an example of the rumors that were floating around. That article is a long way from “lazy FPU is broken”.

                                                                                    Theo has offered to discuss your concerns, what you think you know, what he knew, when and how. He’s made a good-faith effort to get his cellphone number to you. If you don’t have it, ask.

                                                                                    If you do have his number, call him. Ask him what he meant by “We heard a rumor that this is broken.” Ask him what rumor they heard. Ask him whether he was referring to the Spectre-NG article.

                                                                                    Seriously, how hard does this have to be? You engaged productively with me when I called you out. You’ve called Theo out. Talk to him.

                                                                                    And yes, I get it. Your chief criticism at this point is responsible disclosure. But as witnessed by the broader discussion in the security community, there’s no single agreed-upon solution.

                                                                                    While you’ve got Theo on the phone you can discuss responsible disclosure. Frankly, I suggest beer for that part of the discussion.


                                                                                    Edit: Clarify that Florian wasn’t saying he knew heise.de were the source.

                                                                                  2. 0

                                                                                    Reread the second sentence in my reply you linked.

                                                                                  3. 2

                                                                                    This is plain libel, pure and simple.

                                                                                    1. -2

                                                                                      It is Bayesian reasoning, pure and simple.

                                                                                      That said, this is a tempest in a teacup, so call it whatever you want; I’m gonna go floss my cat.

                                                                                2. 6

                                                                                  Sorry – I’m not accusing anyone of espionage; apologies if I came across that way.

                                                                                  What I am saying is that however Theo obtained information – and indeed, even if that information didn’t originate with the leak but rather by “guessing” as he is now apparently claiming – how he handled it was not responsible. And I am also saying that Theo’s irresponsibility has made the job of including OpenBSD more difficult.

                                                                                  1. 9

                                                                                    The spectre paper made it abundantly clear that addtional side channels will be found in the speculative execution design.

                                                                                    This FPU problem is just one additonal bug of this kind. What I’d like to learn from you is:

                                                                                    1. What was the original planned public disclosure date before it was moved ahead to today?

                                                                                    2. Do you really expect that a process with long embargo windows has a chance of working for future spectre-style bugs when a lot of research is now happening in parallel on this class of bugs?

                                                                                    1. 5
                                                                                      1. The original date for CVE-2018-3665 was July 10th. After the OpenBSD commit, there was preparation for an earlier disclosure. After Theo’s talk and after Colin developed his POC, the date was moved in from July 10th to June 26th, with preparations being made to go much earlier as needed. After the media attention today, the determination was made that the embargo was having little effect and that there was no point in further delay.

                                                                                      2. Yes, I expect that long embargo windows can work with Spectre-style bugs. Researchers have been responsible and very accommodating of the acute challenges of multi-party disclosure when those parties include potentially hypervisors, operating systems and higher-level runtimes.

                                                                                      1. 10

                                                                                        Thanks for disclosing the date. I must say I am happy that my systems are already patched now, rather than in one month from now.

                                                                                        I’ll add that some new patches with the goal of mitigating spectre-class bugs are being developed in public without any coordinated disclosure:

                                                                                        http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/9474cbef7fcb61cd268019694d94db6a75af7dbe

                                                                                        https://patchwork.kernel.org/patch/10202865/

                                                                                    2. 5

                                                                                      Thanks for the clarification.

                                                                                      I don’t think early disclosure is always irresponsible (the details of what and when matter). Others think it’s never irresponsible; and some that it’s always irresponsible. Good arguments can be made for each position that reasonable people can disagree about and debate.

                                                                                      One thing I hope we can all agree on is that we need clear rules for how embargoes work (probably by industry). We need clear, public criteria covering who, what, when and how long. And how to get in the program, ideally with little or no cost.

                                                                                      It’s a given that large companies like Microsoft will be involved. Open-source representatives should have a seat at the table as well. But “open source” can’t just mean Red Hat and a few large foundations. OSs like OpenBSD have a presence in the ecosystem. We can’t just write the rules with a “You must be this high to ride” sign at the door.

                                                                                      And yeah, Theo’s talk might make this more difficult going forward. Hopefully both sides will use this event as an opportunity to open a dialog and discuss working together.

                                                                                      1. 6

                                                                                        Right, I completely agree: I’m the person that’s been advocating for that. I was furious with Intel over Spectre/Meltdown (despite our significant exposure, we learned about it when everyone else did), and I was very grateful for the work that OpenBSD and illumos did together to implement KPTI. This time around, I was working from inside the embargo to get OpenBSD included. We hadn’t been able to get to where we needed to get, but I also felt that progress was being made – and I remained optimistic that we could get OpenBSD disclosure under embargo.

                                                                                        All of this is why I’m so frustrated: the way Theo has done this has made it much more difficult to advocate this position – it has strengthened the argument of those who believe that OpenBSD should not be included because they cannot be trusted. And that, in my opinion, is a shame.

                                                                                        1. 11

                                                                                          Look at it from OpenBSD’s perspective though. They (apparently) tried emailing Intel to find out more, and were told “no”. What were they supposed to do? Just wait on the hope that someone, somewhere, was lobbying on their behalf to be included, with no knowledge of that lobbying?

                                                                        1. 4

                                                                          I think Bryan’s straw-manning the Amazon leadership principles a bit. Integrity is woven throughout essentially all of them; earning trust, insisting on the highest standards, and customer obsession all mandate a high foundational level of integrity. That’s not to say every Amazon product, process or leader succeeds all the time, but it’s a disservice to tens of thousands of high-integrity Amazonians to suggest we don’t care deeply about it.

                                                                          (disclaimer: I work for Amazon, but don’t speak for Amazon here)

                                                                          1. 7

                                                                            I don’t agree in that I don’t think that integrity is in fact implied by any of them. For example, you can easily earn trust without integrity (e.g. Madoff Investment Securities), insist on high standards without integrity (e.g., Enron), and be customer obsessed without integrity (e.g., Uber). I’m not accusing Amazon of not having integrity (or at least, not necessarily), but I do accuse it of not having integrity as an espoused core principle (or even among the first 14!). And the bigger problem isn’t Amazon (frankly), it’s those that Amazon inspires, like (say) Uber. Uber’s core values (the idea for which supposedly came from an exec who came from Amazon) are entirely devoid of integrity – and indeed, some (like “fierceness” and “encourage toe-stepping”) practically assure that integrity be sacrificed to satisfy them.

                                                                            1. 1

                                                                              I’ll agree it’s possible to read the Amazon LPs the way you do, but I think that’s unnecessarily incharitable in service of your (otherwise great and thought provoking) presentation.

                                                                              At Amazon, ‘Earns Trust’ means what it literally says on its face; to earn the trust of your customers, your peers, your management, and your reports, without equivocation. And it’s drilled in, correctly, that trust is fragile; it is far easier to break it than to ever fix it once it’s broken. In order to keep and maintain trust, you have to work with integrity, compassion, an adult viewpoint, introspection, deliberation, openness, thoughtfulness, and a bunch of other non-mentioned (but critically important) motivations, and you have to keep those front and center all the time.

                                                                              I think it’d be a pretty great idea if we exposed some of the internal training materials about the principles to the public. I think it’d help in discussions like this. I’ll bring that up.

                                                                            2. 2

                                                                              My new minimum bar for a companies value statement is “Would Uber’s behaviour break these principles?”. Looking at Amazon’s principles, I don’t really see anything there that would be inconsistent with how Uber runs their business except for perhaps “Earn Trust”. I don’t think they have the same ethical values, but they don’t distinguish themselves from Uber in those principles.

                                                                            1. 1

                                                                              This person going on about integrity is of the company that dropped lifetime plans they’d sold once they became too expensive, right?

                                                                              1. 22

                                                                                Yes – though the lifetime plans long pre-date me (indeed, they date back to 2004), and the decision to turn them off in 2012 was one that I (unfortunately) had nothing to do with. (I was the VP of Engineering at the time.) No executive involved in any aspect of lifetime accounts (either the decision to offer them or the unfortunate decision to end them) has anything to do with the company – and indeed, the Joyent of that era is what many of us veterans refer to as the Bad Old Days. When I came to Joyent in 2010, I quickly realized that the founders of the company and I had some fundamental disagreements at some very basic levels – of which the lifetime plans (both that they were offered and the way in which they were terminated) is a particularly vivid example. The lesson from it all for me was that values absolutely must emanate from the top of a company, and that the most important (and most difficult) property to vet in any executive is their integrity. Fortunately, we at Joyent now have the luxury of working for a high-integrity CEO – but it took a while to get there (he was hired in 2014, over two years after the founding CEO was fired), and then longer still for him to build an organization around him.

                                                                                If anything, this entire odyssey has made me much more overt and assertive about my own values. So yes, it’s the same company (or the same name, anyway) – and in a perverse and roundabout way, that’s not an accident.

                                                                                1. 2

                                                                                  It would be a violation of integrity if they did not honor the lifetime plans that they sold. There’s nothing wrong with changing your pricing structure. You can’t make them sell lifetime plans forever.

                                                                                  1. 2

                                                                                    100% agreed. My understanding was that Joyent had done the former.

                                                                                1. 2

                                                                                  “I’m sorry, if you’ve got one principle in your organisation, its integrity? Right?”

                                                                                  Wrong. Effectiveness at achieving organization’s goals is the most important principle. Everything else comes second. If Im wrong, just get a market report on desktop OS’s where Microsoft shoukd either be bankrupt or less than 1% share. Similarly for mainframes where IBM probably is barely scraping by. Im probably typing this on an OpenMoko with cutting-edge hardware whose specs are open to aid trustworthy, competitive, driver development. As is EDA and fab’s cell libraries. Licensing for Oracle is a one-click process with exceedingly cheap results.

                                                                                  Good that high integrity prevails in the corporate world. I can wait to see what President(s) got elected and laws passed through the campaign contributions of good corporations. The future is probably bright for all economic classes.

                                                                                  1. 28

                                                                                    Okay, so to play that out: if an organization has to choose between integrity and achieving its goals, it should choose to achieve its goals?! Perhaps we’re operating under different definitions of integrity (perhaps you have inferred it to mean “technical purity”?), but I can’t imagine how anyone would want to work in an organization where integrity is knowingly sacrificed to achieve larger goals – and I further don’t see how this could possibly be controversial.

                                                                                    1. 3

                                                                                      To add another wrench to the mix: if your value system is that breaking the rules is ok to achieve goals, you can still have integrity (in the technical sense of the word) as long as you are clear that your values allow this. To use Uber as an example as well: the results of the Uber scandals, I find, ethically problematic. However they are completely in line with my understanding of the Uber culture and I’m not surprised or feel scandals demonstrate any dualism in Uber. Their communications have always been to do whatever it takes to succeed.

                                                                                      Note that this definition of ‘integrity’ is that someone/group is the same internally and externally. Unified. Which is, perhaps, a different definition that many people have or how it’s used day-to-day. This definition comes from Ray Dalio’s Principles book.

                                                                                      1. 3

                                                                                        So, to play devil’s advocate, what competitive edge does integrity give in a modern business? What is the dollars-and-cents value of scruples?

                                                                                        Also, what do you you think caused this shift, if indeed it was a shift?

                                                                                        1. 17

                                                                                          To me, it’s not a competitive edge – it’s a constraint. (And one that isn’t very constraining, honestly – especially if one takes a slightly longer view in terms of outcomes.)

                                                                                          As to what has caused a seeming shift away from valuing integrity, I can only speculate, but I suspect that it’s due to a confluence of factors – and I very much hope that it’s a localized and transient! That is, I hope that we will collectively look back on (say) Uber as symptomatic of the excesses of an era rather than a portent of worse to come…

                                                                                          1. 1

                                                                                            And one that isn’t very constraining, honestly – especially if one takes a slightly longer view in terms of outcomes.

                                                                                            Yeah… Especially leadership changes or post-acquisition. I could never work for a company intending to get acquired or IPO while telling my customers I had their best interests in mind. Unless, I open-source almost everything and otherwise gave them a clean, exit strategy so a future tyrant couldn’t lock them in. Most tech people in proprietary or SaaS don’t do that. I know your company open-sourced a bunch of stuff. I don’t remember all the product details. Might be an exception. :)

                                                                                          2. 6

                                                                                            Is this what we’re saying…

                                                                                            Do whatever makes money. If breaking the law doesn’t reduce profits, then break it. If being amoral (whose morality?!) doesn’t reduce profits, be amoral! etc.

                                                                                            Because that seems to be the contemporary mode.

                                                                                            1. 1

                                                                                              I’m not saying that. I’m saying effectiveness. In some organizations, that means focus on money, In others, it might mean other things. Depends on the goal. Let’s illustrate with an more obvious and extreme example that happens in the real-world: non-profit that gets money to pay for cancer treatments, esp kids. The people on the sales end are naturally going to be talking to a lot of people who didn’t want anyone to sell them anything. They’re also going to talk to people that will only do it if they like the seller, works within their ideology, etc. As in sales in general, the person doing the selling should be bending their personality and truth to the ears of the listener to increase odds they’ll accepts while simultaneously constraining themselves enough to avoid long-term, negative impact. The latter is not even always necessary as the organization can “fire” the one person that got caught saying they were out of line and not representing the organization properly.

                                                                                              Meanwhile, this person acting on effectiveness more than integrity is saving a lot of lives getting a lot of donations. Acting with highest integrity for most people would reduce persuasive ability, reduce donations, and kill people as a side effect. Would you kill people regularly in such a job just to say you have higher integrity? ;)

                                                                                              1. 1

                                                                                                Would you kill people regularly in such a job just to say you have higher integrity?

                                                                                                If you were receiving cancer treatment, would you accept it if you knew it came from swindling people?

                                                                                                1. 2

                                                                                                  I don’t know. That would be a tough decision. I stay realist. So, I’d decline and turn them in if I believed the money would get back to the victims. Otherwise, I’d be choosing between taking it or dying for the toys a LEO would buy for themselves after seizing it. I won’t die for that. So, I’d take the money and go after them if Im in remission. Also, a proxy to do same if I die.

                                                                                          3. 1

                                                                                            I didn’t say I wanted to work at such a place. I countered the idea that integrity was most important by noting that most successful businesses sacrifice it in some way regularly to gain market share, profit, etc. All the market leaders do as well for all far as I can tell. So do almost all politicians and companies that write the laws that people with integrity are forced to operate under.

                                                                                            Given all that, I think businesses should be willing to sacrifice integrity when it helps them achieve goals, esp long-term. I prefer them to operate with integrity but their environment means most won’t. Many will even go under when competitors use low integrity marketing or cost-cutting strategies. Example: try competing in hardware market by having well-paid, first-world people assemble the components with 40 hour work week, breaks, vacations, and so on. You’ll go bankrupt if it ain’t defense or some high-margin industry.

                                                                                          4. 6

                                                                                            I think both you and Bryan Cantrill have somehow conflated values & morals with capitalism. It’s apples and oranges. There is an argument to be made for (moral) principles/values, and there is an (economic) argument to be made for achieving goals when operating within the capitalist system. But the actions that these arguments would imply are not necessarily aligned and perhaps not even reconcilable.

                                                                                            Incidentally, I hold integrity in the highest regard in all endeavours (not just business). Amazing things can be built on the foundation of integrity.

                                                                                            1. 4

                                                                                              the actions that these arguments would imply are not necessarily aligned and perhaps not even reconcilable.

                                                                                              So an action that’s carried out to achieve a capitalist end is simply external to any moral universe? The “values & morals” just don’t apply once an economic motivation comes into play?

                                                                                              1. 2

                                                                                                many people here would love to say “no”, but it seems “yes” is the right answer here. Look at the “shareholder value” driven world today. More, more, more is the only thing that counts.

                                                                                                1. 1

                                                                                                  That’s not what I said. I said that acting morally may not be aligned with achieving success in a capitalist system, and may even be impossible in some cases (case in point: today’s oil companies). I’m not at all advocating amorality, but my point is that capitalism as a system has no built-in morality; morality is an independent set of constraints. People, however, have to operate within both sets of constraints, and choose their course of action accordingly.

                                                                                                  1. 2

                                                                                                    Sure. I was just trying to poke a little at “conflated values & morals with capitalism”, because I don’t think the rant in this post does that. On my understanding, it’s saying rather that the problem with unconstrained capitalism is exactly that it doesn’t have values & morals applied to it often enough, if at all. I agree with that, and in my experience that’s usually the case because people metaphorically shrug their shoulders regarding values & morals, arguing “well of course not, my hands are tied, financial motivations have to take precedence, what would you have me do?”, as if this gives people and organisations free passes to behave badly in the name of profit. What I’d have people do would be to take a bit of personal responsibility for the world they live in and exercise a bit of values & morals in order to make that world a little bit less shitty, rather than them a little bit richer. In my mind, that’s what civilization is about; a generalized move away from selfish actions towards those aimed at benefiting a collective; and moves towards increased selfishness and individual people’s and organisation’s gains at the expense of others are steps back from civilization. But I recognise I probably live in cloud cuckoo land as far as most people are concerned, and the way politics and societies are moving right now seem to bear that out.

                                                                                                    1. 1

                                                                                                      I may have misinterpreted the comment about Scott McNealy as I don’t know the details of that; it sounded like a moral judgement was cast as an act of a capitalist.

                                                                                                      I agree with your view of civilisation, and I think it’s beneficial in that it’s aligned with our biological underpinnings - we evolved as social creatures, and hence complete selfishness and individualism are not natural for us.

                                                                                                      1. 6

                                                                                                        Ah, got you. I thought the point of the McNealy reference was that even he, the most die-hard fan of free-market capitalism, still found it important to act morally and not lie or cheat or have to hide things from his children. Thus, that one could probably always make more profit by being less moral, but would then have to hide it from one’s children. Thus, that acting morally and making a profit aren’t incompatible, and being a capitalist doesn’t mean you have to act amorally (or immorally), but that you can reach some kind of compromise. And what Bryan Cantrill seemed to be saying was, hello, aren’t more and more companies refusing that compromise, targeting profit above all else, moral compass and/or integrity be damned, and isn’t that something we should be fighting against?

                                                                                                        The thing is, coming from a Western European background, and having been raised by post-war Europeans, I find it hard to accept the kind of moral arguments for self-interest that US libertarians often make (e.g. that it’s immoral to provide health care, welfare, or any kind of help that stops people having to rely on themselves alone and fighting tooth and nail for survival) - and so I naturally assume that any morally positive act carried out in a capitalist system is morally positive because it’s intended to temper the capitalist/profit motive. Meaning that there are interpretations of ‘moral’ that are completely different from mine; meaning, although I disagree with them (natch), I might have misinterpreted the article as well, and almost certainly interpreted it according to my bias.

                                                                                                        Regardless, you make a good point in your last paragraph too. I lapse all too easily into thinking “humans are all too selfish, there’s no way forward”, and what I see happening around right now tends to feed into that - but the reality is that humans are neither completely selfish and individualistic nor completely altruistic and collectivist, and the historical swing between the two is a reflection of that fact. Thanks for the reminder.

                                                                                                        1. 3

                                                                                                          I like your interpretation of the post, I think it’s better than mine.

                                                                                                          I also find it hard to accept arguments for self-interest. I come from an Eastern European background, where I saw the rise of kleptocratic capitalism and the tidal wave of self-interest that came along with it. I don’t think the results have been good.

                                                                                                    2. 1

                                                                                                      That’s not what I said. I said that acting morally may not be aligned with achieving success in a capitalist system

                                                                                                      That’s true. Those doing the most good need the most success in such a system if voters are apathetic. We see this with the big corporations straight-up writing laws to help them while hurting others. These pass quite a bit of the time. That makes capitalist outcomes and morality tied together in the current system in the U.S.. Probably many places. Good news is nothing stopped many of these billionaires from playing things beneficially once they got the market and got rich. People can still do good taking over as CEO’s, managers with decent budgets, and so on. Not always but plenty of the time.

                                                                                                2. 3

                                                                                                  P. T. Barnum thought integrity was important enough to make it one of his golden rules of money getting -

                                                                                                  https://www.fourmilab.ch/etexts/www/barnum/moneygetting/moneygetting_chap21.html

                                                                                                  1. 0

                                                                                                    Cites anecdote. Fails to disprove the rule.

                                                                                                1. 1

                                                                                                  Roger’s :proc appears twice in the index. The second link goes to the URL for DNS and the Art of Making Systems “Just Complex Enough” but the page loaded is, again, Roger’s :proc.

                                                                                                  7074 says Hello World is missing entirely.

                                                                                                  Anyone know to whom to forward these issues?

                                                                                                  1. 3

                                                                                                    Very sad 7074 Hello World wasn’t taped :( Mainframe hacking is super cool in my book!

                                                                                                    1. 2

                                                                                                      I’d try Bryan Cantrill. He might at least be able to direct you to the right person.

                                                                                                      https://twitter.com/bcantrill

                                                                                                      1. 2

                                                                                                        Sorry about that! We got the DNS link corrected. As for Marianne’s amazing talk, I’m afraid that we can’t make the recording available for legal reasons: while Marianne was authorized to give the talk (and her content was approved) she didn’t have the necessary White House approval to make the video available. Now, one can obviously be bummed out that the White House (and the government more generally) is getting in its own way here (there was nothing in her talk that shouldn’t be made public), but (for my own sanity) I choose to look at it the other way: we are (very!) lucky that technologists like Marianne not only put up with the government but actively try to improve it! So… sorry about that. You missed a great one, but if we put everything up, what reason would there be to make it to the next Systems We Love? ;)

                                                                                                        1. 1

                                                                                                          According to All the talks from Systems We Love, 7074 says Hello World wasn’t recorded.

                                                                                                          DNS and the Art… can be found 7h4m4s into the livestream.

                                                                                                        1. 4

                                                                                                          Genius! I really enjoyed watching it remotely. And, Bryan, if you ever wanna do that in Europe, happy to help ;)

                                                                                                          1. 11

                                                                                                            So, we’re trying to figure that out. Clearly, we’ve tapped into something here – but I imagine I don’t have to tell you that running a conference isn’t exactly a low-stress or low time-commitment endeavor. We also want to figure out what, exactly, constitutes the Systems We Love “brand” so we can franchise it out for others to run their own Systems We Love conferences. We also need to figure out sponsorships for future conferences; we at Joyent have no interest in making money off this, but losing less money would probably be nice. ;)

                                                                                                            Definitely welcome ideas on any/all of this! The one thing we feel we know is that there is clearly demand for this content – which itself is inspiring!

                                                                                                            1. 4

                                                                                                              Right. Yeah, as a co-chair of the European Data Forum 2013 and countless user groups I think I have a bit of an idea what it means. Suggest you have a look at the super successful Devoxx franchise here in Europe as well, as a reference. I’d immediately know two locations in Europe (incl. partner and logistics): CodeNode in London (with SkillsMatter) and Software Circus in Amsterdam (with Container Solutions) + I can certainly ask our friends at MSFT if they wanna support this as well.

                                                                                                              This is such an awesome format and it would be a really great thing to make this accessible in person for Europe. Happy to do whatever it takes to make it happen!

                                                                                                          1. 5

                                                                                                            This looks like it’ll be a lot of fun. I’m planning on taking a 1 day trip up from southern California to attend.

                                                                                                            1. 5

                                                                                                              We’re glad you’re coming, and excited to see the enthusiasm for it – as simple as the notion is, it seems to have struck a nerve. And please consider submitting a talk idea as well!

                                                                                                              1. 2

                                                                                                                The whole conference reminds me a lot of !!Con, which I spoke at in 2014. I plan to submit a talk for this, though haven’t quite stumbled upon a good idea yet.

                                                                                                                Thanks for putting this together!

                                                                                                                1. 3

                                                                                                                  The similarity to !!Con is half (but only half) coincidence: when I approached Julia about being on the PC for Systems We Love, she mentioned her work on !!Con, a conference that I have never attended but have coveted from afar. Julia agreed to be on the Systems We Love PC (w00t!) and gave some very helpful feedback in terms of both drafting the call for proposals and structuring the mechanics of the PC. So while not directly inspired by !!Con, it’s also true that it taps into the same zeitgeist – may it be as successful!

                                                                                                                2. 1

                                                                                                                  Nice write-up on i432. Glad you took some time to read on it. I’ll add that Intel learned from their mistake quite a bit in the design of i960. It’s worth checking out. It’s basically a RISC + object descriptors + error handling + fault tolerance + parallel processing. It overdid it with more mistakes due to requirements of ridiculous BiiN project. They canceled that then started selling it as an embedded CPU. It performed well. Moreover, I see a subset of it that would’ve made a fine alternative to Intel 286, etc with way better security, reliability, and multicore later.

                                                                                                                  https://en.wikipedia.org/wiki/Intel_i960

                                                                                                                  What did you think of it?