1. 10

    I would say a subset of git is simple to use.

    I’m personally looking forward to pijul world domination. It absolutely is not ready today, but when it is…there will be no reason to use anything else besides for legacy reasons.

    1. 3

      pijul wouldn’t be appropriate here as it’s GPL’ed.

      1. 5

        Well, so is git. I don’t know what Stefan is doing, but unless he’s doing a clean-room reverse engineering of the git storage formats (and this isn’t easy; the packfiles in particular are not very well documented by git), his work is also likely to be derivative of git.

        1. 4

          Clean room reverse engineering is overkill in most jurisdictions, as reverse engineering for compatibility purposes is often expressly allowed. Reading the GPL source of git to understand its internals doesn’t necessarily taint any code that you write with its license.

          1. 6

            Are you a lawyer? Can I rely on your opinion to avoid legal risk?

            I think it would be reallllly weird for the git copyright holders to create a big stink over it (hell, most Linux copyright holders refuse to ascertain their copyleft), but I don’t know which jurisdictions you’re talking about and the FSF has told me, that when working on Octave, clean-room reverse engineering is what I should be doing.

            Different circumstances, of course, but still, if it could happen to Octave, it could happen with git.

            1. 10

              For virtually everything I needed to know but didn’t, I asked questions to both the current and former maintainers of libgit2 who are friends of mine.

              Copyleft (i.e. copyright) cannot protect ideas. I have read some Git code to figure out how pack files work because this isn’t documented anywhere else in sufficient detail. Not a single line of code was copied. Feel free to compare my pack parser to Git’s. You will find they’re not derived works of each other by any stretch of the imagination. They deal with the same on-disk format of course, so naturally they will parse identical inputs and (hopefully) produce identical outputs, like any Java API function in different JDK implementations ;-)

              I feel legally safe here and would feel confident to hire the gplviolations.org laywer again if needed (I have hired him before for unrelated stuff).

              1. 3

                The FSF lawyers made it very clear that we shouldn’t read Matlab code, but the situation there is different, with a much more hostile relationship. If you’re friends with the libgit devs, it seems unlikely they would ever complain.

                When the gloves come off, avoiding copyright violations isn’t just about making sure things look different. And even APIs are a bit uncertain right now with the Google v. Oracle case.

                1. 7

                  But.. “making things look different” is not what I did at all :-(

                  1. 2

                    I’m pretty sure it doesn’t matter since nobody is going to sue over this, but if you read the libgit code and you used structures, variable names, or anything else that looks like it could suggest that you read the libgit code, they could argue derivative work. I don’t know what you did. But if you read code, it’s hard to not unconsciously copy some of the structure of that code into your own work.

                    I mean, look at the current Katy Perry case. They successfully argued copyright infringement over some superficial similarities and the high probability that Katy Perry was familiar with the infringed work.

                    1. 3

                      I respect the decision of Git’s authors to release their code under GPL and I would never copy any code without keeping its licence intact.

                      To be frank, I think this line of thinking where free software projects could sue each other over copyleft/non-copyleft licencing after having read each other’s code is silly and dangerous. I would hope that FSF lawyers you are talking to wouldn’t support this.

                      The problem with Octave/Matlab you’re describing looks like a policital battle being faught with lawyers who are advising you as an Octave developer in that particular situation. I don’t see how this could possibly be related to my situation.

                      1. 3

                        The FSF has never sued anyone and I can’t imagine they would ever sue a free project that didn’t keep the copyleft terms – they would probably just argue that the copyleft terms apply regardless. The FSF has done all of its GPL compliance without ever going to court – and they do a lot of it. They talk to people until they come into compliance. Their goal is compliance, not any monetary damages.

                        This whole thing is quite academic. Again, you are almost certainly in the clear. Personally, if I relied on GPLed work to write software of my own, I feel morally obligated to make my own work copyleft, but if you feel no such compunction, I don’t think you need to do anything further.

                        The Octave/Matlab thing I describe relates to the nature of what has been tested in court before. We never gone to court over Octave, but if we ever do, we want to make sure we can win. Therefore, we do not read Matlab code, even though a lot of it is available.

              2. 2

                Are you a lawyer? Can I rely on your opinion to avoid legal risk?

                No.

                but I don’t know which jurisdictions you’re talking about

                EU and US are ones I’m aware of that permit RE for interoperability purposes.

                1. 2

                  To be honest even if they were a lawyer, one should not rely on internet discussion as legal advice or opinion. Lobsters may be informative but it is primarily discussion for entertainment. If you are genuinely concerned with legal risk you should seek proper legal counsel.

          2. 2

            Why is pijul not ready?

            1. 9

              Well, opengit describes itself as a clone of Git, which Got is not, so that’s probably why not.

              1. 10

                Farhan already has commit access to Got. But he hasn’t used it much yet.

            1. 7

              I hope they get the UX right, git sure failed on that.

              But for simple, sane version control [0] fossil pretty much solves that problem. It doesn’t use Git, or use Unveil/Pledge, but it’s pretty safe and sane and is BSD(2-clause) licensed. It can import/export to Git, so one can interact with the git world if one chooses.

              0: https://www.fossil-scm.org

              Regardless, it will be interesting to see what comes of this, and what it means for longer-term OpenBSD sources.. will they move off of CVS someday?

              1. 29

                (Game of Trees author here)

                I haved used Fossil for some projects, mostly OpenBSD driver projects. Fossil is good for small projects and I have successfully used it to overlay local versioning for selected files on top of a CVS checkout, which not many VCS will do as painlessly as Fossil does. It is also the only well-known version control system I’m aware of with a licence that would be accepted into OpenBSD base (BSD-like, ISC preferred).

                But when I attempted a full conversion a few years ago, Fossil simply did not scale to the size of OpenBSD’s source tree. I did try to convert the OpenBSD repo from CVS via git to fossil (which was the officially recommended way of migrating from CVS to Fossil at the time, which is probably still the case today). I aborted Fossil’s import of git’s fast-export stream after several days(!) at which point it was done with history between 1995 and 2000, with at least 10 more years of history to go through. This was the performance I saw after I had already hacked Fossil to batch multiple commits into a single sqlite transaction, instead of using one sqlite transaction per commit (sqlite is not optimized to run many transactions per second, see https://sqlite.org/faq.html#q19).

                Fossil has its own particular design goals for its own particular niche (sqlite development with issue tracking and wiki tooling built-in).

                Got is meant for another niche (OpenBSD developers) and as such it is not meant to be a drop-in replacement for Git. At present it just provides a small but convenient subset of what Git can do on a local repo. It already serves my own OpenBSD development needs better than Fossil ever did.

                1. 3

                  Thanks for the reply!

                  It’s great that you tried Fossil. I haven’t ever tried it on a large repo. Sad that it couldn’t handle it very well. Days to import is definitely a bad sign for what every day use might be like. SQLite has ~ 20 years of history in it now, so I would guess it’s more just the size of the OpenBSD repo that’s the issue here, not really the history. OpenBSD despite being small in size for an OS, is way larger than SQLite.

                  I wish you luck on the project! I might give it a try as a better way to interact with Git.

                  1. 1

                    NetBSD has been doing conversion from CVS to Fossil since 2011, and it supposedly only took under 5 hours in 2011 to convert NetBSD CVS to Fossil, so, it’s not exactly clear why OpenBSD couldn’t be converted in several days.

                    In fact, NetBSD repository is known for having pretty crazy branching allowances, both for release engineering and for lots of user-initiated feature-branches, which are prohibited in OpenBSD, so, if anything, I’d expect the NetBSD repository to be far more complex and nuanced than OpenBSD as far as conversion needs are concerned.


                    https://2011.eurobsdcon.org/papers/sonnenberger/fossilizing.pdf
                    § 4.6 Performance

                    The run time for the conversion process on an Opteron 1389 (Quad-core, 3 GHz) with a RAID-1 of two 7200rpm SATA disks with 4GB memory is as following:

                    • CVS import — 34min
                    • Vendor branches — 4min
                    • Branch creation time — 24s
                    • Fossil import — 3h 56min

                    http://wiki.netbsd.org/mailing-lists/tech-repository/

                    We have a successful conversion from cvs to fossil since ~mid 2011, mostly thanks to the work of Jörg Sonnenberger.

                  2. 6

                    It looks like a more CVS-ified Git, which I think allows the CVS metaphors (which may be more user-friendly depending on your experience) to coexist with a real Git repository. I’m liking what I’m seeing on the man page: https://gameoftrees.org/got.1.html

                    1. 2

                      ooh, I missed the man page. I agree, looks pretty nice.

                      1. 2

                        The got man page seems to be missing an easy way to apply a diff was created with got diff and send to the mailing list. At least I didn’t see anything in the example section about it.

                        Maybe they plan to defer to git am for that?
                        Everything implies this is still a work-in-progress, so maybe it just hasn’t be done yet.

                      2. 2

                        I hope they get the UX right, git sure failed on that.

                        What major gripes do you have with git’s UX?

                        1. 6

                          One could argue: is there any UX? :P

                          But really you generally need to understand all the inner workings of git to successfully use the git command line. Tools should make your life easier. git generally makes your life harder, until you expend the time and energy to really understand all of git. svn, hg, fossil, etc all generally make your life easier. Git was created to make Linus’s life easier, which I’m sure it does very well, but basically nobody has his level of problems when it comes to VCS. Most people would be better served with something semi-idiot proof like mercurial, fossil or svn.

                          There are tons of blogs and pages around that spam endlessly around the horribleness of git’s user experience. This is but one[0]:

                          0: https://stevebennett.me/2012/02/24/10-things-i-hate-about-git/

                          Also I’ll just add, when basically every developer conference has talks about how to survive with git in your life, and these talks have been going on for years, with no end in sight, I think it’s fair to say, git has failed UX.

                        2. 1

                          git sure failed on that

                          Of course it failed there as Linus created Git with bare minimum tooling and hoped that there will be other UI implemented by community. Instead it organically grew to what it is today and bazaar style development isn’t the best for UX. There were projects like Easy Git but these are long gone.

                          1. 3

                            I agree with your history. LOL on bazaar style being terrible on UX(I agree). Even Microsoft has failed at UX sometimes. One could argue that UX hasn’t really changed very much since Xerox PARC days. when the mouse and the “desktop metaphor” was created.

                            1. 2

                              UX on mobile is arguably new.

                            2. 1

                              TortoiseGit on Windows (and other Tortoise stuff for other repositories) is pretty cool.

                          1. 5

                            I’ve been trying to get a patchfix into OpenBSD with no luck. No response to my patch on tech@openbsd.org. This isn’t the first time. Can any OpenBSD contributor help me out?

                            1. 7

                              If you didn’t get any feedback, just keep asking the list for feedback every two weeks by replying to your own post. There’s a bit of luck to it because each patch has to catch someone’s interest in a moment when they have time to deal with it.

                              1. 4

                                Cool I can do that, thanks for the tip.

                              2. 3

                                What stsp said, but also, can you link us to the thread?

                                  1. 2

                                    I just get

                                    I expected an e-mail address, but none was defined.

                                    1. 2

                                      Sorry I’m not entirely sure what the best way is to post a link to a thread on the OpenBSD listserv. If you log in you should be able to see the thread.

                                      EDIT: use this http://openbsd-archive.7691.n7.nabble.com/lib-libfuse-Handle-signals-that-get-sent-to-any-thread-tp352472p353099.html

                                      1. 2

                                        marc.info works pretty well. I’d say it’s the preferred interface for most people.

                                        1. 1

                                          thanks for the pointer

                                1. 2

                                  Not a contributor, but I figure it might help to point out what patch you sent.

                                1. 1

                                  This conference is currently running (October 18 + 19)

                                  Videos of talks are being uploaded as they become available.

                                  There is a live stream as well: http://streaming.media.ccc.de/osmocon18

                                  1. 2

                                    Previously and previously on Lobsters.

                                    1. 1

                                      Oh, sorry for the dupe. I hadn’t seen those. Thanks!

                                    1. 1

                                      Don’t miss this link in the article if you want technical details: https://homebrewserver.club/low-tech-website-howto.html

                                      1. 13

                                        The OS itself in my opinion is not ready for widespread desktop usage…

                                        Would I install it on my granma’s computer? Most likely not, but nor would I GNU/Linux. However, it is just right for my kind of usage (workstation on a Thinkpad Carbon Gen 3).

                                        OpenBSD is by far the most stable and predictable OS I am running (that includes OSX and GNU/Linux) and I am running -current. It does everything that needs being done and does it well.

                                        I agree with OP that one has to like configuring stuff by editing files and reading manpages on the CLI. That being said, configurations are usually pretty terse, man pages well detailed and examples in the man abundant.

                                        OpenBSD is a powertool for powerusers. It’s not being developed for mass market appeal and that’s actually one of its most attractive features.

                                        1. 11

                                          Actually, it is exactly the system I would install on my granma’s computer: A clean OpenBSD desktop with two icons: “Internet” and “Mail”.

                                          She will never get a virus, break it, or fail to fake windows phone scams.

                                          My mother ran a Linux box for many years before jumping to a mac, and she was happy. Everything worked, nothing ever broke. It was predictable. Nowadays Linux is less predictable, especially after an upgrade, but OpenBSD is :)

                                          Edit: However I wouldn’t recommend OpenBSD to a “regular user” friend.

                                          1. 7

                                            Actually, it is exactly the system I would install on my granma’s computer: A clean OpenBSD desktop with two icons: “Internet” and “Mail”.

                                            Geez, what an assumption ;), maybe grandma is a UNIX wizard and uses qutebrowser and mutt and launches them from the terminal.

                                            At any rate, as far as I understand from various posts (haven’t tried OpenBSD since the early 00s), “Internet” would be very slow. Moreover, she would not be able to watch Netflix, since Widevine is not supported on OpenBSD. Oh, and she probably can’t Skype with her grandchildren, etc.

                                            Do you non-tech beloved ones a favor and buy them an iPad. Despite the problems of Apple or Apple hardware, it is the most secure consumer platform, that gets updates for at least half a decade, and probably supports any popular application they’d want (Skype, Netflix, Youtube, Facebook, etc.).

                                            1. 6

                                              An iPad would work well for some people, but for many of my older relatives, they have trouble with the touchscreen input. They can all type reasonably well, since they’re of a generation where Typing was an entire course you took in school, but find touchscreen-typing to be frustrating. As far as something similar but with a kb, not sure whether iPad+bluetooth kb, or just a MacBook would be easier.

                                              1. 4

                                                An iPad would work well for some people, but for many of my older relatives, they have trouble with the touchscreen input. They can all type reasonably well, since they’re of a generation where Typing was an entire course you took in school, but find touchscreen-typing to be frustrating.

                                                That’s interesting and a good point. Though it does not apply to everyone. My mother is in her sixties and never used a computer until 5 years ago (well, except for a domain specific-terminal application when she worked in a library in the 90ies). Despite doing some courses, etc. she always found computers too complex. However, since my dad bought an iPad for her ~5 years ago she has been using it very actively. She is able to do everything she wants - iMessaging, sending e-mail, and browse the web. Later, she also started using a smartphone, since ‘it is just a small iPad’.

                                                At any rate, iPad + KB vs. MacBook would strongly depend on the person and how much they want beyond a simple media consumption device. Of course, if someone is going to compose documents on a device all day, an iPad is a bad option.

                                                Of course, when it comes to typing you don’t want to buy a MacBook 12”/Pro now either ;). The butterfly keyboard is terribly unreliable (my 2016 MBP’s keys are sticky all the time).

                                                1. 1

                                                  Sounds like my grandmother. She does almost everything through a web browser. I had her use Ubuntu briefly. She had no trouble with using it but just prefered the look and feel of Windows. So she went back. I still get malware calls on occasion.

                                                2. 2

                                                  On phones, touch typing sucks for me cuz I have shaky fingers. Miss the keys and have to backspace a lot. Happens less on tablet with big keys. Doesnt happen with a physical keyboard regardless of size. I think it’s the extra, tactile feedback my brain gets from raised keys.

                                                  1. 1

                                                    I use an iPad (with a bluetooth keyboard) while on vacations as a substitute laptop. And with an SSH client program I can even do development on a remote server [1].

                                                    [1] I may not like it that much, as the bluetooth keyboard I use is hard for me to use [2]. But I can do it.

                                                    [2] Even the keyboards on Mac laptops suck. I generally only use IBM Model M keyboards, but taking one on vacation is a bit overkill I think.

                                                  2. 2

                                                    Geez, what an assumption ;), maybe grandma is a UNIX wizard and uses qutebrowser and mutt and launches them from the terminal.

                                                    Sounds like OpenBSD would work even better for your grandmother than we first thought!

                                                    “Internet” would be very slow.

                                                    Why would that be?

                                                    Do you non-tech beloved ones a favor and buy them an iPad. Despite the problems of Apple or Apple hardware, it is the most secure consumer platform, that gets updates for at least half a decade, and probably supports any popular application they’d want (Skype, Netflix, Youtube, Facebook, etc.).

                                                    Sorry but no way would I ever subject anyone I know to using an iPad. Not only is their hardware crap (overheating the moment you do anything with it), and not only is their software locked-down-to-the-point-of-unusably crap, but tablets in general are absolutely pointless devices that have no reason to exist in the home. Tablets are great if you’re an engineer that needs to have a lightweight device with a good bright screen that they can use to look at plans on site. For my mother? Why wouldn’t she just use a laptop?

                                                    Want to make a spreadsheet of your expenses? Nope, sorry, tablet spreadsheet software is garbage. Hope you like having a keyboard pop up over whatever you’re doing every time you want to input anything. Hope you like being unable to copy a row in a single drag of the mouse like you can on desktop, instead having to apparently click, copy, and manually paste into each cell. etc. They’re just bad devices for doing anything productive with a computer, and contrary to popular belief most people want to sometimes do something productive with their computer, whether it’s making a spreadsheet of their expenses, writing a letter to the editor of their paper, making a newsletter for their knitting association, or whatever. Sure they also want to watch Netflix, but that doesn’t mean that all they want to do is watch Netflix.

                                                    1. 2

                                                      Why would that be?

                                                      https://www.tedunangst.com/flak/post/firefox-vs-rthreads

                                                      but tablets in general are absolutely pointless devices that have no reason to exist in the home. Tablets are great if you’re an engineer that needs to have a lightweight device with a good bright screen that they can use to look at plans on site. For my mother? Why wouldn’t she just use a laptop?

                                                      Both my parents and wife are completely happy iPad users. Outside work, my wife usually uses her tablet, despite having a laptop. They are safe, fast and effortless (require virtually no tech support). Interestingly, I as an engineer don’t need or want one. I had an iPad and Nexi on several occasions, but would never use them.

                                                      YMMV

                                                    2. 1

                                                      ipads serve ads and manipulate you. allowing a manipulator access to a loved one doesn’t sound like a favor, not for the loved one at least.

                                                      1. 5

                                                        You will have to expand on that statement. The iPad I’m using to type this does not serve any ads outside apps. Nor do I feel manipulated.

                                                        1. 0

                                                          ads inside apps are still ads, as are push notifications from apps. and of course ios/app developers aren’t trying to make you feel manipulated.

                                                        2. 3

                                                          What ads? Paid apps typically don’t show ads. Besides that Safari on iOS has a content blocking API. Install e.g. Firefox Focus, which is a Safari ad blocker (besides a privacy-focus browser), and websites in Safari are ad-free.

                                                          I have an iDevice (iPhone) and I never see an ad.

                                                          1. 1

                                                            youtube and facebook both show ads, and many facebook stories are ads even if they don’t look it. you can circumvent that on an ipad? could your grandma?

                                                            1. 3

                                                              What exactly does that have to do with the iPad? Facebook and Youtube are hardly specific to the iPad. Circumvention being ad-blocking? Won’t block facebook stories that are ads.

                                                              1. 1

                                                                the ipad has facebook and youtube apps, as /u/iswrong pointed out.

                                                              2. 1

                                                                Well, the comparison here is unfair. In OpenBSD they wouldn’t even have a Facebook or Youtube app. If they’d use the browser to access Facebook/Youtube in OpenBSD, there would be no difference, since Safari can also do ad blocking. Plus they would get hardware-accelerated video ;).

                                                                1. 1

                                                                  right, BSD and Linux don’t have apps, so their utility isn’t tied to apps which show ads and manipulate you. OpenBSD has alternatives to facebook and youtube which don’t have these problems.

                                                        3. 2

                                                          Feels like a Chromebook would have a lot of the same advantages?

                                                          1. 2

                                                            What do you mean by “predictable” here? In my experience most major Linux distributions care far more about backwards compatibility between releases than OpenBSD does.

                                                            1. 1

                                                              Might pretend on the distro. Ubuntu is annoying about changes that break stuff or needlessly force me to learn new way to do old thing.

                                                          2. 9

                                                            OpenBSD feels to me similar to how Linux felt 10 years ago: precisely aimed at me. Now it feels like the ‘powers that be’ in the Linux community are only interested in targeting mobile devices and turning GNOME into macOS’s awful UI design of not letting you do anything that they didn’t think of beforehand.

                                                            1. 4

                                                              Why not run Gentoo or NixOS? Both give you as many configuration options as you require and neither sacrifice any speed? If you are security conscious I believe Gentoo still runs the “hardened” sources.

                                                              1. 2

                                                                My concerns have nothing to do with security or configuration. I currently run Gentoo.

                                                          1. 2

                                                            Is there a video of the talk up anywhere?

                                                            1. 1

                                                              There will be no videos from EuroBSDcon this year, sadly.

                                                            1. 1

                                                              Author says a common class of gadgets uses such and such registers. Says avoid them in favor of other registers. Maybe the gadget type with those registers is common because the registers themselves are common from compiler choices. Switching registers might lead to gadgets just using those registers instead. Or are there x86-specific reasons that using different registers will do entirely different things you can’t gadget?

                                                              Other than that confusion, slides look like great work. Especially on ARM.

                                                              1. 15

                                                                Author here. Thanks for having a look! It was fun to do this talk.

                                                                Yes, there are X86 specific reasons that other registers don’t result in ROP gadgets. If you look at Table 2-2 in the Intel 64 and IA-32 Architectures Software Developer’s Manual you can see all of the ModR/M bytes for each register source / dest pair, and other places in that section describe how to encode the ModR/M bytes for various instructions using all of the possible registers. When I surveyed the gadgets in the kernel and identified which intended instructions resulted in C3 bytes that were used as returns in gadgets, there were a large number of gadgets that were terminating on the ModR/M byte encoding the BX series registers. You are correct that these gadgets are common because the compiler frequently chooses to use the BX series registers, and the essence of my change to clang is to encourage the compiler to choose something else. By shifting RBX down behind R14, R15, R12 and R13 the compiler will choose these registers before RBX, and therefore reduce the incidence of the use of RBX resulting in a C3 ModR/M byte. We can see that this works because just shifting the BX registers down the list results in fewer unique gadgets.

                                                                To directly answer your inquiry, gadgets arising from using R14, R15, R12, R13 instead (now that they will be more common) are not a problem. The REX prefix is never C3, and we can look at the ModR/M bytes encoding operations using those registers, and none of them will encode to C3. When I look at gadgets that arise from instructions using these registers, they don’t get their C3 bytes from the instruction encoding - they get them from constants where the constant encodes to a C3, so the register used is irrelevant in these cases. So moving RBX down behind R14, R15, R12 and R13 doesn’t result in more gadgets using those registers.

                                                                There are other register pairs that result in a C3 ModR/M byte. Operations between RAX and R11 can result in a C3 ModR/M byte, but these are less common when we survey gadgets in the kernel (~56 in the kernel I have here now). RAX and R11 were already ahead of RBX in the default list anyway, so moving RBX down the list does not result in more gadgets using R11. If you ask why we haven’t moved R11 down next to RBX, the answer is that gadgets using R11 this way are not that numerous, so it hasn’t risen to the top of the heap of most-common-sources-of-gadgets (and therefore has not got my attention). There are many other sources of gadgets that can be fixed and will have a larger impact on overall gadget counts and diversity.

                                                                I hope this clarifies that part of the talk. :-)

                                                                1. 3

                                                                  Thank eveyone for the answers. Thank you in particular for this very-detailed answer that clarifies how x86’s oddities are creating the attack vectors.

                                                                  The reason I wanted to know is that I planned to design around high-end ARM chips instead of x86 where possible because I believed we’d see less ISA-related attacks. Also, certain constructions for secure code might be easier to do on RISC with less performance hit. Your slides seem to support some of that.

                                                                  1. 2

                                                                    To be fair, x86 doesn’t create the attack vectors, but does make any bugs much easier to exploit.

                                                                    ARM doesn’t have nearly the same problem - you can always ROP into a jump to THUMB code on normal ARM instructions, but these entry points are usually more difficult to find than an 0xc3.

                                                                  2. 1

                                                                    I’m curious to learn more about ROP. I’d like to examine adding support for another target to ROPgadget.py. So what designates a gadget? Any sequence of instructions ending in a return? How do attackers compose functionality out of gadgets? By hand, or is there some kind of a ‘compiler’ for them?

                                                                    1. 3

                                                                      You might be interested in the ROP Emporium’s guide. Off the top of my head the only automatic tools I know of are ropper and angrop.

                                                                  3. 5

                                                                    Switching registers might lead to gadgets just using those registers instead. Or are there x86-specific reasons that using different registers will do entirely different things you can’t gadget?

                                                                    If I understand this correctly, it’s because the ebx register causes opcodes to be created that contain a return instruction, i.e., opcodes that are useful in ROP. So by avoiding ebx as much as possible, you also avoid creating collateral ROP gadgets with early returns. This issue only happens because x86/amd64 have variable-length opcodes.

                                                                    1. 4

                                                                      As far as I understand, the register allocation trick is indeed x86-specific. The point is to avoid C3 bytes because these will polymorph into the RET instruction when used in unaligned gadgets. See the “polymorphic gadget” and ‘register selection’ sections in the slide set.

                                                                    1. 1

                                                                      So, is this about using OpenBSD as a development environment for working on MirageOS applications?

                                                                      1. 7

                                                                        Yes. More precisely, it is about running mirageos unikernels in vmm: See https://marc.info/?l=openbsd-tech&m=150743896827764&w=2

                                                                      1. 6

                                                                        Wow, there are a lot of self-hosters here. I self hosted back in University, then used Gmail for a number of years, and went back to self hosting around 2013. I recently migrated my server from openSUSE on Linode to OpenBSD on Vultr. Here an Ansible role if anyone is interested:

                                                                        https://github.com/sumdog/bee2/tree/master/ansible/roles/openbsd-email

                                                                        My stack: Inbound: OpenSMTPD -> SpamPD (spam assassin) -> OpenSMTPD -> ClamAV -> OpenSMTPD -> procmail -> dovecot Outbound: OpenSMTPD -> DKIM proxy -> OpenSMTPD (relay)

                                                                        1. 1
                                                                        1. 9

                                                                          It landed, but I still have some follow up work to get full support for all types of snapshots. The process that does disk I/O starts with the fds preopened, and is chrooted and pledged, which makes opening the base images of the multi disk snapshots hard.

                                                                          1. 2

                                                                            Thanks for working on this.

                                                                            (I’m really hoping that someday I’ll be able to install Debian under vmm(4) from official install media. Currently it doesn’t detect any CD drives, and I’ve not been able to figure out why).

                                                                            1. 1

                                                                              Because Debian install media lack virtio drivers.

                                                                              1. 1

                                                                                I wonder if we could persuade them to include them?

                                                                                I suppose virtualbox and qemu emulate physical CD drives rather than virtio?

                                                                          1. 12

                                                                            Lots of null pointer dereferences, use-after-free, and double free. OpenBSD really needs a language with affine types or smart pointers that integrates with C. ;)

                                                                            1. 6

                                                                              Such a language needs to work on every hardware platform they support and have a BSD licensed compiler/toolchain 🙃

                                                                              1. 5

                                                                                I actually think starting to use C++ in kernel is no-brainer, like GCC did. C++ doesn’t have hardware or toolchain problem, does it?

                                                                                1. 1

                                                                                  Although I’m against C++, it’s clearly an option with more safety features and low-cost abstractions all the time. I”ll note that folks developing L4 microkernels and Genode started using it for those reasons. At this point, I’d rather whatever it is be a safer C with better abstractions that outputs vanilla C. That would solve most of tooling and integration issues that come with language switch. It also dodges C++‘s huge complexity. It’s ridiculously complex.

                                                                                  1. 3

                                                                                    a safer C with better abstractions that outputs vanilla C.

                                                                                    Sounds like Nim to me. MIT license.

                                                                                    1. 3

                                                                                      It’s close! I’m eyeballing it for that use with Brute-Force Assurance. It would have way more acceptance than a Scheme-based solution. I’d have to swap its syntax out since C developers switch to C-like languages more than Python-like languages. The compiler for this purpose should produce C that looks like what a person would write more than a machine. It should at least be an option. Lets it get used incrementally in existing, C projects. Finally, the people I see online griping about the compiler means they need to focus hard on getting it in good shape or someone has to build a separate, certifying compiler.

                                                                                      So, that’s what I was thinking when I assessed Nim as C replacement in general and for safety critical. Oh yeah, contracts! Frama-C or Ada-style contracts supported by default. Lets you encode whatever extra stuff the type system doesn’t already handle. I don’t know if they have contracts.

                                                                                2. 2

                                                                                  Im sure they could build the language or C extensions given they built a whole OS and maintained (still do?) a compiler for it. It would also help them achieve their security goals better than their developers are doing now with C language. A good investment I’d say.

                                                                                  1. 10

                                                                                    This could happen if one or more people with interest and motivation showed up and managed to work well with the project to integrate this with the system as yet another form of mitigation.

                                                                                    As for the existing devs, they are all already very busy scratching their own itches and pursue their own ideas, some related to security, some not. And generally they don’t like to be told what to work on in the time they volunteer.

                                                                                    1. 1

                                                                                      Exactly. The average coder in Rust is currently outperforming the OpenBSD team on these kinds of bugs due to type system. That means these bugs happen since they don’t care enough to prevent them. They’re about QA and mitigation tech up to a certain point with certain bug-adding tech (eg C language). Past that point or with different mitigations (esp language), they start making excuses about time, itches, and so on. I’ll keep pointing this out every time evidence of easily-prevented bugs comes in. Maybe something will click in a reader’s head that leads to a solution.

                                                                                      Many of them also tell other people how they should be doing UNIX design, quality or security. Sometimes even in a snooty way. They like doing that despite aggravation it might cause others. You say those same people don’t like “to be told” they should use more secure tech in a security-focused project. It sounds like there’s a life lesson in there somewhere on top of some security lessons.

                                                                                      1. 5

                                                                                        since they don’t care enough to prevent them

                                                                                        That’s a tad inflammatory nay? Suggesting that not using rust in tantamount to not caring. Its not like the Linux/BSD kernel could be rewritten in rust in a day, there is 20+ years of development in there.

                                                                                        And while its not exactly a fair compairson as its been run against linux for longer, 9 issues (which have been fixed) versus quite a few in linux suggests something in OpenBSD is working.

                                                                                        1. 1

                                                                                          Yeah, a tad inflammatory to match the style of their mailing lists talking about other OS’s or hardware vendors not doing enough for security. I always give them credit for their strong points of simplified UNIX, code review/quality, mitigations, and great documentation. Plus, I like a few of them personally.

                                                                                          Far as your counterpoint, it’s a strawman (full rewrite) that’s not even what Im proposing. I’m saying folks that cared seeing the language cause issues would make a safer version like others did in other projects (eg Clay, Cyclone). One highly-compatible with C. They’d write new code in that language. The extensive rewrites of existing code they already do would be done in that language. Over time (years), most or all the OS would be converted to the safer language. Someone might even write tools to automate this.

                                                                                          1. 5

                                                                                            The idea of a slightly modified C which would somehow prevent use-after-free and similar bugs is good. It’s similar to other ideas OpenBSD has already realized such as adding C API functions which are easier to use safely, or hardening of the C run-time against ROP. And it’s not as if the C we’re writing did not contain non-standard extensions already (packed structs, gcc-isms inherited by clang, etc.)

                                                                                            Now, where are some compiler-writing C langauge lawyer academics with the needed skills who would sit down with a bunch of OpenBSD hackers and volunteer a lot of their spare time for this? In over 10 years of involvement with the project I’ve never met a person with this skill set. In a volunteer project you have to work with the skills you happen to get.

                                                                                            1. 1

                                                                                              Glad you’re open to the possibility if you had help for it. The people behind Clay and Cyclone might have helped given they were already doing hardest parts. It’s possible you didn’t know those languages exist. The folks good at researching and developing languages usually aren’t good at polish, outreach, and so on.

                                                                                              It’s possible we need a sponsor organization or new type of volunteer for such a role. One that’s a middle-person between the team with time to build compilers and the people that would use them. Such a person would need to be able to influence compiler developers to ensure they don’t do anything that kills adoption. I figure there’d be a lot of negotiations with middle person doing tie breakers on stuff people were divided on. Probably also need to be a compiler developer themselves so they can do the polish, packaging, and later maintenance.

                                                                                  2. 2

                                                                                    I realize this is mostly bikeshedding, but does the core team regularly (or ever) consider this? Or is this seen as too much overhead - learning the subtleties of a new language/implementation on top of the difficulty of os/kernel development. I would think the D language folks would love to team up with one of the BSDs to focus on whatever language demands the OS team would come up with.

                                                                                1. 7

                                                                                  I’m not familiar with the size of the OpenBSD kernel, can anyone give a sense of what fraction of the kernel 226 syscalls is?

                                                                                  1. 9

                                                                                    That’s pretty close to every syscall I believe. The last syscall is 330, but there are several large gaps.

                                                                                    http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/kern/syscalls.master?rev=1.188&content-type=text/x-cvsweb-markup

                                                                                    1. 12

                                                                                      syscalls like ioctl() trigger so many code paths that it would make sense to count each ioctl type as a separate syscall for purposes of fuzzing.

                                                                                    2. 1

                                                                                      I second this question. That’s a big number for security-focused OS. Might be reasonable, too, given it’s monolithic kernel with batteries included. I did a quick check on the site. Didn’t see a full list of system calls.

                                                                                    1. 5

                                                                                      Germany has an equivalent called Freifunk that is quite popular.

                                                                                      1. 2

                                                                                        Was it not true that it was illegal to have unsecured wifi in Germany? Or has this been overturned?

                                                                                        1. 8

                                                                                          It was never illegal, but the operator was liable for all network activity. That was recently reversed.

                                                                                          1. 2

                                                                                            The new law instead allows content owners to force individual wifi operators to block certain web sites from being reachable via their open network. The technical ‘how’ is of course left unspecified. It will be interesting to see how this rule gets applied in practice.

                                                                                          2. 3

                                                                                            What tedu says is correct. The liability problem iis gone.

                                                                                            The official Freifunk firmware dodged that by routing outbound traffic through a VPN though.

                                                                                        1. 1

                                                                                          Although this response is reasonable up to and including the first point, the second point is a little less convincing. The idea that it’s okay to be bad at security simply because someone else was bad at security is unfortunate at best.

                                                                                          The first point is - although not completly wrong - definitely debatable since these connections can be made simultaneously and aren’t blocking each other, which seems like they are trying to insinuate here.

                                                                                          1. 10

                                                                                            I think there’s a distinction to be made between “bad at security” and “not actually a security boundary”. If you retroactively redefine public info to be a secret, it shouldn’t be surprising that everyone is “bad” at protecting it, or that someone might pushback and say not a bug.

                                                                                            1. 1

                                                                                              And how and why is a username considered public information, they asked? https://lobste.rs/u

                                                                                              1. 1

                                                                                                If you assume every user on your system is public info then you are making privacy decisions on behalf of and at the expense of others, and the alternate solution of treating them as private information doesn’t harm anyone. What are the concerns with the user information being private as opposed to public?

                                                                                                If there isn’t a reason for it to be public, then I’d say that it’s absolutely bad in this case. Both from a security and from a general user perspective.

                                                                                              2. 3

                                                                                                They mentioned the MaxStartups parameter, which does seem like it will cause connections to block.

                                                                                                1. 1

                                                                                                  Ah! Legit :) Hopefully that’s a low number by default

                                                                                              1. 4

                                                                                                This is really interesting to get an idea of how people are taking advantage of BSD! I now have a much nicer idea of why people are going to it (and am a bit tempted myself). That feeling of having to go through ports and simply not having 1st-class support for some software seems… rough for desktop usage though

                                                                                                1. 3
                                                                                                  1. 1

                                                                                                    I mean “someone talks to me about an application and I’m interested in trying it out on my system”?

                                                                                                    I feel like the link to the CVE database is a bit of an unwarranted snipe here. I’m not talking too much about security updates, just “someone released some software and didn’t bother to confirm BSD support so now I’m going to need to figure out which ways this software will not work”.

                                                                                                    To be honest I don’t really think that having all userland software come in via OS-maintained package managers is a great idea in the first place (do I really need OS maintainers looking after anki?). I’m fine downloading binaries off the net. Just nicer if they have out of the box support for stuff. I’m not blaming the BSDs for this (it’s more the software writer’s fault), just that it’s my impression that this becomes a bit of an issue if you try out a lot of less used software.

                                                                                                    1. 4

                                                                                                      As an engineer that uses and works on a minority share operating system, I don’t really think it’s reasonable to expect chiefly volunteer projects to ship binaries for my platform in a way that fits well with the OS itself. It would be great if they were willing to test on our platform, even just occasionally, but I understand why they don’t.

                                                                                                      Given this, it seems more likely to expect a good experience from binaries provided by somebody with a vested interest in quality on the OS in question – which is why we end up with a distribution model.

                                                                                                      1. 2

                                                                                                        Yep, this makes a lot of sense.

                                                                                                        I’m getting more and more partial to software relying on their host language’s package manager recently. It’s pretty nice for a Python binary to basically always work so long as you got pip running properly on your system, plus you get all the nice advantages of virtual environments and the like letting you more easily set things up. The biggest issue being around some trust issues in those ecosystems.

                                                                                                        Considering a lot of communities (not just OSes) are getting more and more involved in distribution questions, we might be getting closer to getting things to work out of the box for non-tricky cases.

                                                                                                        1. 8

                                                                                                          software relying on their host language’s package manager

                                                                                                          In general I’m not a fan. They all have problems. Many (most?) of them lack a notion of disconnected operation when they cannot reach their central Internet-connected registry. There is often no complete tracking of all files installed, which makes it difficult to completely remove a package later. Some of the language runtimes make it difficult to use packages installed in non-default directory trees, which is one way you might have hoped to work around the difficulty of subsequent removal. These systems also generally conflate the build machine with the target machine (i.e., the host on which the software will run) which tends to mean you’re not just installing a binary package but needing to build the software in-situ every time you install it.

                                                                                                          In practice, I do end up using these tools because there is often no alternative – but they do not bring me joy.

                                                                                                          Operating system package managers (dpkg/apt, rpm/yum, pkg_add/pkgin, IPS, etc) also have their problems. In contrast, though, these package managers tend to at least have some tools to manage the set of files that were installed for a particular package and to remove (or even just verify) them later. They also generally offer some first class way to install a set of a packages from archive files obtained via means other than direct access to a central repository.

                                                                                                          1. 3

                                                                                                            For development I use the “central Internet-connected registry.”, for production I use DEB/RPM packages in a repository:

                                                                                                            • forces you to limit the number of dependencies you use, otherwise too much work to package them all;
                                                                                                            • force you to choose high quality dependencies that are easy to package or already packaged;
                                                                                                            • makes sure every dependency is buildable from source (depending on language);
                                                                                                            • have an “offline” copy of the dependencies, protect against “left-pad” issues;
                                                                                                            • run unit tests of the dependencies during package build, great for QA!;
                                                                                                            • have (PGP) signed packages that uses the distribution’s tools to verify.

                                                                                                            There are probably more benefits that escape me at the moment :)

                                                                                                  2. 1

                                                                                                    That feeling of having to go through ports and simply not having 1st-class support for some software seems… rough for desktop usage though

                                                                                                    What kind of desktop software do you install from these non-OS sources?

                                                                                                    1. 2

                                                                                                      Linux is moving more and more towards Flatpak and Snap for (sandboxed) application distribution.

                                                                                                      1. 2

                                                                                                        I remember screwing around with Flathub on the command line in Fedora 27, but right now on Fedora 28, if you enable Flatpak in the Gnome Software Center thingy, it’s actually pretty seamless - type “Signal” in the application browser, and a Flatpak install link shows up.

                                                                                                        With this sort of UX improvements, I’m optimistic. I feel like Fedora is just going to get easier and easier to use.

                                                                                                  1. 11

                                                                                                    Note that SMT doesn’t necessarily have a posive effect on performance; it highly depends on the workload. In all likelyhood it will actually slow down most workloads if you have a CPU with more than two cores.

                                                                                                    In case you’re wondering, this refers to OpenBSD’s giant-locked kernel. Some parts of this kernel are now unlocked (e.g. network stack) but for some workloads 2 CPUs can be faster than 3 or more due to lock contention.

                                                                                                    1. 1

                                                                                                      Per my understanding, every “physical” CPU can have many cores, and each core can have multiple hardware thread if SMT is supported. So every “hardware thread” is a “logical” CPU. For OpenBSD kernel, does it do special operations according to physical CPU, core and hardware thread? Or just consider “logic” CPU? Thanks!

                                                                                                      1. 2

                                                                                                        As far as I know the SMT threads were simply exposed as additional CPUs to the scheduler.

                                                                                                        1. 1

                                                                                                          @stsp Thanks for your response!

                                                                                                          If I understand correctly, disable SMT means cut half the “logical” CPU, right? For example, if the server has one CPU, 2 cores, and every core has 2 hardware threads, in theory, the server has 4 “logical” CPUs. Assume my workload has 4 thread, and every thread is independent and computing-intensive (mostly user-space computation, not involved kernel part, such as syscall, or accessing network, etc.). Currently the workload can occupy the whole 4 “logical” CPUs. But now, if the count of “logical” CPU is halved, and my workload’s 4 thread need to contend for 2 “logical” CPUs. So in this scenario, the workload’s performance should be downgraded.

                                                                                                          Is it correct? Thanks in advance!

                                                                                                          1. 3

                                                                                                            At least when HT was new, it also meant the caches would be halved unless you disabled HT in bios. So if your threads are doing different things they might suffer from it.

                                                                                                            1. 1

                                                                                                              As far as I understand, it doesn’t mean that all 4 threads can progress in parallel, it will depend on which unit in the CPU each thread is utilizing.

                                                                                                      1. 5

                                                                                                        I can’t follow this at all the way it’s presented.

                                                                                                        1. 4

                                                                                                          No worries, you are not the only one who is having trouble following it.

                                                                                                          This is not an editorialized piece of writing trying to guide you towards a particular point of view. It just shows unredacted facts. The intent is to allow anyone to be a bystander in the discussion that actually occurred and make up their own minds about related questions if they have an interest in doing so. And it is only happening in public because interpretations of what happened contradicting the facts were circulated in public (most recently at BSDcan).

                                                                                                          There are no easy answers to the questions raised by the full- vs coordinated-disclosure debate in general. If you are involved in the disclosure process of a security problem and fix, whatever you do, one way or another someone else might potentially be put at risk as a consequence of your actions. And not every risk assessment will lead to the same conclusions.

                                                                                                          1. 1

                                                                                                            Near as I can figure, there was a bunch of back-channel communications about the issue in the OpenBSD community until the guy who found the issue contacted CERT because he figured out the issue went way beyond OBSD. The OpenBSD folks apparently don’t trust CERT and decided to push a fix to protect OBSD users possibly at the expense of, well, everyone else because…I don’t know…screw them, I guess.

                                                                                                            You put us in a conundrum. We knew there was a problem and how to fix it. And when you got CERT involved, we had to assume that information about the problem was now leaking beyond your control into government agencies and private companies, and that some of those “in the know” would have had 2 months of extended embargo time to use an exploit against OpenBSD users. I don’t see any reason to trust every single person in those parts of the security community and in these institutions to act responsibly.