1. 5

    One thing that I feel is missing from the introduction: Virtual machines are also excellent for obfuscation purposes. They allow you to encrypt the actual code, the reverse enginering process gets slowed down a lot if you use a particularly obscure instruction set for which no tooling exists (or even worse: invent your own instruction set), they move code out of the executable region into the data region and can help you execute only signed VM code from a remote location after passing authentication checks.

    1. 1

      this sounds really interesting! do you have any examples of this, or research?

      1.  

        Open source examples used in the real world don’t exist because that defeats the entire purpose of the obfuscation.

    1. 6

      Object-Oriented Programming: An Evolutionary Approach

      I actually went on one of my “deep dive into old computer science things” and got obsessed with pre-NeXT Objective-C. I tracked down a copy of the first edition of this book (the second edition is much closer to the more modern Objective-C that we now know and love).

      I even reached out to Tom Love (co-creator of Objective-C along with Brad Cox) and he was kind enough to recommend Object Lessons as an additional suggestion and dig through his garage for some old documents.

      Either way, it’s an excellent book.

      Object-Oriented Software Construction

      Meyer’s approach to software engineering is…I’m not even sure of the right word. “Perfectionist” might be close, but I don’t want the negative connotation to come through on that. Anyone who wants to study OOP could do with reading his work.

      1. 2

        and got obsessed with pre-NeXT Objective-C

        Do you have any resources for that in particular? I’ve been curious about Objective-C before NeXT, but never ended up diving into it and its history.

        1. 5

          The above mentioned Object-Oriented Programming: An Evolutionary Approach is good, of course.

          I read a lot of NeXT documentation, though again I was more interested in the pre-NeXT days.(I briefly had a NeXTstation set up in my living room. That was fun.)

          I bought a copy of “Objective-C: Object-Oriented Programming Techniques” by Pinson for like fifteen cents from Amazon; that was all right but not great.

          Most interesting was the original “Object-Oriented Pre-Compiler” paper, which I believe was published in Communications of the ACM but I’m not exactly sure where I got it. It documented a very early implementation where methods were invoked using a rather…awkward…syntax.

          I found references here and there to the various “ICpaks” that PPI (later Stepstone) released (ICpak101 was the core collection classes and ICpak102 was the GUI, IIRC). These were very different from the later NeXTstep/OPENSTEP classes, and really nice in their own ways. They were somewhat documented in the Evolutionary Approach book as well.

          Sadly, I was never able to get the Holy Grail that I was looking for: copies of the original PPI compiler/ICpak/library manuals. Those whom I reached out to (Brad Cox, Tom Love, and others) were unable to find their copies or were unwilling to part with them (which is understandable).

          If you’re interested, the Portable Object Compiler implements a pre-NeXT (but still post-ancient) Objective-C, and its manual describes its “ObjectPak”, which is more in line with the original “ICpaks” than NeXTstep. I still much prefer Objective-C to C++.

          1. 2

            The Object-Oriented Pre-Compiler: programming Smalltalk-80 methods in the C language is the citation (and if you’re an ACM member, the full article is linked there).

            1. 1

              You weren’t kidding about if an ACM member: couldn’t find a legal copy anywhere other than paywalls. ResearchGate’s at least has “request full text” button. Did at least stumble on an interesting, historical submission for Tuesday.

              1. 3

                The Object-Oriented Pre-Compiler: programming Smalltalk-80 methods in the C language

                http://sci-hub.tw/10.1145/948093.948095

                1. 2

                  I did say “legal.” ;)

                  1. 3

                    For all intents and purposes that article should be freely available by now. That it isn’t is just a bug in the system, a blip on the line, a hiccup in the clockwork and as such something the ’net has been designed to route around. Which it does.

            2. 2

              Thank you for the detalied response!

        1. 2

          Was encrypted storage of the secrets intentionally foregone?

          1. 3

            Then pass-otp might be better… at least the secrets are stored gpg-encrypted then :)

            1. 1

              The secrets will be saved as a hidden file named .mina.json in the home directory of the current user.

              1. 1

                This doesn’t offer any protection against other users on the same machine. Encrypting the secrets is the way to go, but in the meantime you should do

                import os
                import stat
                os.chmod(JSON_URL, stat.S_IRUSR | stat.S_IWUSR)
                

                to prevent other users from being able to view the file.

                1. 5

                  That’s susceptible to race conditions. You have to do a little umask dance before creating the file.

            1. 41

              Wow, that’s pretty terrible.

              On the other hand, I can’t help but to feel sorry about Dominic, we all make mistakes, this public shaming is pretty violent.

              I guess we should sometimes take some time off to read the license before using a library:

              THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

              (F)OSS is not a consumer good.

              1. 11

                I agree that shaming people is toxic and unproductive. No one wants to be shamed and no one is perfect.

                But I see another dimension to the negative responses Dominic has received. Non-hierarchical, self-governing communities like open source software are organized by social norms. Social norms work through peer pressure - community members conform to the norms of the community not because they are compelled to by law but because it would cost them standing in the community not to. This isn’t inherently good. Some norms are toxic and self-policing via peer pressure can lead to shaming. What I see in some of the critical comments addressed to Dominic is an attempt to establish a clear social norm about what to do when you are ready to abandon a package. The norm is desirable because it increases the general level of trust. Even if the landscape is generally untrustworthy, you can have some confidence that people aren’t handing their packages off to strangers because it’s the norm not to do that. The desire for some norm here, whatever it is in the end, is reasonable.

                Ending the discussion with “don’t worry about it Dominic, everyone makes mistakes, and anyways you’re not liable for it” signals to everyone that they’re not responsible for the consequences of what they do. In a strictly legal sense, that might be true. Even then, I’m skeptical that the warranty clause would cover negligence in the distribution of the software rather than the software itself. But in either case, don’t we want a community where people do feel responsible for the actions they take and are open to receiving feedback when an action they’ve taken has a bad result? This dialogue can occur without shaming, without targeting anyone personally, and can be part of the same give-and-take process that produces the software itself.

                1. 6

                  Blaming people in any security issue is toxic, no matter what happens. In any organization with paid people where you should expect better, the most important rule of a post-mortem is to remain blameless. It doesn’t get anyone anywhere and doesn’t get remotely close to actual root cause. Instead of asking about why Dominic gave away a critical package, people should be asking why some random maintainer were able to give away a critical package.

                  Ending the discussion with “don’t worry about it Dominic, everyone makes mistakes, and anyways you’re not liable for it” signals to everyone that they’re not responsible for the consequences of what they do.

                  By putting blame on Dominic, people are not taking responsibilities. The main issue is that many core libraries in the JavaScript ecosystems still depends on external, single-file, non-core, likely unmaintained library. People who should take responsabilities are the ones who chose to add a weak single point of failure by depending on event-stream.

                  1. 2

                    It depends what you mean by blame. If you mean assigning moral responsibility, especially as a pretext for shaming them, then I agree it’s toxic. I think I was clear that I agree this shouldn’t happen. But if blame means asserting a causal relationship between Dominic’s actions and this result, it’s hard to argue that there isn’t such a relationship. The attack was only possible because Dominic transferred the package. This doesn’t mean he’s a bad person or that he should be “in trouble” or that anything negative should happen to him as a consequence. A healthy social norm would be to avoid transferring packages to un-credentialed strangers when you’re ready to abandon the package because we’ve seen this opens an attack vector. Then what’s happened here is instructive and everyone benefits from the experience. And yes, ideally these dilemmas are prohibited by the system. Until that is the case, it helps to have norms around the best way to act.

                    1. 1

                      I understand you don’t condone the attacks and shaming going around. However I would argue that even if you agree that the blaming is toxic, that building some social norm around it is better than nothing, I believe that even hinting that it was somehow Dominic’s fault is a net negative.

                      The attack was only possible because Dominic transferred the package.

                      This is exactly what I’m condoning. By looking at individual and their action you scope the issue at that level. The attack was taking over a dependancy. It is possible to do so in so many way, especially for packages such as Dominic’s. This time it was a case of social engineering, next time it might as well be a case of credential hijacking, phishing or maintainer going rogue.

                      A healthy social norm would be to avoid transferring packages to un-credentialed strangers when you’re ready to abandon the package because we’ve seen this opens an attack vector.

                      I would say pushing this rethoric is actually unhealty and only lead people to rely on those social norm and use it as an excuse to disown their accountability. It would be much healthier to set expectation right and learn proper risk assessment about dependancies management.

                      Then what’s happened here is instructive and everyone benefits from the experience. And yes, ideally these dilemmas are prohibited by the system. Until that is the case, it helps to have norms around the best way to act.

                      The same issue have come up so many time in the past few years, especially in the NPM ecosystem, it should be well past the “learn from the experience” and I believe it’s time the relevant actors actually move toward a solution.

                2. 17

                  I’ve done a similar thing before. After leaving the Elm community, I offered to transfer most of my repos over to the elm-community organisation. They accepted the most popular ones, but not elm-ast (and maybe one or two others). A few months later I received an e-mail from @wende asking if he could take over so I took a look at his profile and stuff he’s done in the past and happily gave him commit access thinking users would continue getting updates and improvements without any hassle. Now, @wende turns out to be a great guy and I’m pretty sure he hasn’t backdoored anyone using elm-ast, but I find it hilarious that people somehow think that maintainers should be responsible for vetting who they hand over control of their projects to or that they could even do a good job of it OR that it would even make any sort of a difference. Instead of trusting one random dude on the internet (me) you’re now trusting another.

                  Don’t implicitly trust random people on the internet and run their code. Vet the code you run and keep your dependency tree small.

                  1. 25

                    Vet the code you run

                    Or trust well-known, security-oriented distributions.

                    keep your dependency tree small

                    Yes, and stay away from environment, frameworks, languages that force dependency fragmentation on you.

                    1. 4

                      Or trust well-known, security-oriented distributions.

                      That too! :D

                      1. 3

                        and stay away from […] frameworks

                        I wouldn’t say that as absolutely for the web. I suspect that things would likely go a lot more haywire if people started handling raw HTTP in Python or Ruby or what have you. There’s a lot of stuff going on under the hood such as content security policies, CSRF protection and the like. If you’re not actively, consciously aware of all of that, a web framework will probably still end up providing a net security benefit.

                        1. 5

                          Please don’t quote words without context:

                          […] that force dependency fragmentation on you

                          Frameworks and libraries with few dependencies and a good security track record are not the problem. (If anything, they are beneficial)

                          1. 2

                            I interpreted “Yes, and stay away from environment, frameworks, languages that force dependency fragmentation on you.” as (my misunderstandings in brackets) “Yes, and stay away from [(a) integrated development] environments, [(b)] frameworks, [(c)] languages that force dependency fragmentation on you.” with a and b being separate from the “that” in c.

                            I apologize for the misunderstanding caused.

                        2. 2

                          Isn’t it the case that reputable, security-focused distributions acquire such status and the continuity thereof by performing extensive vetting of maintainers?

                          The responsible alternative being abandoning the project and letting the community fork it if they want to.

                          1. 1

                            Or trust well-known, security-oriented distributions.

                            Then how do You deal with things like that: “The reason the login form is delivered as web content is to increase development speed and agility” ?

                            1. 2

                              As a distribution? Open a bug upstream, offer a patch, and sometimes patch the packaged version.

                              1. 1

                                That’s a good idea in general but sometimes the bug is introduced downstream.

                        3. 9

                          Most proprietary software also comes with pretty much the same warranty disclaimer. For example, see section 7c of the macOS EULA:

                          https://images.apple.com/legal/sla/docs/macosx107.pdf

                          I mean, have we held accountable Apple or Google or Microsoft or Facebook in any substantial ways for their security flaws?

                          1. 4

                            In many other products accountability is enforced by law and it overrides any EULA. And that is tied to profit in the broad sense: sales or having access to valuable customer data & so on.

                            Software companies got away with zero responsibility and this only encourages bad software.

                            1. 1

                              And how have we enforced that by law for those companies, regardless of what those EULAs have said? When macOS allowed anyone to log in as root, what were the legal consequences it faced?

                              1. 3

                                other products

                                e.g. selling cars without safety belts, electrical appliances without grounding…

                          2. 2

                            It is a security disaster given how easy it is for js stuff to hijack cookies and sessions.

                            1. 1

                              It really isn’t if a well thought out CORS policy is defined.

                          1. 21

                            To start, the ZFS filesystem combines the typical filesystem with a volume manager. It includes protection against corruption, snapshots and copy-on-write clones, as well as volume manager.

                            It continues to baffle me how “mainstream” filesystems like ext4 forgo checksumming of the data they contain. You’d think that combatting bitrot would be a priority for a filesystem.

                            Ever wondered where did vi come from? The TCP/IP stack? Your beloved macOS from Apple? All this is coming from the FreeBSD project.

                            Technically, vi and the BSD implementations of the TCP/IP stack can be attributed to 4.xBSD at UCB; FreeBSD is not the origin of either.

                            1. 10

                              It continues to baffle me how “mainstream” filesystems like ext4 forgo checksumming of the data they contain. You’d think that combatting bitrot would be a priority for a filesystem.

                              At least ext4 supports metadata checksums:

                              https://wiki.archlinux.org/index.php/ext4#Enabling_metadata_checksums

                              At any rate Ted T’so (the ext[34] maintainer) has said as far back as 2009 that ext4 was meant to be transitional technology:

                              Despite the fact that Ext4 adds a number of compelling features to the filesystem, T’so doesn’t see it as a major step forward. He dismisses it as a rehash of outdated “1970s technology” and describes it as a conservative short-term solution. He believes that the way forward is Oracle’s open source Btrfs filesystem, which is designed to deliver significant improvements in scalability, reliability, and ease of management.

                              https://arstechnica.com/information-technology/2009/04/linux-collaboration-summit-the-kernel-panel/

                              Of course, the real failing here is not ext4, but that btrfs hasn’t been able to move to production use in more than ten years (at least according to some people).

                              That said, ZFS works fine on Linux as well and some distributions (e.g. NixOS) support ZFS on root out-of-the-box.

                              1. 3

                                Of course, the real failing here is not ext4, but that btrfs hasn’t been able to move to production use in more than ten years (at least according to some people).

                                I think it’s good to contrast “some people’s” opinion with the one from Facebook:

                                it’s safe to say every request you make to Facebook.com is processed by 1 or more machines with a btrfs filesystem.

                                Facebook’s open-source site:

                                Btrfs has played a role in increasing efficiency and resource utilization in Facebook’s data centers in a number of different applications. Recently, Btrfs helped eliminate priority inversions caused by the journaling behavior of the previous filesystem, when used for I/O control with cgroup2 (described below). Btrfs is the only filesystem implementation that currently works with resource isolation, and it’s now deployed on millions of servers, driving significant efficiency gains.

                                But Facebook employs btrfs project lead.

                                There is also the fact that Google is now using BTRFS on Chromebooks with Crostini.

                                As for opinions I’ve seen one that claims that “ZFS is more mature than btrfs ON SOLARIS. It is mostly ok on FreeBSD (with various caveats) and I wouldn’t recommend it on Linux.”.

                                1. 2

                                  I wouldn’t recommend it on Linux.

                                  I’d still say that ZFS is more usable than lvm & linux-softraid. If only due to the more sane administration tooling :)

                              2. 9

                                Ext4, like most evolutions of existing filesystems, is strongly constrained by what the structure of on-disk data and the existing code allows it to do. Generally there is no space for on-disk checksums, especially for data; sometimes you can smuggle some metadata checksums into unused fields in things like inodes. Filesystems designed from the ground up for checksums build space for checksums into their on-disk data structures and also design their code’s data processing pipelines so there are natural central places to calculate and check checksums. The existing structure of the code matters too because when you’re evolving a filesystem, the last thing you want to do is to totally rewrite and restructure that existing battle-tested code with decade(s) of experience embedded into it; if you’re going to do that, you might as well start from scratch with an entirely new filesystem.

                                In short: that ext4 doesn’t have checksums isn’t surprising; it’s a natural result of ext4 being a backwards compatible evolution of ext3, which was an evolution of ext2, and so on.

                                1. 4

                                  It continues to baffle me how “mainstream” filesystems like ext4 forgo checksumming of the data they contain. You’d think that combatting bitrot would be a priority for a filesystem.

                                  Ext4 doesn’t aim to be that type of filesystem, for desktop use on the average user, this is fairly okay since actual bitrot in data the user cares about is rare (most bitrot occurs either in system files or empty space or in media files where the single corrupt frame barely matters).

                                  If you want to check out a more modern alternative, there is bcachefs. I’ve been using it on my laptop for a while (until I stopped but now I’m back on it) and it’s been basically rock solid. The developer is also working on erasure coding and replication in a more solid way than btrfs currently has.

                                1. 6

                                  With https it would be even nicer.

                                  1. 2

                                    I’d been hoping that Chrome showing warnings for non-HTTPS pages would make them do it. txti.es has been around for at least two years now, but it seems HTTPS definitely isn’t coming at this rate.

                                    1. 0

                                      https can really hurt availability in countries with extremely slow internet. It is a solid win for people in like the US, but questionable in some other situations - a cost for no benefit.

                                      1. 2

                                        Cant you do both?

                                    1. 8

                                      I agree with this, and I believe the door should’ve been slammed shut before I was invited. The past three or four months have definitely had more “HN-flavored” stories, or so it feels at least. There’s no way to keep a community of over 10000 users on track and focussed on the original concept.

                                      1. 7

                                        Could you post some examples of ‘HN flavoured stories’ ?

                                        It is hard to pinpoint what it is about Lobsters that makes me prefer it to other similar communities, but I would summarise this place as:

                                        Stories for those who enjoy the details
                                        

                                        I had to omit the word ‘technical’, I don’t think it is critical to describe what goes on around here. I would much rather read some well informed and passionate write up on a non-technical subject than some clickety-markety new CSS grid framework piece.

                                        1. 7

                                          Omitting the “technical” part is a big problem.

                                          1. 5

                                            You’ve made this claim many times, but it’s clear that there is a large contingent of members–including similarly long-tenured members–who feel otherwise, and it isn’t clear why your vision for the site should be dispositive.

                                            1. 3

                                              This is pretty much the only description of Lobsters on the About page:

                                              Lobsters is a computing-focused community centered around link aggregation and discussion, launched on July 1st, 2012.

                                              Content is added and somewhat curated by the active community. Whatever is posted today will be the hallmark for what is posted tomorrow. This would evolve over time.

                                              I retracted calling it a ‘technical’ community as I think the word is ambiguous to the point where it can mean pretty much anything. Stories should just match up with the tags defined, and if they are beneficial to enquiring minds then they should have a place here.

                                              1. 1

                                                These posts don’t seem to be overwhelming the conversation on the site. Is the problem that these haven’t been downvoted below zero?

                                          1. 6

                                            Disclaimer: I’m a new user myself, only having been around for about a year.

                                            I don’t think the question of human rights and technology is necessarily off-topic for Lobsters. It does, however, need the culture tag with the hotness modifier. However, that comment thread was off-topic to the story at hand. The culture tag exists for a reason, so that people can ignore non-technical aspects if they so desire.

                                            Why wasn’t the off-topic flag used more? If the community can’t be trusted to police itself, do you think it’s going to be allowed to do so?

                                            In my mind at least, flagging is the ultimate last resort. In debates about ethics, flagging and disagreeing can be dangerously close. That’s why I’d tend to abstain from voting on such threads entirely. I’d also argue that it is loosely (if not straight-up poorly) defined: Is it off-topic for Lobsters, the story at hand, both, neither?

                                            At some point, it also just makes no difference anymore. When a comment is in the double digits, why even bother anymore?

                                            1. 6

                                              Seems this site got hug-of-deathed.

                                              1. 1

                                                Thinks seem to have calmed down now and it seems back up. It’s a remote ssh session to an SDF server that spins up VMs with historical UNIXes on demand.

                                              1. 1

                                                Has The UNIX Heritage Society taken notice of this yet? It seems they’re still debating how to celebrate UNIX’s 50th.

                                                1. 5

                                                  Title is slightly wrong. You can boot it but you can’t install it because the OS is blocked from seeing the internal storage.

                                                  1. 15

                                                    I don’t think “blocked from seeing the internal storage” is quite the correct characterization. The T2 chip is acting as an SSD controller, I bet if somebody takes the time to write a T2 driver for Linux everything will work just fine. The difficulty there will likely be that there is no datasheet available for the chip so the driver will have to be reverse engineered from mac OS which is certainly not trivial.

                                                    1. 5

                                                      This has shades of the “Lenovo is blocking Linux support” “incident” where Lenovo just forced the storage controller into a RAID mode Linux didn’t have a driver for.

                                                      1. 2

                                                        At least from what the system report tool says the drive appears as an NVME SSD and just an iteration on the one from previous generations (AP0512J vs AP0512M in the 2018 Air). So it might just work with the Linux NVME drivers once there’s a working UEFI shim that’s trusted. At that point this tutorial seems plausible.

                                                        1. 3

                                                          Trust is not an issue because secure boot can be completely disabled.

                                                          As the article mentions, people who tried live USBs found out that the internal storage is not recognized. So looks like T2 is indeed actually acting as an SSD controller. (And of course macOS would report the actual underlying SSD even if there is no direct connection to it. The T2 could be reporting that info to the OS.)

                                                      2. 8

                                                        The difficulty there will likely be that there is no datasheet available for the chip

                                                        Unless they completely and utterly butchered the initialization, no amount of datasheets will save you. From the T2 documentation:

                                                        By default, Mac computers supporting secure boot only trust content signed by Apple. However, in order to improve the security of Boot Camp installations, support for secure booting Windows is also provided. The UEFI firmware includes a copy of the Microsoft Windows Production CA 2011 certificate used to authenticate Microsoft bootloaders.

                                                        NOTE: There is currently no trust provided for the the Microsoft Corporation UEFI CA 2011, which would allow verification of code signed by Microsoft partners. This UEFI CA is commonly used to verify the authenticity of bootloaders for other operating systems such as Linux variants.

                                                        To bypass the check of the cryptographic signature, you’d probably have to find some kind of exploitable vulnerability in the verification code (or even earlier in the boot process so that you get code execution in the bootloader before the actual check).

                                                        1. 8

                                                          As the article says, you can disable the T2 Secure Boot so the code signature verification is not the problem at that point. The problem then is that the T2 acts as the SSD controller, and nobody has taught Linux yet how to talk to a T2 chip. The article incorrectly conflates the two issues.

                                                          1. 5

                                                            Doesn’t look like it’s conflating them. You might have to scroll down further :) but there’s a screenshot of the Startup Security Utility and this text:

                                                            However, reports have come in that even with it disabled, users are still unable to boot a Linux OS as the hardware won’t recognize the internal storage device. Using the External Boot option (pictured above), you may be able to run Linux from a Live USB, but that certainly defeats the purpose of having an expensive machine with bleeding-edge hardware.

                                                          2. 2

                                                            Secure boot can be disabled. Then the machine will boot anything you tell it to boot, bringing the security inline with machines predating the T2.

                                                            Source: I tried it out on my iMac pro which is a T2 machine.

                                                            1. 1

                                                              edit: mis-read that. Yeah until they add partner support you’re probably pretty stuck. Although somebody like RedHat or Canonical that have relationships with Microsoft might be able to have them cross-sign their shim to support booting on the new Air. Either that or we’re stuck waiting for Apple to support the UEFI CA.

                                                        1. 5

                                                          Oy, another build tool.

                                                          I kind of weary of seeing them show up. Each subtle in their own right, with deep strangenesses and incompatibilities. One Ring to Rule Them All would be grand.

                                                          1. 4

                                                            An important thing to preserve: for most projects, you can cd into the project directory, and type make (with sometimes configure) and end of the story.

                                                            1. 6

                                                              (this reply may or may not contain trolling)

                                                              • Doesn’t apply to Windows,
                                                              • Sometimes you need to run ./autogen.sh
                                                              • Sometimes you need to install a few packages (you have to know the names) because autoconf isn’t bundled as one package on most distros,
                                                              • Sometimes autoconf scripts require its tool packages to be installed in specific versions,
                                                              • Learning to use autoconf requires you to learn a build system which contains backward compatibility for shells/systems that are installed on maybe 10 machines worldwide.
                                                              1. 3

                                                                autotools can be used by developers or maintainers, the release tarballs will not require you to run autogen nor install autoconf/automake

                                                                1. 4

                                                                  Except cases where you’re the user and you need to use the git version, because it contains a fix for some obscure bug only you’re encountering ;)

                                                                  1. 4

                                                                    Or, as is increasingly common, there are no releases and the git repo is rolling-release.

                                                                2. 2

                                                                  And then you spend hours tracing m4 scripts because there’s a bug in autofools.

                                                                  1. 1

                                                                    What a joy to dig into the autogenerated configure file to debug what is going wrong when you compile statically a project with 10+ libraries !

                                                                    1. 1

                                                                      That’s true, but normally you shouldn’t dig into autogenerated makefiles, unless you’re debugging CMake itself. Standard case is that you debug your build on CMakeLists level (if you’re using CMake).

                                                                    2. 1

                                                                      Doesn’t apply to Windows,

                                                                      I’m there now with a work project that can build on Linux with some of it also on Windows but uses all of CMake and premake on top of autotools, gmake/nmake, gcc toolchain/VC++ toolchain. For a mixed Python/C project this is too much baggage for external users so I’m trying the waf build system. I happen to be stuck on a peculiar and possibly locally inflicted Windows linking behavior but writing rules in a Python DSL is great.

                                                                      1. 1

                                                                        I’ve tried waf some time ago. It was nice on the beginning, but after a year of using it I’ve stopped understanding my own build systems, because they were nearly standalone Python programs in their own right. Still it was better than pure Makefiles though.

                                                                      2. 1

                                                                        Learning to use autoconf requires you to learn a build system which contains backward compatibility for shells/systems that are installed on maybe 10 machines worldwide.

                                                                        And people like me thank them for that!

                                                                      3. 4

                                                                        yeah…. that is why, despite its manifest and many defects, I tend to default to make for projects that aren’t deeply intermeshed into a single build system. It often calls out to the ecosystem-specific toolsystem (my home work these days is mostly ocaml, for instance).

                                                                        ./build.sh is also a nice standard to have.

                                                                        I’m not opposed to redo or another build system. But new generalized systems IMO have to be clearly and visibly The Better Way Forward: 10x or more the obvious effectiveness of make, for someone comfortable writing make.

                                                                        (mumble: maybe if we stopped writing C/C++, build systems coagulated around that arcane world would stop appearing, letting us get on with writing new scala build systems)

                                                                        1. 3

                                                                          I often write a trivial makefile that calls whatever other build tool I’m using, just to preserve this.

                                                                          1. 2

                                                                            In all my projects, the Makefile is the entrypoint to building, developing, testing & sometimes even deploying the software.

                                                                            Yes, you usually end up calling out to programming-language-specific tools underneath (like mix or cabal), but the ability to organize tasks in a dependency tree and to have a single place where they’re all listed is great. Especially when you come back to a project after a long break.

                                                                          2. 4

                                                                            One Ring to Rule Them All would be grand

                                                                            Unlikely. But here’s One Theory to Classify Them All:

                                                                            (Yes, same work posted twice, six months apart, by different people.)

                                                                            1. 1

                                                                              Thanks for links. I just tied a knot around them to facilitate easier discovery of both for anyone who lands on just one.

                                                                          1. 2

                                                                            Not enough animations around everything.

                                                                            1. 1

                                                                              “Windows as a Service” absolutely does not mesh with “two feature releases a year”

                                                                              Continuous release or bust! Web developers know how much nicer it is in this space, and if Windows is really going down this route they should go for it!

                                                                              1. 5

                                                                                Continuous release or bust!

                                                                                Which they actually are doing, just with Office 365.

                                                                              1. 5

                                                                                I’ve been trying to get a patchfix into OpenBSD with no luck. No response to my patch on tech@openbsd.org. This isn’t the first time. Can any OpenBSD contributor help me out?

                                                                                1. 7

                                                                                  If you didn’t get any feedback, just keep asking the list for feedback every two weeks by replying to your own post. There’s a bit of luck to it because each patch has to catch someone’s interest in a moment when they have time to deal with it.

                                                                                  1. 4

                                                                                    Cool I can do that, thanks for the tip.

                                                                                    1. -1

                                                                                      just keep asking the list for feedback every two weeks by replying to your own post.

                                                                                      What a ridiculous response. Not even an apology. That’s no way to run a welcoming community or encourage people to contribute.

                                                                                      1. 9

                                                                                        Nothing to apologize for - what did you expect? Sending reminders is a common idiom on tech@ where a mail gets drown easily by other threads.

                                                                                        Making sure your submissions are well tested and reasoned helps getting a response, but you cannot demand anything.

                                                                                        1. 1

                                                                                          what did you expect?

                                                                                          Maybe this is how OpenBSD runs things, if that’s the culture there, that’s fine, but don’t expect it to attract very many contributors.

                                                                                          1. 5

                                                                                            It does attract contributors. In fact, this culture is one of the reasons joined the project.

                                                                                            So I eventually started reviewing the diff but failed to do so because it was both malformed (did not apply) and broken (did not compile). That is, instead of focusing on the intented changes, reviewers get thrown back because they did not test it. Note how I explicitly mentioned this in my previous reply.

                                                                                            Edit: I mixed you up with the OP/diff author, text adjusted.

                                                                                            1. 4

                                                                                              Thank you for the review kn, very much appreciated. I hastily reposted an old version of the patch. I’ll make sure the diff applies cleanly in my reply and fix up the SIGCHLD typo.

                                                                                            2. 6

                                                                                              Maybe this is how OpenBSD runs things, if that’s the culture there, that’s fine, but don’t expect it to attract very many contributors.

                                                                                              Ah but whose job is it to reply to every mail? Whose job is it to apologize if whoever had the first job failed to deliver? What is this sentient entity called OpenBSD that supposedly runs things? Does it have the power to appoint an individual for such a role?

                                                                                              1. -2

                                                                                                What is this sentient entity called OpenBSD that supposedly runs things?

                                                                                                It’s called the OpenBSD Foundation. You can read about it on its website. This year, it has about half a million to spend on answering your other questions.

                                                                                                1. 8

                                                                                                  You gotta be joking. They provide funding for the project. They don’t run the project.

                                                                                                  1. 0

                                                                                                    I assumed that in order to provide funding for a project you need to decide what to fund and what not to fund, and that sort of decision-making is called “running the project”, but I guess I was mistaken, my bad.

                                                                                                    1. 7

                                                                                                      I just decided to fund you as my personal assistant. Your salary is $20 a month, you work 24/7, aren’t you so glad that I run you now? Hand over the keys to your house by the way, because with this decision, I run it…

                                                                                                      Actually the OpenBSD Foundation isn’t the OpenBSD Project. The OpenBSD Foundation doesn’t own OpenBSD, and there are things it cannot do because it does not own OpenBSD. It can’t hand out commit bits, it can’t change the website, it can’t turn people into mailing list admins.. it does not run OpenBSD. If someone or something really “runs” OpenBSD, I’d say it’s Theo… and no, Theo doesn’t run the Foundation. The Foundation doesn’t run Theo either. The Foundation doesn’t decide what Theo or the individual developers (volunteers mainly!) of the project do, though they can choose to support whatever it is by providing funding.

                                                                                                      1. -1

                                                                                                        What is this sentient entity called OpenBSD that supposedly runs things?

                                                                                                        If someone or something really “runs” OpenBSD, I’d say it’s Theo

                                                                                                        1. 5

                                                                                                          Which leads to the follow up question.. you want him to force the volunteers to reply to every mail and apologize for every mail that wasn’t responded to? Or you want him to employ people for that purpose? Out of his own pocket?

                                                                                                          Sorry, I just don’t see the issue of some messages directed at a volunteer-driven software group going unresponded to because the volunteers happened to be volunteering their time for something else at the time (or whatever the reason).

                                                                                                          If people are so entitled to responses, I no longer wonder why some people get burned out on OSS development. I wouldn’t, at least not for that reason, because I have no trouble ignoring issues I don’t have time for. It is my own time.

                                                                                                          IMHO kn is right, there is nothing to apologize for.

                                                                                                          1. 0

                                                                                                            I’ve seen small businesses provide better support to their users and developers on far less budget than OpenBSD has.

                                                                                                            For the past 5 or so years they’ve received hundreds of thousands of dollars each year, and each year they had a surplus averaging ~$100k that they didn’t seem to use for anything.

                                                                                                            Are you telling me they can’t afford to pay someone to say, “we’re looking into this”, or “we’re sorry the patch didn’t compile”, or even setup an automated patch submission system? Because if you are, according to their public finances page, that would be a lie.

                                                                                                            1. 3

                                                                                                              The OpenBSD Project isn’t a business. I think you’re just trolling here and it’s dumb.

                                                                                                              1. -1

                                                                                                                I’m not trolling, and I’m done with this conversation because it’s clear it’s going nowhere fast.

                                                                                                                EDIT: and to be clear, from the OSS projects I’ve seen — even those that do not have a half-million dollar budget and a foundation — still somehow manage to reply to developers who’ve put in the time and effort into submitting a pull request. They also have pull requests. And automated build systems. And aren’t stuck in 1990 with their version control system.

                                                                                                                1. 6

                                                                                                                  You are generalizing from one example and you don’t know our comunity well enough to judge it.

                                                                                                                  During almost 10 years now I have committed many patches from other contributors and never had my own patches go ignored, which is why I stuck around in OpenBSD in the first place.

                                                                                                                  1. 0

                                                                                                                    You are generalizing from one example and you don’t know our comunity well enough to judge it.

                                                                                                                    And how do you know how well I know the OpenBSD community? You have no clue.

                                                                                                                    Over on Mastodon I pointed out that OpenBSD “perpetuates false and negative stereotypes that security people don’t care about usability, or that security must come at a cost of usability”.

                                                                                                                    That’s a fact. And then OpenBSD developer @mulander jumped in to call me a troll, and on top of it, demand that I work for free to submit patches to the project. So I pointed out to him how the OpenBSD community treats those who work for free and submit patches.

                                                                                                                    I’ve observed this project for many years, and I think it gets a bit too much hype on Lobsters lately for delivering a terrible user experience. Sure, there are lots of things to praise about it, but I don’t see anyone criticizing it for its glaringly obvious faults, so the end result is a community that is delusional, and a harmful role model.

                                                                                                                    1. 4

                                                                                                                      Link the thread so people can judge by themselves.

                                                                                                                      Also link yourself trying to spin the thing around on Mastodon and on twitter.

                                                                                                                      1. 0

                                                                                                                        I did, see my reply below from before your comment. But sure I should have linked it here as well.

                                                                                                                      2. 1

                                                                                                                        Your opinions are not facts. I don’t think the “community” is what’s delusional here.

                                                                                                                        1. -1

                                                                                                                          It’s not an opinion, it’s a fact, and one OpenBSD fanbois don’t dispute.

                                                                                                                    2. 1

                                                                                                                      Great. I hope you feel better now that you’ve got this all out of your system.

                                                                                              2. 4

                                                                                                There is nothing to apologize for. It is a volunteer project. Developers are people who live lives, not borg drones assimilating other people’s patches.

                                                                                                1. 4

                                                                                                  All of your comments in this thread of inappropriate. They are inappropriate regardless of whether other folk’s comments are or are not appropriate and regardless of whether they do or do not contain true statements.

                                                                                                  Please drop the issue, do not bring it back up, and do not engage in this style of discussion again on lobste.rs.

                                                                                              3. 3

                                                                                                What stsp said, but also, can you link us to the thread?

                                                                                                  1. 2

                                                                                                    I just get

                                                                                                    I expected an e-mail address, but none was defined.

                                                                                                    1. 2

                                                                                                      Sorry I’m not entirely sure what the best way is to post a link to a thread on the OpenBSD listserv. If you log in you should be able to see the thread.

                                                                                                      EDIT: use this http://openbsd-archive.7691.n7.nabble.com/lib-libfuse-Handle-signals-that-get-sent-to-any-thread-tp352472p353099.html

                                                                                                      1. 2

                                                                                                        marc.info works pretty well. I’d say it’s the preferred interface for most people.

                                                                                                        1. 1

                                                                                                          thanks for the pointer

                                                                                                1. 2

                                                                                                  Not a contributor, but I figure it might help to point out what patch you sent.

                                                                                                1. 9

                                                                                                  it is not a release actually, it’s just a work-in-progress description of 6.4

                                                                                                  1. 5

                                                                                                    Indeed, you are right. It is going to be released in November 1st. I’m happy to see:

                                                                                                    • qcow2 support for vmm and vmd.
                                                                                                    • Go 1.11
                                                                                                    • Lua 5.3.5
                                                                                                    • Rust 1.29.1

                                                                                                    It always amazes me how such a small team can keep up with changes, and manage to maintain the most consistent and coherent Unix system still in development.

                                                                                                    1. 2

                                                                                                      ah .. that’s probably why the upgrade link doesn’t work yet.

                                                                                                      1. 1

                                                                                                        Well, now it is a release see this e-mail to the tech@ list and this tweet, but the link can’t be resubmitted yet.

                                                                                                      1. 6

                                                                                                        It is either better than this, or possibly much worse:

                                                                                                        1. better because people are using distribution packages, where the maintainers of the distribution take care of fixing security issues, e.g. Red Hat Enterprise Linux/CentOS which still uses PHP 5.4 (already unsupported by upstream);
                                                                                                        2. worse, because hosting providers built PHP from source, a long time ago, and never bother to update it at all, so even if they run 7.x, they could run a version with security problems…

                                                                                                        So, to just focus on the version number could be misleading if you talk about security. It could very well be the other way around, i.e.: running the official CentOS PHP package could be more secure than some hoster’s build of PHP 7.x last updated 1 year ago.

                                                                                                        Then again: I just found this: https://access.redhat.com/security/cve/cve-2018-17082 with not a lot of action so far. So yeah.

                                                                                                        1. 2

                                                                                                          Regarding the linked CVE-2018-17082: I think it’s impossible to make the browser issue a request with Transfer-Encoding: chunked and so do others.

                                                                                                          I’ll owe you a beverage of your choice[1] if you can prove otherwise, though :-)

                                                                                                          [1] within reasonable financial bounds, of course.

                                                                                                          1. 1

                                                                                                            Right, so it makes sense there is no rush to fix this issue… Thanks for the link!

                                                                                                          2. 2

                                                                                                            Thank you for being the voice of reason. I’m getting sick of all the frantic warnings coming from PHP software about this. When I try to engage with the developers, they flip out and tell me I’m irresponsible, basically for not running the latest ‘greatest’ version of PHP. They seem horrified that anyone could have the audacity to backport relevant security fixes into older versions, and insist that doing this is somehow less secure. In fact, the opposite is true, since as far as I can tell, 100% of PHP security bugs arise from the work of the official developers, not from people doing backports.

                                                                                                            1. 4

                                                                                                              They seem horrified that anyone could have the audacity to backport relevant security fixes into older versions

                                                                                                              There’s probably part of the general sentiment developers seem to have that running anything but the latest verison borders on blasphemy. The amount of programs that don’t support language/library/tooling versions available on RHEL 7 is astounding. Personally, I consider the latest major release of RHEL after going back for a year (i.e. latest is not included until a year after its release) the baseline of platforms my code must be able to run on.

                                                                                                              RHEL 7 was released only four years ago. For comparison, Firefox supported Windows XP (released 17 years ago) until September of this year.

                                                                                                          1. 1

                                                                                                            Author says a common class of gadgets uses such and such registers. Says avoid them in favor of other registers. Maybe the gadget type with those registers is common because the registers themselves are common from compiler choices. Switching registers might lead to gadgets just using those registers instead. Or are there x86-specific reasons that using different registers will do entirely different things you can’t gadget?

                                                                                                            Other than that confusion, slides look like great work. Especially on ARM.

                                                                                                            1. 15

                                                                                                              Author here. Thanks for having a look! It was fun to do this talk.

                                                                                                              Yes, there are X86 specific reasons that other registers don’t result in ROP gadgets. If you look at Table 2-2 in the Intel 64 and IA-32 Architectures Software Developer’s Manual you can see all of the ModR/M bytes for each register source / dest pair, and other places in that section describe how to encode the ModR/M bytes for various instructions using all of the possible registers. When I surveyed the gadgets in the kernel and identified which intended instructions resulted in C3 bytes that were used as returns in gadgets, there were a large number of gadgets that were terminating on the ModR/M byte encoding the BX series registers. You are correct that these gadgets are common because the compiler frequently chooses to use the BX series registers, and the essence of my change to clang is to encourage the compiler to choose something else. By shifting RBX down behind R14, R15, R12 and R13 the compiler will choose these registers before RBX, and therefore reduce the incidence of the use of RBX resulting in a C3 ModR/M byte. We can see that this works because just shifting the BX registers down the list results in fewer unique gadgets.

                                                                                                              To directly answer your inquiry, gadgets arising from using R14, R15, R12, R13 instead (now that they will be more common) are not a problem. The REX prefix is never C3, and we can look at the ModR/M bytes encoding operations using those registers, and none of them will encode to C3. When I look at gadgets that arise from instructions using these registers, they don’t get their C3 bytes from the instruction encoding - they get them from constants where the constant encodes to a C3, so the register used is irrelevant in these cases. So moving RBX down behind R14, R15, R12 and R13 doesn’t result in more gadgets using those registers.

                                                                                                              There are other register pairs that result in a C3 ModR/M byte. Operations between RAX and R11 can result in a C3 ModR/M byte, but these are less common when we survey gadgets in the kernel (~56 in the kernel I have here now). RAX and R11 were already ahead of RBX in the default list anyway, so moving RBX down the list does not result in more gadgets using R11. If you ask why we haven’t moved R11 down next to RBX, the answer is that gadgets using R11 this way are not that numerous, so it hasn’t risen to the top of the heap of most-common-sources-of-gadgets (and therefore has not got my attention). There are many other sources of gadgets that can be fixed and will have a larger impact on overall gadget counts and diversity.

                                                                                                              I hope this clarifies that part of the talk. :-)

                                                                                                              1. 3

                                                                                                                Thank eveyone for the answers. Thank you in particular for this very-detailed answer that clarifies how x86’s oddities are creating the attack vectors.

                                                                                                                The reason I wanted to know is that I planned to design around high-end ARM chips instead of x86 where possible because I believed we’d see less ISA-related attacks. Also, certain constructions for secure code might be easier to do on RISC with less performance hit. Your slides seem to support some of that.

                                                                                                                1. 2

                                                                                                                  To be fair, x86 doesn’t create the attack vectors, but does make any bugs much easier to exploit.

                                                                                                                  ARM doesn’t have nearly the same problem - you can always ROP into a jump to THUMB code on normal ARM instructions, but these entry points are usually more difficult to find than an 0xc3.

                                                                                                                2. 1

                                                                                                                  I’m curious to learn more about ROP. I’d like to examine adding support for another target to ROPgadget.py. So what designates a gadget? Any sequence of instructions ending in a return? How do attackers compose functionality out of gadgets? By hand, or is there some kind of a ‘compiler’ for them?

                                                                                                                  1. 3

                                                                                                                    You might be interested in the ROP Emporium’s guide. Off the top of my head the only automatic tools I know of are ropper and angrop.

                                                                                                                3. 5

                                                                                                                  Switching registers might lead to gadgets just using those registers instead. Or are there x86-specific reasons that using different registers will do entirely different things you can’t gadget?

                                                                                                                  If I understand this correctly, it’s because the ebx register causes opcodes to be created that contain a return instruction, i.e., opcodes that are useful in ROP. So by avoiding ebx as much as possible, you also avoid creating collateral ROP gadgets with early returns. This issue only happens because x86/amd64 have variable-length opcodes.

                                                                                                                  1. 4

                                                                                                                    As far as I understand, the register allocation trick is indeed x86-specific. The point is to avoid C3 bytes because these will polymorph into the RET instruction when used in unaligned gadgets. See the “polymorphic gadget” and ‘register selection’ sections in the slide set.

                                                                                                                  1. 4

                                                                                                                    We should get rid of votes altogether. After watching this site recently I see poor relationship between upvoted comments and how much effort/information a commentator puts in. Down votes have always been to indicate disagreement or displeasure (merely indicating an unpopular opinion) despite guidelines being that these are to be reserved for disruptive content.

                                                                                                                    I find myself skimming all the comments because I know many good comments will be down below, past the anodyne, popular ones.

                                                                                                                    1. 9

                                                                                                                      I’d kind of like to get rid of the bimonthly discussions about voting.

                                                                                                                      1. 3

                                                                                                                        It’s the only standup we have.

                                                                                                                        1. 2

                                                                                                                          Come on: some of the political commentary should count as standup. Saying nothing more, everyone will imagine something different. Haha.

                                                                                                                      2. 4

                                                                                                                        I’d upvote this comment to express my complete agreement… but I can’t because of the little ~. Oh well.

                                                                                                                        The de-facto cultural meaning of up and down votes has been largely defined by Reddit, and will not be easily redefined by site-specific guidelines off on a wiki somewhere. People don’t work like that.

                                                                                                                        Simple chronological ordering of comment would work just fine for Lobste.rs, and discourage groupthink. We don’t need to rank users by popularity, or hand out little micro-rewards to those who post popular things. Or, do we?

                                                                                                                        1. 2

                                                                                                                          Simple chronological ordering of comment

                                                                                                                          That would be interesting. The moderators do want the agitated or lower-quality stuff toward the bottom and collapsed for a reason, though. Even I compromised in the metas on censorship that I’d take “Collapsed, not Deleted/Banned” as a default if we couldn’t compromise on anything better. It lets casual readers get lower-stress, lower-noise experience with little effort while people with more patience to explore riskier threads can still hit plus to see them. I always do that just to see what’s going on in community if nothing else. There’s also often some good comments in there somewhere if it’s a debate.

                                                                                                                        2. 4

                                                                                                                          I mean, Lobsters already have a reason system for downvotes that is mandatory and I believe that does deter some downvotes for mere disagreement. One wonders if a categorization of upvotes and upvote caps like Slashdot has might help deter upvotes for mere agreement.

                                                                                                                          1. 4

                                                                                                                            We already did this, or at least removed downvotes.

                                                                                                                            The results were…not encouraging.

                                                                                                                            1. 1

                                                                                                                              Wait, when did this happen?

                                                                                                                              1. 3

                                                                                                                                @jcs removed downvotes for a time a while back.

                                                                                                                          1. 2

                                                                                                                            I can’t help but be surprised that authenticated encryption/AEAD doesn’t seem to be intended to be or become the default for symmetrical encryption in Percival. crypto_secretbox in NaCl and libsodium as well as crypto_lock in Monocypher all default to some AE or AEAD construction.