1. 2

    Did they completely abandon paths[]? Filesystem access is typically the thing I want to restrict :(

    1. 3

      Not abandoned but split into a different syscall, pledgepath(2), which is not committed, yet. As Theo mentioned a few months ago:

      The other is pledgepaths. The semantics are still being tuned a bit. Before the first call to pledge() in a process, one can pledgepath() directories. Then later after pledge(), file access operations only work if the traversal of the path crosses one of those pre-declared directories (but better make sure you don’t move a directory, because the kernel remembers and reasons about the vnode of the directory rather than the path). Something similar is being worked on for files, but we are still adjusting that, as well as a flag parameter for the pledgepath() call which may constrain the operations done on such files. As such, pledgepath() will become a filesystem containment mechanism unlike chroot() because paths will still be based upon true /.

    1. 4

      Why does this work?

      1. 13

        The graph is the colored directed graph with vertices 0, 1, 2, 3, 4, 5, 6 and the following rules

        • the white vertex is 0, all others are black
        • the black arrows connect k to (k+1)%7
        • the white arrows connect k to (10*k)%7

        So each vertex has two outgoing arrows, one black, one white.

        If you know the remainder of n modulo 7, this graph allows you to read off the remainders of n+1 and 10*n by following the black and white arrows emanating from the vertex n%7. An integer is divisible if and only if its remainder modulo 7 is 0.

        In the example given on the page, you calculate the remainder of 325 as follows:

        325 = ((3 *10 + 2) *10 + 5 = ((0+1+1+1) *10 +1+1) *10 +1+1+1+1+1.

        Start on the white vertex, follow three times a black arrow, then once a white arrow, twice a black arrow, once a white arrow then five times a black arrow and you land on vertex 3 = 325%7.

        1. 3

          What makes 7 special? Could I construct a similar graph for 5 or 11?

          1. 5

            There’s nothing special about 7 here. Except that it’s the first number for which all divisibility tests in base 10 are kinda tricky, so it’s neat that you can visualize one such test in a planar graph.

            You can define similar graphs G(b,m) for every base b>=2 and every modulus m>=1: the set of vertices is {0,...,m-1} and k is connected to (k+1)%m and (b*k)%m for each vertex k. Thus, the graph in the post is G(10,7).

            The author lists the planar graphs for small values of b and m in this comment.

            1. 1
        1. 32

          I wasn’t implying. I was stating a fact.

          And he’s wrong about that.

          https://github.com/uutils/coreutils is a rewrite of a large chunk of coreutils in Rust. POSIX-compatible.

          1. 12

            So on OpenBSD amd64 (the only arch rust runs on… there are at least 9 others, 8 7 or 6 of which rust doesn’t even support! )… this fails to build:

            error: aborting due to 19 previous errors
            
            error: Could not compile `nix`.
            warning: build failed, waiting for other jobs to finish...
            error: build failed
            
            1. 8

              Yep. The nix crate only supports FreeBSD currently.

              https://github.com/nix-rust/nix#supported-platforms

            2. 8

              The openbsd guys are stubborn of course, though they might have a point. tbh somebody could just fork a BSD OS to make this happen. rutsybsd or whatever you want to call it.

              edit: just tried to build what you linked, does cargo pin versions and verify the downloads? fetching so many dependencies at build time makes me super nervous. Are all those dependencies BSD licensed? It didn’t even compile on my machine, maybe the nixos version of rust is too old - i don’t know if the rust ecosystem is stable enough to base an OS on yet without constantly fixing broken builds.

              1. 10

                just tried to build what you linked, does cargo pin versions and verify the downloads?

                Cargo pins versions in Cargo.lock, and coreutils has one https://github.com/uutils/coreutils/blob/master/Cargo.lock.

                Cargo checks download integrity against the registry.

                For offline builds, you can vendor the dependencies: https://github.com/alexcrichton/cargo-vendor, downloading them all and working from them.

                Are all those dependencies BSD licensed?

                Yes. Using: https://github.com/onur/cargo-license

                Apache-2.0/MIT (50): bit-set, bit-vec, bitflags, bitflags, block-buffer, byte-tools, cc, cfg-if, chrono, cmake, digest, either, fake-simd, filetime, fnv, getopts, glob, half, itertools, lazy_static, libc, md5, nodrop, num, num-integer, num-iter, num-traits, num_cpus, pkg-config, quick-error, rand, regex, regex-syntax, remove_dir_all, semver, semver-parser, sha2, sha3, tempdir, tempfile, thread_local, time, typenum, unicode-width, unindent, unix_socket, unreachable, vec_map, walker, xattr

                BSD-3-Clause (3): fuchsia-zircon, fuchsia-zircon-sys, sha1

                MIT (21): advapi32-sys, ansi_term, atty, clap, data-encoding, generic-array, kernel32-sys, nix, onig, onig_sys, pretty-bytes, redox_syscall, redox_termios, strsim, term_grid, termion, termsize, textwrap, void, winapi, winapi-build

                MIT OR Apache-2.0 (2): hex, ioctl-sys

                MIT/Unlicense (7): aho-corasick, byteorder, memchr, same-file, utf8-ranges, walkdir, walkdir

                It didn’t even compile on my machine, maybe the nixos version of rust is too old - i don’t know if the rust ecosystem is stable enough to base an OS on yet without constantly fixing broken builds.

                This is one of my frequent outstanding annoyances with Rust currently: I don’t have a problem with people using the newest version of the language as long as their software is not being shipped on something with constraints, but at least they should document and test the minimum version of rustc they use.

                coreutils just checks against “stable”, which moves every 6 weeks: https://github.com/uutils/coreutils/blob/master/.travis.yml

                Can you give me rustc --version?

                Still, “commitment to stability” is a function of adoption. If, say, Ubuntu start shipping a Rust version in an LTS release, more and more people will try to stay backward compatible to that.

                1. 2

                  rustc 1.17.0 cargo 0.18.0

                  1. 11

                    You’re probably hitting https://github.com/uutils/coreutils/issues/1064 then.

                    Also, looking at it, it is indeed that they use combinatior functionality that became available in Rust 1.19.0. std::cmp::Reverse can be easily dropped and replaced by other code if 1.17.0-support would be needed.

                    Thanks, I filed https://github.com/uutils/coreutils/issues/1100, asking for better docs.

                    1. 1

                      thanks for doing that, great community outreach :P

                2. 5

                  Rust is “stable” in the sense that it is backwards compatible. However it is evolving rapidly so new crates or updates to crates may require the latest compiler. This won’t mean you’ll have to constantly fix broken builds; just that pulling in new crates may require you to update to the latest compiler.

                  1. 4

                    Yes, Cargo writes a Cargo.lock file with versions and hashes. Application developers are encouraged to commit it into version control.

                    Dependencies are mostly MIT/Apache in the Rust world. You can use cargo-license to quickly look at the licenses of all dependencies.

                    Redox OS is fully based on Rust :)

                  2. 4

                    Although you’re right to point out that project, one of Theo’s arguments had to do with compilation speeds:

                    By the way, this is how long it takes to compile our grep:

                    0m00.62s real 0m00.63s user 0m00.53s system

                    … which is currently quite undoable for any Rust project, I believe. Cannot say if he’s exaggerating how important this is, though.

                    1. 10

                      Now, at least for GNU coreutils, ./configure runs a good chunk of what rust coreutils needs to compile. (2mins for a full release build, vs 1m20.399 just for configure). Also, the build is faster (coreutils takes a minute).

                      Sure, this is comparing apples and oranges a little. Different software, different development states, different support. The rust compiler uses 4 cores during all that (especially due to cargo running parallel builds), GNU coreutils doesn’t do that by default (-j4 only takes 17s). On the other hand: all the crates that cargo builds can be shared. That means, on a build farm, you have nice small pieces that you know you can cache - obviously just once per rustc/crate pairing.

                      Also, obviously, build farms will pull all kinds of stunts to accelerate things and the Rust community still has to grow a lot of that tooling, but I don’t perceive the problem as fundamental.

                      EDIT: heh, forgot --release. And that for me. Adjusted the wording and the times.

                      1. 5

                        OpenBSD doesn’t use GNU coreutils, either; they have their own implementation of the base utils in their tree (here’s the implementation of ls, for example). As I understand it, there’s lots of reasons they don’t use GNU coreutils, but complexity (of the code, the tooling, and the utils themselves) is near the top of the list.

                        1. 6

                          Probably because most(all?) the openBSD versions of the coreutils existed before GNU did, let alone GNU coreutils. OpenBSD is a direct descendant of Berkeley’s BSD. Not to mention the licensing problem. GNU is all about the GPL. OpenBSD is all about the BSD(and it’s friends) license. Not that your reason isn’t also probably true.

                        2. 2

                          That means, on a build farm, you have nice small pieces that you know you can cache - obviously just once per rustc/crate pairing.

                          FWIW sccache does this I think

                        3. 7

                          I think it would be more fair to look at how long it takes the average developer to knock out code-level safety issues + compiles on a modern machine. I think Rust might be faster per module of code. From there, incremental builds and caching will help a lot. This is another strawman excuse, though, since the Wirth-like languages could’ve been easily modified to output C, input C, turn safety off when needed, and so on. They compile faster than C on about any CPU. They’re safe-by-default. The runtime code is acceptable with it improving even better if outputting C to leverage their compilers.

                          Many defenses of not using safe languages is that easy to discount. And OpenBSD is special because someone will point out that porting a Wirth-like compiler is a bit of work. It’s not even a fraction of the work and expertise required for their C-based mitigations. Even those might have been easier to do in a less-messy language. They’re motivated more by their culture and preferences than any technical argument about a language.

                          1. 3

                            It’s a show stopper.

                            Slow compile times are a massive problem for C++, honestly I would say it’s one of the biggest problems with the language, and rustc is 1-2 orders of magnitude slower still.

                            1. 12

                              It’s a show stopper.

                              Hm, yet, last time I checked, C++ was relatively popular, Java (also not the fastest in compilation) is doing fine and scalac is still around. There’s people working on alternatives, but show stopper?

                              Sure, it’s an huge annoyance for “build-the-world”-approaches, but well…

                              Slow compile times are a massive problem for C++, honestly I would say it’s one of the biggest problems with the language, and rustc is 1-2 orders of magnitude slower still.

                              This heavily depends on the workload. rustc is quite fast when talking about rather non-generic code. The advantage of Rust over C++ is that coding in mostly non-generic Rust is a viable C alternative (and the language is built with that in mind), while a lot of C++ just isn’t very useful over C if you don’t rely on templates very much.

                              Also, rustc stable is a little over 2 years old vs. C/C++ compilers had ample headstart there.

                              I’m not saying the problem isn’t there, it has to be seen in context.

                              1. 9

                                C++ was relatively popular, Java (also not the fastest in compilation) is doing fine and scalac is still around.

                                Indeed, outside of gamedev most people place zero value in fast iteration times. (which unfortunately also implies they place zero value in product quality)

                                rustc is quite fast when talking about rather non-generic code.

                                That’s not even remotely true.

                                I don’t have specific benchmarks because I haven’t used rust for years, but see this post from 6 months ago that says it takes 15 seconds to build 8k lines of code. The sqlite amalgamated build is 200k lines of code and has to compile on a single core because it’s one compilation unit, and still only takes a few seconds. My C++ game engine is something like 80k if you include all the libraries and builds in like 4 seconds with almost no effort spent making it compile fast.

                                edit: from your coreutils example above, rustc takes 2 minutes to build 43k LOC, gcc takes 17 seconds to build 270k, which makes rustc 44x slower…

                                The last company I worked at had C++ builds that took many hours and to my knowledge that’s pretty standard. Even if you (very) conservatively say rustc is only 10x slower, they would be looking at compile times measured in days.

                                while a lot of C++ just isn’t very useful over C if you don’t rely on templates very much.

                                That’s also not true at all. Only small parts of a C++ codebase need templates, and you can easily make those templates simple enough that it has little to no effect on compile times.

                                Also, rustc stable is a little over 2 years old vs. C/C++ compilers had ample headstart there.

                                gcc has gotten slower over the years…

                                1. 6

                                  Even if you (very) conservatively say rustc is only 10x slower,

                                  Rustc isn’t slower to compile than C++. Depends on the amount of generics you use, but the same argument goes for C++ and templates. Rust does lend itself to more usage of generics which leads to more compact but slower-compiling code, which does mean that your time-per-LOC is higher for Rust, but that’s not a very useful metric. Dividing LOCs is not going to get you a useful measure of how fast the compiler is. I say this as someone who has worked on both a huge Rust and a huge C++ codebase and know what the compile times are like. Perhaps slightly worse for Rust but not like a 2x+ factor.

                                  The main compilation speed problem of Rust vs C++ is that it’s harder to parallelize Rust compilations (large compilation units) which kind of leads to bottleneck crates. Incremental compilation helps here, and codegen-units already works.

                                  Rust vs C is a whole other ball game though. The same ball game as C++ vs C.

                                  1. 2

                                    That post, this post, my experience, lines, seconds… very scientific :) Hardware can be wildly different, lines of code can be wildly different (especially in the amount of generics used), and the amount of lines necessary to do something can be a lot smaller in Rust, especially vs. plain C.

                                    To add another unscientific comparison :) Servo release build from scratch on my machine (Ryzen 7 1700 @ 3.9GHz, SATA SSD) takes about 30 minutes. Firefox release build takes a bit more. Chromium… even more, closer to an hour. These are all different codebases, but they all implement a web browser, and the compile times are all in the same ballpark. So rustc is certainly not that much slower than clang++.

                                    Only small parts of a C++ codebase need templates

                                    Maybe you write templates rarely, but typical modern C++ uses them all over the place. As in, every STL container/smart pointer/algorithm/whatever is a template.

                                    1. 2

                                      To add another unscientific comparison :) Servo release build from scratch on my machine (Ryzen 7 1700 @ 3.9GHz, SATA SSD) takes about 30 minutes. Firefox release build takes a bit more. Chromium… even more, closer to an hour. These are all different codebases, but they all implement a web browser, and the compile times are all in the same ballpark. So rustc is certainly not that much slower than clang++.

                                      • Firefox 35.9M lines of code
                                      • Chromium 18.1M lines of code
                                      • Servo 2.25M lines of code

                                      You’re saying compiling 2.25M lines of code for a not feature complete browser that takes 30 minutes is comparable to compiling 18-35M lines of code in ‘a bit more’?

                                      1. 4

                                        Line counters like this one are entirely wrong.

                                        This thing only counted https://github.com/servo/servo. Servo code is actually split among many many repositories.

                                        HTML parser, CSS parser, URL parser, WebRender, animation, font sanitizer, IPC, sandbox, SpiderMonkey JS engine (C++), Firefox’s media playback (C++), Firefox’s canvas thingy with Skia (C++), HarfBuzz text shaping (C++) and more other stuff — all of this is included in the 30 minutes!

                                        plus,

                                        the amount of lines necessary to do something can be a lot smaller in Rust

                                        1. 2

                                          Agreed, it grossly underestimates how much code Chromium contains. You are aware of the horrible depot_tools and the amount of stuff they pull in?

                                          My point was, you are comparing a feature incomplete browser that is a smaller code base at least in one order of magnitude but takes 30 minutes compared to “closer to an hour” of Chromium. If think your argument doesn’t hold - you are free to provide data to prove me wrong.

                                        2. 3

                                          Servo’s not a monolithic codebase. Firefox is monolithic. It’s a bad comparison.

                                          Chromium is also mostly monolithic IIRC.

                                2. 2

                                  Free- and OpenBSD can compile userland from source:

                                  So decent compile times are of essence, especially if you are targeting multiple architectures.

                                3. 6

                                  Well, ls is listed as only semi done, so he’s only semi wrong. :)

                                  1. 11

                                    The magic words being “There has been no attempt”. With that, especially by saying “attempt”, he’s completely wrong. There have been attempts. At everything he lists. (he lists more here: https://www.youtube.com/watch?v=fYgG0ds2_UQ&feature=youtu.be&t=2112 all of what Theo mentions has been written, in Rust, some even have multiple projects, and very serious ones at that)

                                    For a more direct approach at BSD utils, there’s the redox core utils, which are BSD-util based. https://github.com/redox-os/coreutils

                                    1. 2

                                      Other magic words are “POSIX compatible”. Neither redox-os nor the uutils linked by @Manishearth seem to care particularly about this. I haven’t looked all that closely, but picking some random utils shows that none of them is fully compliant. It’s not even close, so surely they can’t be considered valid replacements of the C originals.

                                      For example (assuming that I read the source code correctly) both implementations of cat lack the only POSIX-required option -u and the implementations of pwd lack both -L and -P. These are very simple tools and are considered done at least by uutils…

                                      So, Theo may be wrong by saying that no attempts have been made, but I believe a whole lot of rather hard work still needs to be done before he will acknowledge serious efforts.

                                      1. 5

                                        This rapidly will devolve into a no true scotsman argument.

                                        https://github.com/uutils/coreutils#run-busybox-tests

                                        uutils is running the busybox tests. Which admittedly test for something other than POSIX compliance, but neither the GNU or BSD coreutils are POSIX-compliant anyway.

                                        uutils is based on the GNU coreutils, redox’s ones are based on the BSD ones, which is certainly a step in the right direction and can certainly be counted as an attempt.

                                        For example (assuming that I read the source code correctly) both implementations of cat lack the only POSIX-required option -u and the implementations of pwd lack both -L and -P.

                                        Nobody said they were complete.

                                        All we’re talking about is Theo’s rather strong point that “there has been no attempt”. There has.

                                  2. 1

                                    I’m curious about this statement in TdR in the linked email

                                    For instance, rust cannot even compile itself on i386 at present time because it exhausts the address space.

                                    Is this true?

                                    1. 15

                                      As always with these complaints, I can’t find any reference to exact issues. What’s true is that LLVM uses quite a bit of memory to compile and rustc builds tend not to be the smallest themselves. But not that big. Also, recent improvements have definitely worked here

                                      I do regularly build the full chain on a ACER c720p, with FreeBSD, which has a celeron and 2 GB of RAM, I have to shut down the X server and everything before, but it works.

                                      As usual, this is probably an issue of the kind “please report actual problems, and we work fixing that”. “We want to provide a build environment for OpenBSD and X Y Z is missing” is something we’d be happy support, some fuzzy notion of “this doesn’t fulfill our (somewhat fuzzy) criteria” isn’t actionable.

                                      Rust for Haiku does ship Rust with i386 binaries and bootstrapping compilers (stage0): http://rust-on-haiku.com/downloads

                                      1. 10

                                        As always with these complaints, I can’t find any reference to exact issues.

                                        Only because it’s a thread on the OpenBSD mailing lists, people reading that list have the full context of the recent issues with Firefox and Rust.

                                        I’ll assume you just don’t follow the list so here is the relevant thread lang/rust: update to 1.22.1

                                        • For this release, I had lot of problem for updating i386 to 1.22.1 (too much memory pressure when compiling 1.22 with 1.21 version). So the bootstrap was initially regenerated by crosscompiling it from amd64, and next I regenerate a proper 1.22 bootstrap from i386. Build 1.22 with 1.22 seems to fit in memory.

                                        As I do all this work with a dedicated host, it is possible that ENOMEM will come back in bulk.

                                        And if the required memory still grows, rustc will be marked BROKEN on i386 (and firefox will not be available anymore on i386)

                                        1. 7

                                          Only because it’s a thread on the OpenBSD mailing lists, people reading that list have the full context of the recent issues with Firefox and Rust.

                                          Sure, but has this:

                                          And if the required memory still grows, rustc will be marked BROKEN on i386 (and firefox will not be available anymore on i386).

                                          Reached the Rust maintainers? (thread on the internals mailing list, issue on rust-lang/rust?)

                                          I’m happy to be corrected.

                                          1. 7

                                            Reached the Rust maintainers? (thread on the internals mailing list, issue on rust-lang/rust?)

                                            I don’t know. I don’t follow rust development, however the author of that email is a rust contributor like I mentioned to you in the past so I assume that it’s known to people working on the project. Perhaps you should check on that internals mailing list, I checked rust-lang/rust on github but didn’t find anything relevant :)

                                            1. 7

                                              I checked IRLO (https://internals.rust-lang.org/) and also nothing. (“internals” by the way referring to the “compiler internals”, we have no closed mailing list). The problem on projects of that scale seems to be that information travel is a huge issue and that leads to aggrevation. The reason I’m asking is not that I want to disprove you, I just want to ensure that I don’t open a discussion that’s already happening somewhere just because something is going through social media currently.

                                              Thanks for pointing that out, I will ensure there’s some discussion.

                                              Reading the linked post, it seems to mostly be a regression when doing the jump between 1.21 to 1.22, so that should probably be a thing to keep an eye out for.

                                            2. 2

                                              Here’s a current Rust bug that makes life hard for people trying to work on newer platforms.

                                        2. 2

                                          I’m skeptical; this has certainly worked for me in the past.

                                          I used 32 bit lab machines as a place to delegate builds to back when I was a student.

                                          1. 4

                                            Note that different operating systems will have different address space layout policies and limits. Your effective space can vary from possibly more than 3GB to possibly less than 2GB.

                                      1. 3

                                        The article is honestly quite poor and besides a long intro it only contains three setup instructions:

                                        • machdep.allowaperture=2
                                        • pkg_add xfce xfce-extras firefox vlc
                                        • exec startxfce4

                                        FWIW, I find it much more useful to read Setup OpenBSD with full disk encryption from the same author.

                                        1. 18

                                          FWIW, I find it much more useful to read Setup OpenBSD with full disk encryption from the same author.

                                          Sorry, but that one’s not much better… Why do people keep writing this up, leaving out some steps and adding their own? We’ve documented this in the FAQ and elsewhere for years now. Without suggestions like disabling swap encryption or having only a single partition for / and one for swap and no others. :(

                                          For a relatively up-to-date and more in-depth view, see @stsp’s softraid(4) boot slides for EuroBSDCon 2015.

                                        1. 4

                                          You could simplify a bit and eliminate the two calls to jot(1) and the two for loops by doing this:

                                          if [ $HM -gt 720 ]; then
                                              S=$(( $S + $INC * (1440 - $HM) ))
                                          else
                                              S=$(( $S + $INC * $HM ))
                                          fi
                                          

                                          Running once a minute with an increment of 2 is a bit much since sct rounds the number to a precision of 500, so the sct call in the while loop will only do something once every 250 runs, i.e. roughly every 4 hours.

                                          Finally, since it wasn’t mentioned yet: jung@ polished it a bit and created an OpenBSD port of sct about a year ago.

                                          1. 1

                                            Nice, thanks for the feedback!

                                            1. 1

                                              Now sct is packaged for void linux as well, thanks to @duncaen. So it should be equally easy to get as the port.

                                            1. 3

                                              Fixed in 2.5.4. This is the reason for the libressl erratum that was released yesterday for OpenBSD 6.1-stable. The github thread on libressl-portable may be of interest.

                                              1. 1

                                                First time I’ve used it and syspatch worked perfectly. Rather nicely, openup now uses it too.

                                                1. 6

                                                  A few more biographical details are in the Cantor biography of the MacTutor History of Mathematics archive of St Andrews, Scotland. Their bios generally have plenty of good references and are my go-to place for quickly looking up the biography of a mathematician.

                                                  Wittgenstein’s attacks on set theory as “utter nonsense” and worse are nicely summarized in the Stanford Encyclopedia of Philosophy.

                                                  I’m a bit disappointed to see the well-known “disease from which one has discovered” quote mentioned as “usually attributed to Poincaré” without further comment. There’s little doubt that Poincaré never said that. See Jeremy Gray, Did Poincaré say “set theory is a disease”?, Math. Intelligencer 13 (1991), no. 1, 19–22 (paywalled) for details.

                                                  1. 1

                                                    Remove “CVS tips” section from the web site. This forbidden knowledge is now forbidden.

                                                    Why?

                                                    1. 10

                                                      No deeper reason. I removed them as part of the still ongoing FAQ cleanup: getting rid of stuff that isn’t particularly useful anymore. It’s not as if those particular tips contained many hidden gems. Some of the bullet points are covered elswere, some are no longer relevant and, well, softdeps are softdeps…

                                                    1. 6

                                                      Isn’t that a repost on Ars of the original article that we already had somewhere here?

                                                      1. 13

                                                        This post originally appeared on his blog and is re-printed with his permission.

                                                        It’s only slightly edited and was discussed here: https://lobste.rs/s/r2kckg/im_giving_up_on_pgp

                                                        1. 5

                                                          I suspected it would be discussed here, I’m a new user and relied on it being reported as a duplicate URL on submission. I never thought of checking whether it was a “reprint” from another source.