1. 7

    How can you claim with a straight face that Go is better at concurrency than Java and C# when Go only has green threads, and no user-level control whatsoever on the execution model? That is particularly important for server-side applications where you might need to separate IO-bound tasks from CPU-bound tasks inside the same process.

    1. 22

      Go is excellent at writing server-side applications where you need to separate IO-bound and CPU-bound tasks in the same process. The runtime does it all for you, without requiring you to complect your application code with irrelevant details like thread execution models.

      1. 0

        complect your application code with irrelevant details like thread execution models.

        It’s very disingenous to dismiss threading control as “irrelevant”. If that would be the case, what’s this?

        In a web server that simultaneously does some non-blocking and blocking IO (files and the like) and then also some CPU bound stuff, how can the Go scheduler guarantee the web server can function independently and not be interrupted by the scheduler trying to find a thread that isn’t blocked? This is not a terribly complex thing to solve with user-level control on threads and thread pools, but it becomes quite daunting with only green threads and pre-emption.

        I’m not saying this can’t be done using green threads but it is difficult. GIven user control on threading you can implement your own runtime for fibers and continuations, but you can’t do that if you only have access to green threads!

        1. 1

          I don’t see how the linked issue is relevant. It’s about how Linux does not support non-blocking file I/O, so Go needs a kernel-level thread for each goroutine with a blocking file I/O operation. It’s exactly the same thing in Java and C#: If you want to run tons of file I/O in parallel, you will need tons of kernel-level threads.

    1. 4

      What does BC mean here?

      There are two big, substantial schools of thought in the PHP world. The first likes PHP roughly the way it is - dynamic, with strong BC bias and emphasis on simplicity

      1. 5

        “Backwards compatibility” would be my guess.

      1. 3

        Ah, from the days when we believed in sufficiently smart compilers. :)

        1. 4

          Putting our trust in sufficiently smart processors hasn’t exactly gone well either to be fair.

          1. 2

            I think the bigger issue here is that software is usually compiled once per ISA and not per processor, so the compiler never gets the chance to be very smart.

          1. 1

            Within Google, we have a growing range of needs…

            Something smells fishy. And Fuchsia.

            1. 6

              There’s a reply by someone on the Fuchsia team in the email thread - doesn’t look like it’s created with Fuchsia in mind so far.

              1. 2

                No kidding, since they aren’t even planning to support aarch64 in the initial implementation.

            1. 4

              It warms the heart to know that some people push back against adding syscalls just to be convenient for one set of programs. Progress needs to have reasons and be reasoned about.

              Are minimal syscall OSes akin to RISC?

              1. 5

                Are minimal syscall OSes akin to RISC?

                Microkernels have minimal functionality and thus also very few system calls. In fact, some microkernels only have a single system call for inter-process communication.

                I’m not sure whether it’s useful to reduce the number of system calls in a big monolithic kernel. I think it might lead to a complex system call interface with system calls that perform multiple (possibly unrelated) functions. This is already reality, for example with the ioctl system call in Linux that is used for lots of very different tasks.

                1. 3

                  RISC no longer has anything to do with a reduced instruction count, but instead reduced instruction complexity.

                  1. 1

                    Somewhat relevant to the orthogonality of instruction count/complexity: https://alastairreid.github.io/papers/sve-ieee-micro-2017.pdf

                1. 1

                  The built-in PEG parser is neat, but for such simple parsing tasks, a regular expression seems to be easier to write to me.

                  I’m also wondering: Why is (import sh) necessary? Isn’t it implied that you want the shell functions by running janetsh?

                  1. 1

                    The built-in PEG parser is neat, but for such simple parsing tasks, a regular expression seems to be easier to write to me.

                    When the parsing task get’s more complicated I think the PEG module will scale better. This post is just an educational demonstration intended to be understandable.

                    I’m also wondering: Why is (import sh) necessary? Isn’t it implied that you want the shell functions by running janetsh?

                    I agree, I may fix this in the future. It is just an implementation detail/limitation currently.

                    1. 1

                      When the parsing task get’s more complicated I think the PEG module will scale better. This post is just an educational demonstration intended to be understandable.

                      Yeah, alright. I was mostly writing that because Janet doesn’t appear to support regex (yet?) and I’m wondering whether that’s an intentional omission to make people use PEG.

                      1. 1

                        We were discussing adding regex via a non core but possibly official library.

                        Actually in janetsh you could also pipe the output of ls-remote to grep too.

                  1. 8

                    This paper is in the recent trend of lightweight academic articles based on sneering at things that work. The dismissal of Linux clone is particularly bad:

                    This syscall underlies all process and thread creation on Linux. Like Plan 9’s rfork() which preceded it, it takes separate flags controlling the child’s kernel state: address space, file descriptor table, namespaces, etc. This avoids one problem of fork: that its behaviour is implicit or undefined for many abstractions. However, for each resource there are two options: either share the resource between parent and child, or else copy it. As a result, clone suffers most of the same problems as fork (§4–5).

                    No it doesn’t - precisely because you can select not to duplicate. I also like this

                    The performance advantage of the shared initial state created by fork is less relevant when most concurrency is handled by threads, and modern operating systems deduplicate memory. Finally, with fork, all processes share the same address-space layout and are vulnerable to Blind ROP attacks

                    For sure, let’s use threads in the exact same address space to avoid ROP attacks.

                    The most interesting thing in the paper was that fork takes 1/2 millisecond on the machines they measured. Too slow by 10x.

                    1. 6

                      I agree that there was a very negative tone in the paper, and I found it quite off-putting and unprofessional. Among the scathingly negative phrases that caught my attention:

                      • “fork is a terrible abstraction”
                      • “fork is an anachronism”
                      • “fork is hostile… breaking everything…”
                      • “fork began as a hack”
                      • “we illustrate the havoc fork wreaks”
                      • “this is a deceptive myth”
                      • “multi-threaded programs that fork are plagued with bugs”
                      • “it is hard to imagine a new proposed syscall with these properties being accepted by any sane kernel maintainer”
                      • “caused fork semantics to subvert the OS design”
                      • “get the fork out of my OS”
                      1. 2

                        I hate to say it, but all those statements are either introductions to claims or much milder if you don’t cut off the whole sentence. I won’t go through all, but just some examples:

                        • “We catalog the ways in which fork is a terrible abstraction for the modern programmer to use, describe how it compromises OS implementations, and propose alternatives.”

                          • If you believe something is terrible, it’s fine to say it like that.
                        • “fork began as a hack”

                          • And the chapter that is called like that describes exactly that: how fork was ad-hoc implemented, using quotes by Richie himself. It’s a hack and not structured innovation implemented with farsight. That’s okay, but that’s what we call a hack.
                        • “fork is hostile to user-mode implementation of OS functionality, breaking everything from buffered IO to kernel-bypass networking. Perhaps most problematically, fork doesn’t compose—every layer of a system from the kernel to the smallest user-mode library must support it.”

                          • A technology or API being “hostile” to uses is standard terminology. And they go on to explain why they believe it to be hostile.
                        • “At first glance, fork still seems simple. We argue that this is a deceptive myth, and that fork’s effects cause modern applications more harm than good.”

                          • It is indeed the case that fork has a lot of rules to maintain around it and they are also right that fork is still tought as simple.
                        • “We illustrate the havoc fork wreaks on OS implementations using our experiences with prior research systems (§5). Fork limits the ability of OS researchers and developers to innovate because any new abstraction must be special-cased for it.”

                          • They rightfully claim that every abstraction needs to take into account that at any point in time, the process could be forked and the whole machinery is doubled. If something impacts almost all research in a field, strong wording is probably okay. “wreak havoc” is also not unheard of in professional settings.

                        Like, I’m fine with you no agreeing with their reading, but unprofessional, I would not agree with.

                        1. 2

                          Like, I’m fine with you no agreeing with their reading, but unprofessional, I would not agree with.

                          You omitted the most salacious phrase, “get the fork out of my OS”. This sort of wording has no place in a professional paper.

                          The remaining phrases I would categorize as more scathing than salacious. This wording is appropriate for a newsgroup or mailing list, but in an academic paper, these statements are hyperbolic, and could be replaced by more objective phrasing, without sacrificing meaning or impact.

                          1. 2

                            I took those phrases as tongue-in-cheek, but can agree that if you’re writing an article which may inspire strong opinions perhaps you should be mindful of how others might read it. I wouldn’t go as far as saying it’s unprofessional, however.

                            You omitted the most salacious phrase, “get the fork out of my OS”. This sort of wording has no place in a professional paper.

                            This is a good pun and I will defend to the death the authors’ right to use it in any context :)

                      2. 2

                        As a result, clone suffers most of the same problems as fork (§4–5).

                        No it doesn’t - precisely because you can select not to duplicate.

                        The only issue I see that clone clearly solves is the security problem from share-everything. How does it help with multithreading, performance (or ease-of-use in vfork mode), picoprocesses, heterogeneous hardware?

                        For sure, let’s use threads in the exact same address space to avoid ROP attacks.

                        I think the argument here is more like this:

                        • If you need it to be fast, use threads (most people already do this)
                        • If you need it to be secure, use a proper new process instead of forking from a prototype

                        The authors also argue that there might be better Copy-on-Write primitives than fork.

                        1. 1

                          How does it help with multithreading, performance

                          It’s pretty fast - especially compared to actually existing alternatives.

                          picoprocesses, heterogeneous hardware

                          It’s not intended to address those problems.

                          1. 1

                            It’s pretty fast - especially compared to actually existing alternatives.

                            In the paper, they show that posix_spawn is consistently faster at creating processes than fork, so I’m not sure what you mean?

                            1. 1

                              Posix_spawn is vfork/exec.

                      1. 12

                        This looks like something that was written just so Microsoft can have something to reference when they for the umpteenth time have to explain why CreateProcessEx is slow as balls on their systems.

                        1. 4

                          I doubt MS commissioned a research paper to defend against one argument made about their OS.

                          Besides, the very view that process spawning should be very cheap is just a trade-off. Windows prioritizes apps designed around threads rather than processes. You can argue one approach is superior than the other, but it really reflects different priorities and ideas about how apps are structured.

                          1. 7

                            Besides, the very view that process spawning should be very cheap is just a trade-off. Windows prioritizes apps designed around threads rather than processes.

                            Windows was a VMS clone and iteration designed by the same guy, Dave Cutler. The VMS spawn command was a heavyweight one with a lot of control over what the resulting process would do. Parallelism was done by threading, maybe clustering. Unlike sebcats’ conspiracy, it’s most likely that the VMS lead that became Windows lead just reused his VMS approach for same reasons. That the remaining UNIX’s are doing non-forking, monolithic designs in highest-performance apps corroborates that he made the right choice in long term.

                            That is not to say we can’t improve on his choice carefully rewriting CreateProcessEx or using some new method in new apps. I’ve been off Windows for a while, though. I don’t know what they’re currently doing for highest-performance stuff. Some lean programmers are still using Win32 API based on blog articles and such.

                            1. 3

                              That the remaining UNIX’s are doing non-forking, monolithic designs in highest-performance apps corroborates that he made the right choice in long term.

                              Assuming that you believe the highest performing apps should be the benchmark for how everyone writes everything – which I don’t agree with. Simplicity and clarity are more important for all but a small number of programs.

                              1. 3

                                I think that’s important, too. Forking fails that criteria as not being most simple or clear method to do parallelism or concurrency. It would loose to things like Active Oberon, Eiffel’s SCOOP, Cilk for data-parallel, and Erlang. Especially if you add safety or error handling which makes code more complex. If heterogenous environments (multicore to clusters), Chapel is way more readable than C, forks or MPI.

                              2. 4

                                despite all the bloat in Linux, process creation in Linux is a lot faster than process creation in Windows. Rob Pike was right, using threads is generally an indication that process creation and ipc is too slow and inconvenient and the answer is to fix the OS. [ edited to maintain “no more than one ‘and yet’ per day” rule]

                                1. 3

                                  Which led to faster designs than UNIX before and right after UNIX by using approaches that started with a better, concurrency-enabled language doing threads and function calls instead of processes and IPC. Funny you quote Rob Pike given his approach was more like Hansen (1960’s-1970’s) and Wirth’s (1980’s). On a language level, he also said a major inspiration for Go was his experience with Oberon-2 workstation doing rapid, safe coding. You including Rob Pike just corroborates my opinion of UNIX/C being inferior and/or outdated design from different direction.

                                  Back to process creation, its speed doesn’t matter outside high availability setups. The processes usually get created once. All that matters is speed of handling the computations or I/O from that point on. Processes don’t seem to have much to do with that on modern systems. They prefer close to the metal with lots of hardware offloading. The designs try to bypass kernels whether Windows or UNIX-like. Looking back, both mainframe architectures and Amiga’s used hardware/software approach. So, their models proved better than Windows or UNIX in long term with mainframes surviving with continued updates. Last press release on System Z I saw claimed over 10 billion encrypted transactions a day or something. Cost a fortune, too.

                                  1. 1

                                    The threads/function call design is good for some things and bad for others. The memory protection of fork/exec is very important in preventing and detecting a wide range of bugs and forcing design simplication. Oberon was, like much of Wirth’s work, cool but too limited. I think Parnas’ “Software Jewels” was inspired by Oberon.

                                    As for performance, I think you are 100% wrong and have a funny idea of “better”. System Z is clearly optimized for certain tasks, but as you say, it costs a fortune. You should have to write JCL code for a month as penance.

                                    1. 2

                                      “The memory protection of fork/exec is very important in preventing and detecting a wide range of bugs and forcing design simplication.”

                                      Those programs had all kinds of security bugs. It also didn’t let you control privileges or resource usage. If security is goal, you’d do a design like VMS’s spawn which let you those things or maybe a capability-oriented design like AS/400. Unless you were on a PDP-11 aiming to maximize performance at cost of everything else. Then you might get C, fork, and the rest of UNIX.

                                      “As for performance, I think you are 100% wrong and have a funny idea of “better”. System Z is clearly optimized for certain tasks”

                                      The utilization numbers say I’m 100% right. It comes from I/O architecture, not just workloads. Mainframe designers did I/O differently than PC knowing mixing computation and I/O led to bad utilization. Even at CPU level, the two have different requirements. So, they used designs like Channel I/O that let computation run on compute-optimized CPU’s with I/O run by I/O programs on dedicated, lower-energy, cheaper CPU’s. Non-mainframes usually ditched that since cost was main driver of market. Even SMP took forever to reach commodity PC’s. The shared architecture had Windows, Mac, and UNIX systems getting piles of low-level interrupts, having one app’s I/O drag down other apps, and so on. The mainframe apps responded to higher-level events with high utilization and reliability while I/O coprocessors handled low-level details.

                                      Fast forward to today. Since that model was best, we’ve seen it ported to x86 servers where more stuff bypasses kernels and/or is offloaded to dedicated hardware. Before that, it was used in HPC with the API’s splitting things between CPU’s and hardware/firmware (esp high-performance networking). We’ve seen the software side show up with stuff like Octeon processors offering a mix of RISC cores and hardware accelerators for dedicated, networking apps. Inline-Media Encryptors, RAID, and Netezza did it for storage. Ganssle also told me this design also shows up in some embedded products where the control logic runs on one core but another cheaper, lower-energy core handles I/O.

                                      Knockoffs of mainframe, I/O architecture have become the dominant model for high-performance, consistent I/O. That confirms my hypothesis. What we don’t see are more use of kernel calls per operation on simple hardware like UNIX’s original design. Folks are ditching that in mass in modern deployments since it’s a bottleneck. Whereas, mainframes just keep using and improving on their winning design by adding more accelerators. They’re expensive but their architecture isn’t in servers or embedded. Adding a J2 core for I/O on ancient node (180nm) costs about 3 cents a CPU. Intel added a backdoor, err management, CPU to all their CPU’s without any change in price. lowRISC has minion cores. I/O-focused coprocessors can be as cheap as market is willing to sell it to you. That’s not a technical, design problem. :)

                                      1. 2

                                        Since cores are so cheap now (we’re using 44-core machines, and I expect the count to keep going up in the future), why are we still using system calls to do IO? Put the kernel on its own core(s) and do IO with fast disruptor-style queues. That’s the design we seem to be converging toward, albeit in userspace.

                                        1. 1

                                          Absolutely true that the hugely expensive hardware I/O architecture in IBM mainframes work well for some loads if cost is not an issue. A Komatsu D575A is not better or worse than a D3K2 - just different and designed for different jobs.

                                          1. 2

                                            I just quoted you something costing 3 cents and something that’s $200 on eBay for Gbps accelerators. You only focused on the hugely, expensive mainframes. You must either agree the cheaper ones would work on desktops/servers or just have nothing else to counter with. Hell, Intel’s nodes could probably squeeze in a little core for each major subsystem: networking, disk, USB, and so on. Probably cost nothing in silicon for them.

                                            1. 1

                                              call intel.

                                              1. 2

                                                Nah. Enjoying watching them get what they deserve recently after their years of scheming bullshit. ;) Also, ISA vendors are likely to patent whatever I tell them about. Might talk to SiFive about it instead so we get open version.

                                  2. 1

                                    Windows NT was VMS-inspired, but I didn’t think Dave Cutler had any influence over Windows prior to that. Wasn’t CreateProcess available in the Win16 API?

                                    I suspect the lack of fork has more to do with Windows’s DOS roots, but NT probably would have gained fork if Microsoft had hired Unix developers instead of Cutler’s team.

                                    1. 1

                                      Windows NT was VMS-inspired, but I didn’t think Dave Cutler had any influence over Windows prior to that. Wasn’t CreateProcess available in the Win16 API?

                                      You could have me on its prior availability. I don’t know. The Windows API article on Wikipedia says it was introduced in Windows NT. That’s Wikipedia, though. I do know that Windows NT specifically cloned a lot from VMS via Russinovich’s article.

                                      1. 2

                                        I think was mistaken; looking at the Windows 95 SDK (https://winworldpc.com/product/windows-sdk-ddk/windows-95-ddk), CreateProcess was at the time in Win32 but not Win16. I guess that makes sense – what would CreateProcess do in a cooperatively multitasked environment?

                                        Most of what I know about NT development comes from the book Showstoppers.

                                2. 2

                                  Any source on performance issues with CreateProcessEx? A quick search didn’t yield anything interesting. Isn’t CreateProcessEx very similar to the posix_spawn API which the authors describe as the fast alternative to fork/exec in the paper?

                                  1. 2

                                    Alternatively they could just implement fork(). It’s nowhere near as hard as they’re making it out to be.

                                    1. 3

                                      fork is a bad API that significantly constrains OS implementation, so it is very understandable why Microsoft is reluctant to implement it.

                                      1. 3

                                        Meh, says you but if you already have a process abstraction it’s not really that much harder to clone a memory map and set the IP at the point after the syscall than it is to set up a new memory map and set the IP at the start address. I don’t buy that it “significantly” constrains the implementation.

                                        1. 8

                                          Effort isn’t the argument here, but the semantics of it. fork is a really blunt hammer. For example, Rust has no bindings to fork in stdlib for various reasons, one being that many types (e.g. file handles) have state attached to their memory representation not known to fork and that state becomes problematic in the face of fork. This is a problem also present in programs written in other programming languages, also C, but generally glossed over. It’s not a problem for “plain old data” memory, but once we’re talking about copying resource handles, stuff gets messy.

                                          1. 1

                                            Can you elaborate? FILE* responds to being forked just fine. What Rust file metadata needs to be cleaned up after fork()?

                                            1. 2

                                              Files are the simplest case. They are just racy in their raw form and you need to make sure everyone closes them properly. You can work against that by using RAII and unique pointers (or Ownership in Rust), but all those guarantees break on fork(). But even files with fork have a whole section in POSIX and improper handling may be undefined.

                                              It gets funnier if your resource is a lock and your memory allocator has locks.

                                              Sure, all this can be mitigated again, but that adds ton of complexity. My point is: fork may seem clean and simple, but in practice is extremely messy in that it does not set up good boundaries.

                                              1. 1

                                                Files are the simplest case. They are just racy in their raw form and you need to make sure everyone closes them properly. You can work against that by using RAII and unique pointers (or Ownership in Rust), but all those guarantees break on fork().

                                                There are no close() races on a cloned file descriptor across forked processes, nor are there file descriptor leaks if one forked process does not call close() on a cloned file descriptor.

                                                It gets funnier if your resource is a lock and your memory allocator has locks.

                                                How so? malloc() and fork() work fine together.

                                      2. 1

                                        But one of the main issue with fork that the authors describe is that it gets really slow for processes that use a large address space because it has to copy all the page tables. So I don’t see how implementing fork would help with performance?

                                        1. 0

                                          Nobody calls fork() in a loop. An efficiency argument isn’t relevant.

                                          1. 2

                                            In shell scripts usually most of the work is done by external programs. So shells use fork/exec a lot.

                                            1. 0

                                              But not “a lot” to the point where progress is stalled on fork/exec.

                                            2. 1

                                              I mean, the parent comment by @sebcat complains about process creation performance, and you suggest that implementing fork would help with that, so you do argue that it is efficient. Or am I reading your comment wrong?

                                              1. 1

                                                Ah okay, I see. I was under the impression that he was referring to the case where people complain about fork() performance on Windows because it is emulated using CreateProcessEx() (which he may not have been doing). My point was that If they implemented fork() in the kernel, they wouldn’t have to deal with those complaints (which are also misled since CreateProcessEx / fork() performance should never be relevant).

                                              2. 1

                                                A loop isn’t necessary for efficiency to be come relevant. Consider: most people abandoned CGI because (among other reasons) fork+exec for every HTTP request doesn’t scale well (and this was after most implementations of fork were already COW).

                                                1. 1

                                                  I can’t blame you, but that’s an excessively literal interpretation of my statement. By “in a loop,” I mean that the program is fork+exec bound, which happens on loaded web servers, and by “nobody” I mean “nobody competent.” It isn’t competent to run a high trafficked web server using the CGI model and expect it to perform well since process creation per request obviously won’t scale. CGI was originally intended for small scale sites.

                                            3. 1

                                              The bureaucracy at large, slow and old corporations partly explains this. This paper maybe took 6 months - 1 year. Adding fork() (with all the formalities + technical details + teams involved) would take 5-10 years IMHO.

                                              Much easier to just include it as “yet another app”, e.g. WSL:

                                              When a fork syscall is made on WSL, lxss.sys does some of the initial work to prepare for copying the process. It then calls internal NT APIs to create the process with the correct semantics and create a thread in the process with an identical register context. Finally, it does some additional work to complete copying the process and resumes the new process so it can begin executing.

                                              https://blogs.msdn.microsoft.com/wsl/2016/06/08/wsl-system-calls/

                                          1. 3

                                            Not a lot of technical content. The issue with modern zip archives is apparently new addressing modes to handle larger files that aren’t universally supported in unzip tools.

                                            A different issue I’ve come across with zip files on Linux is filename encoding. Some compression tools still don’t use UTF-8 and some of the decompression tools on Linux don’t handle the filename encoding at all and just dump whatever bytes they get into the file system. There doesn’t seem to be an easy way to fix the filenames then.

                                            1. 2

                                              The issue with modern zip archives is apparently new addressing modes to handle larger files that aren’t universally supported in unzip tools.

                                              It also seems to note that if you store your ZIP on a FAT32 disk it might get truncated and be called corrupt by mistake or the file that it decompresses might be larger than 2GB which leads to problems.

                                              I don’t know, it seemed mostly like a rant about stuff failing but the ZIP format can do pretty little about that. Plus, most of these old files are way smaller than 2GB (back in the day your HDD was smaller than 2GB to begin with), so it seems like quite an edge-case.

                                              1. 5

                                                I don’t think this piece is advocating for anyone to abandon the ZIP format. You could make similar complaints about literally any format, including plain text. If the article has a point, it’s to spread awareness of the challenges that archivists face, for the purpose of highlighting the importance of funding that work appropriately.

                                            1. 9

                                              It’s unclear to me how exactly the week value affects time and date calculations.

                                              Of the “normal” ways to represent time using a single number, the usual are to use “seconds since an epoch” (Unix time) or “fractional days since an epoch” (Julian date).

                                              I’m imagining that the GPS system would represent the date as “Weekday X of Week Y (in “week epoch” Z)” which raises all sorts of weird casting issues when you want to translate it into a human-readable date. I mean, conceptually it’s the same as the methods outlined above, it just feels a sui generis.

                                              1. 16

                                                I found this overview which gives a good description. Apparently, it’s week number plus seconds since week start, and many devices used to just hard code the current epoch.

                                                1. 4

                                                  Perfect, just the info I was looking for.

                                                  Edit interestingly, the issue doesn’t seem to be with GPS per se, instead it’s part of the protocol (NMEA) that the satellites use to talk to receivers.

                                                  Edit edit I am an idiot, I will read the entire document before saying anything.

                                              1. 4

                                                I think, these days, with hardware for some algorithms in recent CPUs and other algos designed to work very well with the vector unit, crypto-primitive-based PRNGs often end up good candidates even when you don’t need cryptographic security.

                                                If you check out https://bench.cr.yp.to/results-stream.html, for example, ChaCha20/8 runs on a recent-ish Core i7 at 0.28 cycles/byte, and AES-128 counter mode at 0.65 cycles/byte. Using a well-studied crypto primitive mostly dispenses with uncertainties about whether the output is ‘random enough’; any bias big enough to cause practical problems in your non-cryptographic application would lead to a blockbuster cryptography paper to say the least.

                                                Note that the linked page, though it’s for a non-cryptographic algorithm, treats unpredictable as good-to-have–search for “Predictable & Insecure”.

                                                1. 4

                                                  I agree.

                                                  PCG does provide “jumpahead” (moving to state after N steps without generating intermediate states), which is useful for some applications and is usually not provided by CSPRNG.

                                                  1. 2

                                                    AES and ChaCha/Salsa both generate blocks from a key + counter value (+ nonce for ChaCha/Salsa), so with them you can jump around in the stream by changing the counter. There are other stream ciphers don’t work that way, though.

                                                  2. 2

                                                    ChaCha20/8 runs on a recent-ish Core i7 at 0.28 cycles/byte, and AES-128 counter mode at 0.65 cycles/byte

                                                    But only if you’re encrypting rather large buffers at once, which is not what you usually want from a random number generator. The results of the 8 Byte column are more relevant, I think.

                                                    1. 1

                                                      Right tradeoff is probably somewhere in between 8 bytes and the “long messages” number I quoted (which, right, is an ideal). If you know up front you’re going need a lot of randomness, you might as well generate little chunks at once and have random() just read the next bytes out until the buffer is empty. That has its own costs (cache effect is probably the biggest), but GBs/s of noise with at most KBs of extra memory use still seems achievable.

                                                      If you don’t like much memory use ChaCha is probably the better fit: no key-schedule step or expanded key to store, and it can use SIMD parallelism within a block or two whereas I think Intel’s AES implementation would rather you pipeline work on several blocks in parallel.

                                                  1. 10

                                                    For modern Intel processors, the instruction of choice for sleeping is actually MWAIT rather than HLT. MWAIT allows selecting deeper sleep states and can monitor memory modifications for wakeup (in addition to interrupts). That monitoring feature is pretty cool for coordination across multiple processors, but unfortunately only available to kernel-mode code…

                                                    1. 2

                                                      On the upside, some of the earlier models of the XPS 13 I tested had a tendency to produce a high-pitched whine in some situations. If you read users’ complaints around the Web, you’ll get a mix of theories, the most likely being coil whine. Whatever it was, I have not noticed it with this model.

                                                      As someone who has owned a few of these from the Haswell version up through the Kaby Lake version, and experienced the coil whine on every one[0], I really hope they solved that issue.

                                                      1. Sometimes it took a few weeks before it showed up, but once it does it’s almost impossible for me to not hear.
                                                      1. 1

                                                        I have the Kaby Lake version as well. For me, the coil whine got significantly better with later firmware versions.

                                                        1. 1

                                                          Coil whine, or power supply whine seems to be a problem with modern laptops. Apple has had similar problems too - a colleague had to send their new top-of-the-line Macbook straight back because the whine was so loud.

                                                          1. 1

                                                            Coil whine is not an easily fixable problem (tl;dw the whiny kind of inductors is the one that’s electrically superior)

                                                            1. 1

                                                              I’ve got the Skylake version and indeed this is a problem. Besides, this laptop had so many issues over the years I’m actually wondering if I’m still using due to Stockholm Syndrome or did it get better over the years…

                                                              Personally the biggest annoyance is extremely slow boot, can take up to 7 seconds until UEFI app (OS bootloader) kicks in (if using TB16 dock +3s as it cannot start until the dock “starts”). I don’t remember this kind of slowness in my 2 year older than Skylake XPS 12.

                                                              1. 1

                                                                Does it whine always or on AC power? I have a chrome book that occasionally gets loud enough to annoy me, but only plugged in. So I just unplug it (and hope I remember to plug back in when I’m done).

                                                              1. 4

                                                                The tl:Dr is: use Wayland because X just wasn’t designed for this.

                                                                1. 3

                                                                  If all monitors have the same scale, X can look fine. But it absolutely cannot handle both low-DPI and high-DPI monitors at the same time.

                                                                  1. 1

                                                                    Although this does work fine with Qt5.

                                                                    1. 1

                                                                      It can, using a specialized xrandr configuration, and it looks great. The only problem I had (which stopped me from using it) is that there’s a bug in the Intel driver that makes the cursor on the laptop monitor flicker, which is more annoying than you’d think.

                                                                  1. 2

                                                                    I’m using entr for this.

                                                                    1. 4

                                                                      The most concerning part is DigitalOcean will shut off your server and then contact you second. There should be at least a day given to respond before your service gets shut off.

                                                                      1. 3

                                                                        On the other hand, if your server actually got hacked due to some security vulnerability, you might be happy they blocked it immediately. It would also potentially stop spreading.

                                                                        1. 2

                                                                          They prob. do that for a bigger customer.

                                                                          1. 2

                                                                            I suspect that anti-phishing policies are derived from safe harbor policies at most web hosting companies (since they both involve shutting down or making inaccessible other people’s files or services). DMCA takedowns are performed immediately & then investigated (or never investigated, more likely) in order to avoid liability under safe harbor provisions, since there’s a time limit of either 24 or 48 hours (I’ve forgotten which).

                                                                            If the author actually was a professional spammer/phisher, and was dumb enough to be paying for his own hosting from a place with anti-phishing policies, then he could use the 24 hours warning to register new accounts & move his operation, so even if you want to take the position that DigitalOcean’s primary goal is to be proactive against phishers / protect users rather than to cover their ass, there’s a case to be made.

                                                                            Certainly, under these circumstances the behavior is user- (and customer-) hostile, but that’s business: as long as you’re sure they’ll pay you, hostility doesn’t affect the bottom line, and you can feel free to alienate customers in direct proportion to other kinds of risk. Running a public IPFS gateway on a rented machine is an unusual behavior (of dubious utility, since running a private gateway is so easy & pure-JS gateways that run in the browser exist for those who can’t), & it’s unlikely that DigitalOcean is going to adapt their policies to support any kind of open gateway or proxy.

                                                                          1. 3

                                                                            It’s disappointing that there are still no extensions for mobile Chrome.

                                                                            1. 5

                                                                              Then uBlock Origin would be there, that is undesirable, I would guess.

                                                                              1. 3

                                                                                That’s why using Firefox on mobile is a no-brainer.

                                                                              1. 3

                                                                                I would love to have a feedback post, three years later. I don’t really know the status of Neovim right now

                                                                                1. 12

                                                                                  All of the points made in the post mentioned are still true.

                                                                                  Neovim is still developed actively and the community is stronger than ever. You can see the latest releases with notes here: https://github.com/neovim/neovim/releases

                                                                                  Vim’s BDFL ultimately caved and released his own async feature that is incompatible with Neovim’s design that has been in use by various cross-compatible plugins for years (no actual reason was provided for choosing incompatibility despite much pleading from community members). Some terminal support has also been added to recent Vim. IMO both implementations are inferior to Neovim’s, but that doesn’t matter much for end-users.

                                                                                  There are still many additional features in Neovim that haven’t been begrudgingly ported to Vim.

                                                                                  At this point, I choose to use Neovim not because of the better codebase and modern features and saner defaults, but because of the difference in how the projects are maintained and directed.

                                                                                  1. 20

                                                                                    Vim’s BDFL ultimately caved and released his own async feature

                                                                                    No, he didn’t. He didn’t cave. He was working on async, for a long time, with the goal of producing an async feature that actually fit in with the rest of Vim’s API and the rest of VimL, which he did. Did he probably work on it more and more quickly due to NeoVim? Sure. Did he only work on it because of pressure as you imply? No.

                                                                                    that is incompatible with Neovim’s design that has been in use by various cross-compatible plugins for years (no actual reason was provided for choosing incompatibility despite much pleading from community members).

                                                                                    NeoVim is incompatible with vim, not the other way around.

                                                                                    Some terminal support has also been added to recent Vim. IMO both implementations are inferior to Neovim’s, but that doesn’t matter much for end-users.

                                                                                    Async in vim fits in with the rest of vim much better than NeoVim’s async API would have fit in with vim.

                                                                                    There are still many additional features in Neovim that haven’t been begrudgingly ported to Vim.

                                                                                    The whole point of NeoVim is to remove features that they don’t personally use because they don’t think they’re important. There are a lot of Vim features not in NeoVim.

                                                                                    At this point, I choose to use Neovim not because of the better codebase and modern features and saner defaults, but because of the difference in how the projects are maintained and directed.

                                                                                    Vim is stable, reliable and backwards-compatible. I don’t fear that in the next release, a niche feature I use will be removed because ‘who uses that feature lolz?’, like I would with neovim.

                                                                                    1. 10

                                                                                      No, he didn’t. He didn’t cave. He was working on async, for a long time, with the goal of producing an async feature that actually fit in with the rest of Vim’s API and the rest of VimL, which he did.

                                                                                      Where did you get this narrative from? The original post provides links to the discussions of Thiago’s and Geoff’s respective attempts at this. I don’t see what you described at all.

                                                                                      Can you link to any discussion about Bram working on async for a long time before?

                                                                                      NeoVim is incompatible with vim, not the other way around.

                                                                                      Huh? Vim didn’t have this feature at all, a bunch of plugins adopted Neovim’s design, Vim broke compatibility with those plugins by releasing an incompatible implementation of the same thing, forcing plugin maintainers to build separate compatibility pipelines for Vim. Some examples of this is fatih’s vim-go (some related tweets: https://twitter.com/fatih/status/793414447113048064) and Shougo’s plugins.

                                                                                      I get the whole “Vim was here first!” thing this is about the plugin ecosystem.

                                                                                      Async in vim fits in with the rest of vim much better than NeoVim’s async API would have fit in with vim.

                                                                                      How’s that?

                                                                                      Here is the discussion of the patch to add vim async from Bram, where he is rudely dismissive of Thiago’s plea for a compatible design (no technical reasons given): https://groups.google.com/forum/#!topic/vim_dev/_SbMTGshzVc/discussion

                                                                                      The whole point of NeoVim is to remove features that they don’t personally use because they don’t think they’re important. There are a lot of Vim features not in NeoVim.

                                                                                      What are some examples of important features or features you care about that have been removed from Neovim?

                                                                                      The whole point of Neovim (according to the landing page itself: https://neovim.io/) is to migrate to modern tooling and features. The goal is to remain backwards-compatible with original vim.

                                                                                      Vim is stable, reliable and backwards-compatible. I don’t fear that in the next release, a niche feature I use will be removed because ‘who uses that feature lolz?’, like I would with neovim.

                                                                                      Do you actually believe this or are you being sarcastic to make a point? I honestly can’t relate to this.

                                                                                      1. 3

                                                                                        The vim vs. neovim debate is often framed a bit in the style of Bram vs. Thiago, and the accusation against Thiago is typically that he was too impatient or should not have forked vim in the first place when Bram did not merge Thiago’s patches. I have the feeling that your argumentation falls into similar lines and I don’lt like to view this exclusively as Bram vs. Thiago, because I both value Bram’s and Thiago’s contributions to the open source domain, and I think so far vim has ultimatetively profitted from the forking.

                                                                                        I think there are two essential freedoms in open source,

                                                                                        • the freedom of an open source maintainer not to accept / merge contribution,
                                                                                        • in the very essence of open source, that users have the right to fork, when they feel that the maintainers are not accepting their contributions (preferably they try to make a contribution to the source project first).

                                                                                        Both of this happend when neovim was forked. There is no “offender” in any way. Thus, all questions on API compatibility following the split cannot be lead from the perspective of a renegade fork (nvim) and an authorative true editor (vim).

                                                                                        1. 8

                                                                                          It was absolutely 100% justified of Thiago to fork vim when Bram wouldn’t merge his patches. What’s the point of open source software if you can’t do this?

                                                                                          1. 3

                                                                                            And as a follow up my more subjective view:

                                                                                            I personally use neovim on my development machines, and vim on most of the servers I ssh into. The discrepancy for the casual usage is minimal, on my development machines I feel that neovim is a mature and very usable product that I can trust. For some reason, vim’s time-tested code-base with pre-ANSI style C headers and no unit tests is one I don’t put as much faith in, when it comes to introducing changes.

                                                                                        2. 4

                                                                                          @shazow’s reasoning and this post are what I link people to in https://jacky.wtf/weblog/moving-to-neovim/. Like for a solid release pipeline and actual digestible explanations as to what’s happening with the project, NeoVim trumps Vim every time.

                                                                                      1. 2

                                                                                        When accessing more bytes per cache line, the code is running more instructions. Each of these instructions depend on the previous as they’re all writing to the same counter. So I assume it’s actually just slower because it’s more sequential code, not because L1 cache reads are expensive.

                                                                                        A better benchmark would compare the sizes you can access with a single instruction, or would have multiple instructions that can be run in parallel.

                                                                                        1. 2

                                                                                          Deployed via PDF font installer, that is. It seems the biggest issue here is that the installer never checks the signatures of the font installers it downloads, but the article never suggests that.