1. 3

    Flying back from the ESSLLI summer school in Sofia, Bulgaria, where we taught a course about neural dependency parsing:

    http://esslli2018.folli.info/neural-dependency-parsing-of-morphologically-rich-languages/

    1. 4

      This is really interesting to get an idea of how people are taking advantage of BSD! I now have a much nicer idea of why people are going to it (and am a bit tempted myself). That feeling of having to go through ports and simply not having 1st-class support for some software seems… rough for desktop usage though

      1. 3
        1. 1

          I mean “someone talks to me about an application and I’m interested in trying it out on my system”?

          I feel like the link to the CVE database is a bit of an unwarranted snipe here. I’m not talking too much about security updates, just “someone released some software and didn’t bother to confirm BSD support so now I’m going to need to figure out which ways this software will not work”.

          To be honest I don’t really think that having all userland software come in via OS-maintained package managers is a great idea in the first place (do I really need OS maintainers looking after anki?). I’m fine downloading binaries off the net. Just nicer if they have out of the box support for stuff. I’m not blaming the BSDs for this (it’s more the software writer’s fault), just that it’s my impression that this becomes a bit of an issue if you try out a lot of less used software.

          1. 4

            As an engineer that uses and works on a minority share operating system, I don’t really think it’s reasonable to expect chiefly volunteer projects to ship binaries for my platform in a way that fits well with the OS itself. It would be great if they were willing to test on our platform, even just occasionally, but I understand why they don’t.

            Given this, it seems more likely to expect a good experience from binaries provided by somebody with a vested interest in quality on the OS in question – which is why we end up with a distribution model.

            1. 2

              Yep, this makes a lot of sense.

              I’m getting more and more partial to software relying on their host language’s package manager recently. It’s pretty nice for a Python binary to basically always work so long as you got pip running properly on your system, plus you get all the nice advantages of virtual environments and the like letting you more easily set things up. The biggest issue being around some trust issues in those ecosystems.

              Considering a lot of communities (not just OSes) are getting more and more involved in distribution questions, we might be getting closer to getting things to work out of the box for non-tricky cases.

              1. 8

                software relying on their host language’s package manager

                In general I’m not a fan. They all have problems. Many (most?) of them lack a notion of disconnected operation when they cannot reach their central Internet-connected registry. There is often no complete tracking of all files installed, which makes it difficult to completely remove a package later. Some of the language runtimes make it difficult to use packages installed in non-default directory trees, which is one way you might have hoped to work around the difficulty of subsequent removal. These systems also generally conflate the build machine with the target machine (i.e., the host on which the software will run) which tends to mean you’re not just installing a binary package but needing to build the software in-situ every time you install it.

                In practice, I do end up using these tools because there is often no alternative – but they do not bring me joy.

                Operating system package managers (dpkg/apt, rpm/yum, pkg_add/pkgin, IPS, etc) also have their problems. In contrast, though, these package managers tend to at least have some tools to manage the set of files that were installed for a particular package and to remove (or even just verify) them later. They also generally offer some first class way to install a set of a packages from archive files obtained via means other than direct access to a central repository.

                1. 3

                  For development I use the “central Internet-connected registry.”, for production I use DEB/RPM packages in a repository:

                  • forces you to limit the number of dependencies you use, otherwise too much work to package them all;
                  • force you to choose high quality dependencies that are easy to package or already packaged;
                  • makes sure every dependency is buildable from source (depending on language);
                  • have an “offline” copy of the dependencies, protect against “left-pad” issues;
                  • run unit tests of the dependencies during package build, great for QA!;
                  • have (PGP) signed packages that uses the distribution’s tools to verify.

                  There are probably more benefits that escape me at the moment :)

        2. 1

          That feeling of having to go through ports and simply not having 1st-class support for some software seems… rough for desktop usage though

          What kind of desktop software do you install from these non-OS sources?

          1. 2

            Linux is moving more and more towards Flatpak and Snap for (sandboxed) application distribution.

            1. 2

              I remember screwing around with Flathub on the command line in Fedora 27, but right now on Fedora 28, if you enable Flatpak in the Gnome Software Center thingy, it’s actually pretty seamless - type “Signal” in the application browser, and a Flatpak install link shows up.

              With this sort of UX improvements, I’m optimistic. I feel like Fedora is just going to get easier and easier to use.

        1. 3

          I’d never heard of “snap” before, and I’m not sure I understand what it’s trying to do. Is it just a lazy way to publish closed source software on Linux? I really don’t see why I’d choose a snap over using my regular package manager.

          1. 6

            Yes and no. Snaps are the last of many tries to have a single, uniform package format among a lot of different Linux distributions. They can be installed on *buntu, debian, arch, fedora, opensuse, …. They also offer built-in sandbox.

            You are correct that a lot of propritiatory software is using it to distribute packages because it saves some time, but that’s not the main goal of the format.

            1. 2

              Snaps are the last of many tries to have a single, uniform package format among a lot of different Linux distributions.

              Unfortunately, there are at least three competing standards (Snap, Flatpak, AppImage). However, Flatpak seems to be supported in more distributions than Snap:

              https://kamikazow.wordpress.com/2018/06/08/adoption-of-flatpak-vs-snap-2018-edition/

              Regardless of what one thinks of such formats, they have already lead to interesting phenomena. Flatpak is, for instance, quite popular among pirates who use it to pack Windows games with a custom Wine configuration:

              https://www.nag.co.za/2017/10/27/pirates-are-starting-to-package-windows-games-for-linux-better-than-the-developers/

              Then there is the Winepak project, which packages redistributable Windows software:

              https://winepak.org/

            2. 3
              1. 2

                Think docker for desktop apps. With all of the good and the bad it implies.

              1. 22

                After writing Go for 5 years, I’d recommend Rust for C developers. It’s more complicated than Go for sure, but also has more to offer. The lack of garbage collection and support of generics are definitely a plus compared to Go.

                Go is a better language for junior devs, but I wouldn’t call C programmers junior. They should be able to digest Rust’s complexity.

                1. 9

                  They should be able to digest Rust’s complexity.

                  Non trivial amount of C programmers are still doing C to avoid additional complexity. Not everyone wants a kitchen & sink programming language.

                  1. 6

                    Rust can definitely get overly complex if the developers show no constraint (i.e. type golf), but the control afforded by manual memory management makes up for it, IMHO. Unless it’s a one-run project, performance will eventually matter, and fixing bad allocation practices after the fact is a lot harder than doing it right from the beginning.

                    1. 1

                      Couldn’t they just start with a C-like subset of Rust adding from there to their arsenal what extra features they like? It’s what I was going to recommend to those trying it for safety-critical use since they likely know C.

                      1. 9

                        I think it’s rather difficult to write rust in a C like manner. This contrasts with go, where you can basically write C code and move the type declarations around and end up with somewhat unidiomatic but working go.

                        1. 3

                          I think C++ as a better C works because you still have libc besides the STL, etc. The Rust standard library uses generics, traits, etc. quite heavily and type parameters and lifetime parameters tend to percolate to downstream users.

                          Though I think a lot of value in Rust is in concepts that may initially add some complexity, such the borrow checker rules.

                          1. 3

                            The problem with C++ is its complexity at the language level. I have little hope of teams of people porting various tools for static analysis, verification, and refactoring to it that C and Java already have. Certifying compilers either. C itself is a rough language but smaller. The massive bandwagon behind it caused lots of tooling to be built, esp FOSS. So, I now push for low-level stuff either safer C or something that ties into C’s ecosystem.

                          2. 4

                            You could argue the same for C++ (start with C and add extra features). Complexity comes with the whole ecosystem from platform support (OS, arch), compiler complexity (and hence subtle difference in feature implementations) to the language itself (C++ templates, rust macros). It’s challenging to limit oneself to a very specific subset on a single person project, it’s exponentially harder for larger teams to agree on a subset and adhere to it. I guess I just want a safer C not a new C++ replacement which seems to be the target for newer languages (like D & Rust).

                            1. 4

                              It’s challenging to limit oneself to a very specific subset on a single person project, it’s exponentially harder for larger teams to agree on a subset and adhere to it.

                              I see your overall point. It could be tricky. It would probably stay niche. I will note that, in the C and Java worlds, there’s tools that check source code for compliance with coding standards. That could work for a Rust subset as well.

                              “I guess I just want a safer C not a new C++ replacement which seems to be the target for newer languages (like D & Rust).”

                              I can’t remember if I asked you what you thought about Cyclone. So, I’m curious about that plus what you or other C programmers would change about such a proposal.

                              I was thinking something like it with Rust’s affine types and/or reference counting when borrow-checking sucks too much with performance acceptable. Also, unsafe stuff if necessary with the module prefixed with that like Wirth would do. Some kind of module system or linking types to avoid linker errors, too. Seemless use of existing C libraries. Then, an interpreter or REPL for the productivity boost. Extracts to C to use its optimizing and certifying compilers. I’m unsure of what I’d default with on error handling and concurrency. First round at error handling might be error codes since I saw a design for statically checking their correct usage.

                              1. 3

                                I can’t remember if I asked you what you thought about Cyclone. So, I’m curious about that plus what you or other C programmers would change about such a proposal.

                                I looked at it in the past and it felt like a language built on top of C similar to what a checker tool with annotations would do. It felt geared too much towards research versus use and the site itself states:

                                Cyclone is no longer supported; the core research project has finished and the developers have moved on to other things. (Several of Cyclone’s ideas have made their way into Rust.) Cyclone’s code can be made to work with some effort, but it will not build out of the box on modern (64 bit) platforms).

                                However if I had to change Cyclone I would at least drop exceptions from it.

                                I am keeping an eye on zig and that’s closest to how I imagine a potentially successful C replacement - assuming it takes up enough community drive and gets some people developing interesting software with it.

                                That’s something Go had nailed down really well. The whole standard library (especially their crypto and http libs) being implemented from scratch in Go instead of being bindings were a strong value signal.

                                1. 2

                                  re dropping exceptions. Dropping exceptions makes sense. Is there another way of error handling that’s safer or better than C’s that you think might be adoptable in a new, C-like language?

                                  re Zig. It’s an interesting language. I’m watching it at a distance for ideas.

                                  re standard library of X in X. Yeah, I agree. I’ve been noticing that pattern with Myrddin, too. They’ve been doing a lot within the language despite how new it is.

                                  1. 4

                                    Dropping exceptions makes sense. Is there another way of error handling that’s safer or better than C’s that you think might be adoptable in a new, C-like language?

                                    Yes, I think Zig actually does that pretty well: https://andrewkelley.me/post/intro-to-zig.html#error-type

                                    edit: snippet from the zig homepage:

                                    A fresh take on error handling that resembles what well-written C error handling looks like, minus the boilerplate and verbosity.

                                    1. 2

                                      Thanks for the link and tips!

                        2. 7

                          Short build/edit/run cycles are appreciated by junior and senior developers alike. Go currently has superior compilation times.

                          1. 10

                            Junior and senior developers also enjoy language features such as map, reduce, filter, and generics. Not to mention deterministic memory allocation, soft realtime, forced error checking, zero-cost abstractions, and (of course) memory safety.

                            1. 3

                              Junior and senior developers also enjoy language features such as map, reduce, filter, and generics.

                              Those are great!

                              deterministic memory allocation, soft realtime, forced error checking, zero-cost abstractions, and (of course) memory safety.

                              Where are you finding juniors who care about this stuff? (no, really - I would like to know what kind of education got them there).

                              1. 8

                                I cared about those things, as a junior. I am not sure why juniors wouldn’t care, although I suppose it depends on what kind of software they’re interested in writing. It’s hard to get away with not caring, for a lot of things. Regarding education, I am self-taught, FWIW.

                              2. 1

                                Map, reduce and filter are easily implemented in Go. Managing memory manually, while keeping the GC running, is fully possible. Turning off the GC is also possible. Soft realtime is achievable, depending on your definition of soft realtime.

                                1. 1

                                  Map, reduce and filter are easily implemented in Go

                                  How? Type safe versions of these, that is, without interface{} and hacky codegen solutions?

                                  1. 1

                                    Here are typesafe examples for Map, Filter etc: https://gobyexample.com/collection-functions

                                    Implementing one Map function per type is often good enough. There is some duplication of code, but the required functionality is present. There are many theoretical needs that don’t always show up in practice.

                                    Also, using go generate (which comes with the compiler), generic versions are achievable too. For example like this: https://github.com/kulshekhar/fungen

                                    1. 9

                                      When people say “type safe map/filter/reduce/fold” or “map, reduce, filter, and generics” they are generally referring to the ability to define those functions in a way that is polymorphic, type safe, transparently handled by the compiler and doesn’t sacrifice runtime overhead compared to their monomorphic analogs.

                                      Whether you believe such facilities are useful or not is a completely different and orthogonal question. But no, they are certainly not achievable in Go and this is not a controversial claim. It is by design.

                                      1. 1

                                        Yes, I agree, Go does not have the combination of type safety and generics, unless you consider code generation.

                                        The implementation of generics in C++ also works by generating the code per required type.

                                        1. 5

                                          The implementation of generics in C++ also works by generating the code per required type.

                                          But they are not really comparable. In C++, when a library defines a generic type or function, it will work with any conforming data type. Since the Go compiler does not know about generics, with go generate one can only generate ‘monomorphized’ types for a set of predefined data types that are defined an upstream package. If you want different monomorphized types, you have to import the generic definitions and run go generate for your specific types.

                                          unless you consider code generation

                                          By that definition, any language is a generic language, there’s always Bourne shell/make/sed for code generation ;).

                                          1. 1

                                            That is true, and I agree that go does not have support for proper generics and that this can be a problem when creating libraries.

                                          2. 3

                                            That’s why I said “transparently handled by the compiler.” ;-)

                                            1. 0

                                              I see your point, but “go generate” is provided by the go compiler, by default. I guess it doesn’t qualify as transparent since you have to type “go generate” or place that command in a build file of some sort?

                                              1. 1

                                                Yes. And for the reasons mentioned by @iswrong.

                                                My larger point here really isn’t a technicality. My point is that communication is hard and not everyone spells out every point is precise detail, but it’s usually possible to infer the meaning based on context.

                                                1. -1

                                                  I think the even larger point is that for a wide range of applications, “proper” and “transparent” generics might not even be needed in the first place. It would help, yes, but the Go community currently thrives without it, with no lack of results to show for.

                                                  1. 1

                                                    I mean, I’ve written Go code nearly daily since before it was 1.0. I don’t need to argue with you about whether generics are “needed,” which is a pretty slimy way to phrase this.

                                                    Seems to me like you’re trying to pick a fight. I already said upthread that the description of generics is different from the desire for them.

                                                    1. -2

                                                      You were the first to change the subject to you and me instead of sticking to the topic at hand. Downvoting as troll.

                                2. 1

                                  By superior, I guess you meant shorter?

                                  1. 2

                                    Compiling a very large go project with a cold cache might take a minute (sub-second once the cache is warm).

                                    Compiling a fairly small rust app with a warm cache has taken me over a minute (I think it’s a little better than that now).

                                    1. 1

                                      Yes, and superior to Rust in that regard. Also the strict requirement to not have unused dependencies contributes to counteract dependency rot, for larger projects.

                                1. 23

                                  Isn’t ublock origin independent from ublock ?

                                  Aren’t they two vastly different tools ?

                                    1. 18

                                      I just found that uBlock Origin even blocks ublock.org by default ;):

                                      https://github.com/gorhill/uBlock/wiki/Badware-risks#ublockorg

                                      (Not that I disagree, ublock.org is a borderline scam.)

                                      1. 4

                                        I noticed that too when clicking on the link. It’s kinda funny. It’s hard to keep everything straight these days, but use, uBlock/AdBlock/AdBlockPlus all are a little shady in how they monetize their development, either by letting it white-list “non-intrusive” ads or by collecting usage data.

                                        As far as I know, the uBlock Origin project is currently uncompromising in this regard. Still, they all use public and community maintained blacklists/greylists.

                                  1. 23

                                    Nix is one of those tools where you don’t know what you aren’t getting until you get it. There are so many things wrong with this post, but I only know that because I spent weeks wrestling with lots of those issues myself.

                                    You basically need to read all the nix pills (https://nixos.org/nixos/nix-pills/), the nix manual, the nixpkgs manual and the nixos manual in a loop gradually filling in what is going on… which takes a long time.

                                    Nix is very confusing at first, but enables things that you would not have thought possible once you know what you are doing. The core people don’t seem to evangelize much because it is just one of those tools that solved their problems so well, they don’t have to care about the outside world anymore.

                                    I use nixos for my laptop, desktop and a few servers, have all my machines config under version control and can roll the machines back to any version whenever I want, remote administer them, build an install on one computer, test it in a VM and then ship it with a single command to another machine. I won’t go back to another OS despite there being room for improvement, because no other OS comes close in terms of what you can do (my path has been windows -> ubuntu -> arch linux -> freebsd -> openbsd -> nixos).

                                    1. 18

                                      I use NixOS on everything and completely agree. It’s a massive investment. It was worth it for me, but it shouldn’t have to be a massive investment. Need better tooling and docs.

                                      1. 5

                                        Yeah, there are lots of things I wish I could explain, but the explanations take a large investment. Take for example the complaint about making a new language instead of using something existing… It seems sensible on the surface, until you understand deeply enough to know why laziness is needed, and features like the pervasive use of interpolation to generate build scripts… Once you understand those, you know why a new language was made.

                                        The lack of tooling IS a valid complaint, and the fact the language isn’t statically typed could also be a valid complaint, but the community is growing despite all those issues, which is a good sign.

                                        1. 6

                                          I’m hoping https://github.com/haskell-nix/hnix will help with that point, and the tooling.

                                      2. 6

                                        You basically need to read all the nix pills (https://nixos.org/nixos/nix-pills/), the nix manual, the nixpkgs manual and the nixos manual in a loop gradually filling in what is going on… which takes a long time.

                                        I’ve tried reading all of this but I found it all horribly confusing and frustrating — until I read the original thesis on it, which I still think is (perhaps surprisingly) still the best resource for learning how nix works. It’s still a pretty big investment to read, but imho it’s at the very least a much less frustrating experience than bouncing from docs to docs.

                                        (I wonder if the same is true of the NixOS paper?)

                                        1. 3

                                          How do you manage secrets in configuration files? Passwords, ssh keys, tls certs and so on. If you put them into a nix-store they must be world-readable, right?

                                          One could put a reference to files outside the store in configuration files, but then you loose a bit of the determinism of NixOS and it’s not always easily possible with third-party software to load e.g. passwords from an external file anyways.

                                          Besides the learning curve, that was the single big problem which kept me from diving deeper into the nix ecosystem so far.

                                          1. 7

                                            You are right, no passwords should ever go in the nix store.

                                            The encryption key for my backup script is in a private root owned file I put under /secrets/ . This file is loaded in my cron job so the nix store simply references the secret but doesn’t contain it. This secret dir isn’t under version control, but is backed up with encrypted backups.

                                            Every daemon with secret config I have seen on nixos has a “password file” option that does the same thing.

                                            1. 3

                                              How do you manage secrets in configuration files?

                                              For my desktop machine I use pass with a hardware key. E.g. nix (home-manager) generates an .mbsyncrc with

                                              PassCmd "pass Mail/magnolia"
                                              

                                              For remote machines, I use nixop’s method for keeping keys out of the store:

                                              https://nixos.org/nixops/manual/#idm140737318276736

                                            2. 1

                                              Nix is one of those tools where you don’t know what you aren’t getting until you get it. There are so many things wrong with this post

                                              I have to disagree, but not with the second sentence - I was sure as I wrote the post that it was full of misconceptions and probably outright errors. I wrote it in part to capture those in the hopes that someone can use them to improve the docs.

                                              But to disagree with the first sentence, I was keenly aware through the learning and writing that I was missing fundamental concepts and struggling to fill the gaps with pieces from other tools that didn’t quite fit. If there is indeed a whole ‘nother level of unknown unknowns, well, that’s pretty disheartening to me.

                                              1. 1

                                                I can’t speak for your experience, but that’s how it was for me anyway, on the plus side it also meant nix solved more problems I was having after I understood better. I even thought nix was over complicated to the point I started writing my own simpler package manager, only to find nix had solved problems I ran into before I knew what they were.

                                            1. 6

                                              OK but the tag line is asinine. As a regular user of a Linux distribution it is actually impossible for me to take the time to do a full analysis on every package I install to get work done.

                                              SOME level of trust has to be there or else the whole idea of a Linux distro can’t work.

                                              1. 10

                                                Well, AUR specifically isn’t part of the actual Arch distro. It’s no safer than the curl | bash invocations on github.

                                                1. 4

                                                  But it makes your wonder if there is no middle-ground between the AUR and the community repository. Have a Git{Hub,Lab,ea} repository where the community can do pull requests for new packages and updates, but the pull requests are reviewed by trusted users or developers. And then build the packages on trusted infrastructure.

                                                  1. 9

                                                    This is how the OpenBSD ports tree works. Anyone can send a new port or an update to the ports@ mailing list. It then gets tested & committed by developers.

                                                    In this specific instance, I think what hurt Arch here is too good tooling. The community developed a lot of automation tools that boil down third party package installs to pressing enter a bunch of times - even with all the warnings present people stopped reviewing the packages. If I recall correctly, the main point of AUR was to gather community packages then promote the popular ones (by votes) to trusted repositories - essentially the promotion to trusted repos lost meaning as everyone can install yaourt/pacaur or $pkgmgr du jour and just go on with their life.

                                                  2. 2

                                                    It’s no safer than the curl | bash invocations on github.

                                                    Highly disagree. Using the AUR without any supporting tools like pacaur, you’re cloning into a git repository to retrieve the PKGBUILD and supporting files, so you have the opportunity to view them. With pacaur, you’re shown the PKGBUILD at first install so you can make sure nothing’s malicious, and then you’re shown diffs when the package version updates. That’s MUCH better than curl | bash already.

                                                    1. 1

                                                      Also, while you shouldn’t rely on others to spot malicious code, the fact that the malicious modifications were spotted and reverted after about 9 hours shows that the AUR is subject to at least slightly more scrutiny than random scripts from github or elsewhere are.

                                                      Admittedly, it doesn’t sound like this particular attack was very sophisticated or well hidden.

                                                1. 6

                                                  Betteridge’s law of headlines: no.

                                                  A markov model is stateless, whereas RNNs can (in principle) learn to percolate information over arbitrary distances. This difference is seen in other tasks, such as source code generation, where RNNs produce syntactically (nearly) correct code with properly closed braces, comments, etc. Yoav Goldberg has made a nice markov model comparison to Kaparthy’s famous blog post, which clearly shows that markov models fail in such cases:

                                                  http://nbviewer.jupyter.org/gist/yoavg/d76121dfde2618422139

                                                  Percolating longer-distance information is also necessary in freer word order languages. There are many examples, but consider, for example, separable verb particles in Dutch.

                                                  `De spanning [loopt] na de uitschakeling van Duitsland op het WK in Rusland behoorlijk [op] deze week.’

                                                  The excitement [increases_verb] after the elimination of Germany at the WorldCup in Russia quite [increases_particle] this week.

                                                  Here, the particle op of the verb oplopen is separated from the finite verb due to v2 movement. Such cases are notoriously hard to handle with simpler models, due to the long distance dependency between the verb parts and, in this case, the possible misinterpretation of [op deze week] as a prepositional phrase. RNNs are generally more capable in modeling such cases (especially with bidirectional RNNs).

                                                  1. 2

                                                    PPP and Freshmeat. The kids today have no idea!

                                                    Generally not a huge fan of the style here, there are things you still can’t do as easily on “Linux” as you can on an Apple computer, like html design.

                                                    Could be that fewer people cared about home computing those 20 years ago, and Apple wasn’t big shit and smart phones didn’t exist, so more people were intrigued and at least tried a distro or two.

                                                    But try showing a regular Joe of today LibreOffice, and what it does to Word documents, and apologize by talking about how cool FOSS is as an ideology, and how there is no one “Linux”. See what kind of enlightenment that leads to.

                                                    1. 3

                                                      But try showing a regular Joe of today LibreOffice, and what it does to Word documents, and apologize by talking about how cool FOSS is as an ideology, and how there is no one “Linux”.

                                                      True. In the meanwhile, the vast majority of worldwide smartphone users have a small Linux machine in their pockets. If they have a home router or smart TV it is probably powered by Linux as well. Of course, they don’t even know it is Linux.

                                                      Many traditional Linux distributions and desktops have for a long time skated to where the puck was (trying to replicate Windows and/or macOS) rather than to where the puck was going (appliance-like computers, where the OS is just a technical detail). Plus some occasional cargo-culting (if we turn our desktop environment into a tablet-fest, the users will come).

                                                      I hope that the traditional Linux desktop community has learned from this and realized that their primary market consists of developers and power-users, and will stop butchering the Linux workstation experience for simplicity. The masses of users will never come, that window of opportunity has closed after around 2007 when Vista was a flop and smartphones and tables were not widely used yet. Unfortunately, it seems like some environments still do things like removing menu bars [1] or system tray icons.

                                                      [1] The GNOME HIG literally states Menu bars increase the vertical footprint of an application’s user interface, introduce a large number of disclosure points, and function as a fixed set of inflexible options. For these reasons, header bars and header bar menus are generally recommended over menu bars,

                                                      1. 1

                                                        PPP?

                                                        1. 4

                                                          I presume that is Point-to-Point-Protocol, commonly used to provide TCP/IP networking over a dial-up modem link.

                                                          UPDATE: yeah the article confirms that.

                                                          1. 1

                                                            Ah thought it might have been something like that. Thanks for clearing that up :).

                                                      1. 10

                                                        You can get 75% of Prolog in SQL, along with some statistical functions that you can’t easily do in Prolog. That’s pretty good.

                                                        1. 2

                                                          75% of database Prolog with only limited support for unification. Prolog especially shines when you need unification, backtracking, meta-programming, etc.

                                                        1. 2

                                                          At home: inspired by the Arch and Nix discussion a few days ago, I decided to try Nix on macOS and NixOS on my workstation with root on encrypted ZFS.

                                                          Liking it so far and nixifying all the things. I use a very thin system configuration now and do most of the heavy lifting using home-manager.

                                                          I am not sure whether I’ll stay yet. I like the Slackware-like simplicity of Arch. On the other hand, working with Nix feels like you have superpowers, especially with nix-shell and direnv’s Nix support.

                                                          1. 3

                                                            Because I am a developer, one SSH client is mostly enough, and fascinating desktop doesn’t appeal to me.

                                                            Which doesn’t mean that Arch doesn’t have nice desktops :). I use GNOME (it supports Wayland and HiDPI well) and Arch probably offers the best GNOME experience I ever had, besides Fedora. Arch tracks the upstream minor versions, so you get bug fixes quickly. This was especially useful in when Wayland support was not as good, many issues where fixed and then available in Arch days after reporting them.

                                                            I also agree that it is really nice to have recent gcc, clang, perf-tools, etc. versions. I also like the simple PKGBUILD format, it’s much easier to package up some software than e.g. Debian/Ubuntu.

                                                            1. 9

                                                              I use the compiler explorer quite regularly and it is really a brilliant tool. Although for deeper diving I use cargo-asm (or just profile and look at the assembly) for short explorations to answer the question ‘what would rustc/gcc/clang/… compile this to?’, the compiler explorer is really awesome.

                                                              1. 2

                                                                Thanks! cargo-asm looks really useful.

                                                              1. 15

                                                                I’m under the impression that opt-out is not allowed under GDPR, only opt-in. The question is, then, which this UI is. I’d argue it’s opt-out.

                                                                1. 1

                                                                  The trick is that, had I clicked the button “Sounds Good, Thanks!”, I would have opt-in.

                                                                  1. 17

                                                                    IANAL, but I think this pretty clearly violates the GDPR. From the GDPR’s preamble:

                                                                    If the data subject’s consent is to be given following a request by electronic means, the request must be clear, concise and not unnecessarily disruptive to the use of the service for which it is provided.

                                                                    It is not clear, because it uses a dark pattern (using color choices) for the user to read: “We care about privacy -> Sounds Good, Thanks”. Also, I would call unticking 338 companies unnecessarily disruptive. Moreover:

                                                                    Consent should not be regarded as freely given if the data subject has no genuine or free choice or is unable to refuse or withdraw consent without detriment.

                                                                    Again, I would call manually unticking 338 pretty detrimental.

                                                                    1. 5

                                                                      GDPR Recital 32 is particularly informative. I’ll reproduce paragraphs 1-3 here (reformatted for clarity):

                                                                      (1) Consent should be given by a clear affirmative act establishing a freely given, specific, informed and unambiguous indication of the data subject’s agreement to the processing of personal data relating to him or her, such as by a written statement, including by electronic means, or an oral statement.

                                                                      (2) This could include ticking a box when visiting an internet website, choosing technical settings for information society services or another statement or conduct which clearly indicates in this context the data subject’s acceptance of the proposed processing of his or her personal data.

                                                                      (3) Silence, pre-ticked boxes or inactivity should not therefore constitute consent.

                                                                      While reading (3) alone you might think that this system would be contrary to law, I think in the broader context it’s probably okay? Your attention is being drawn to the fact that you have to give consent to use of your personal data (the modal). You can either look for more information, or say okay. So I don’t see this as a pre-ticked box within the meaning of paragraph 3.

                                                                      However, it’s definitely shady (and common) practice. I think it’s borderline, and it would be fair for the regulator to raise concerns. I suspect that the “not unnecessarily disruptive to the use of the service” will actually count in InfoWorld’s favor here. The Correct Solution would be to offer a deselect all.

                                                                      1. -1

                                                                        Playing devil’s advocate here, but they don’t actually need to untick 338 checkboxes, they only need to click “deselect all” as the author did.

                                                                        1. 9

                                                                          Ehm… the author had to untick 338 checkboxes because there is no “deselect all”.

                                                                          It took a while, actually, but I hate to be manipulated.

                                                                          1. 3

                                                                            Sorry, somehow I misread that. I stand corrected.

                                                                      2. 4

                                                                        Yeah. I feel like that’s against the spirit of GDPR, if not the text. I guess the courts will decide. 🙂

                                                                        1. 2

                                                                          And if not GDPR violation, Shamar and others can sue them in class action for the damage to their hands from 338 checkboxes. Each violation will pay a fine equal to the sum of their users at risk of arthritis or carpal tunnel. That’s on top of any GDPR fines.

                                                                    1. 3

                                                                      Still my reimplementation of the FastText embeddings papers in Rust. Now mostly working on optimizing hot loops, while testing on 400M and 28B token corpora.

                                                                      I got quite a bit of a performance improvement by replacing use of prefix sums + binary search (Har-peled, 2011, though the technique is much older) for weighted negative samples by drawing from a Zipfian distribution and writing more SIMD-vectorized loops.

                                                                      The implementation’s speed is now in the same ballpark as FastText, and is mostly bound by memory latency. Though I have some ideas to make it faster (e.g. reusing negative samples).


                                                                      Besides that I was updating some of my older Tensorflow RNN models to use dynamic shapes, so that I can use larger batch sizes during prediction.

                                                                      As usual, all of this is a bit delayed by teaching and other ‘distractions’ ;).

                                                                      1. 22

                                                                        just hope you don’t have to do any string manipulation :)

                                                                        1. 13

                                                                          This. I have a good amount of experience writing C code and maintaining larger C applications, and C can be a real pain to deal with. Not to mention that it exposes a whole host of nasty security vulnerabilities. Finally, it seems a bit too low-level for these kinds of applications. I’m very confused by the choice of C here.

                                                                          1. 3

                                                                            I’m guessing it’s because C is the main API for SQLite? I do I agree that C is an interesting choice here, maybe something more like Lua?

                                                                          2. 4

                                                                            I agree. Writing secure C is hard. Sure, you can pledge your way out of it, but that doesn’t help if sensitive data is stolen. But what would be a reasonable alternative? Rust is probably too complex a language for the taste of OpenBSDers. Go?

                                                                            1. 2

                                                                              you can pledge your way out of it,

                                                                              You can’t. Their kernel and firmware still processes network-facing data. It might still do damage. How much is an unknown until the setup gets the kind of rigorous pentesting we see on Windows, Chrome, the SFI schemes, and recently x86 CPU’s. It does have a nice security by obscurity benefit on top of methods that provably increase work for attackers.

                                                                            2. 1

                                                                              There’s no string manipulation in HTTP servers, right?

                                                                              Right?

                                                                            1. 5

                                                                              I am reimplementing the fastText embeddings papers in Rust (at the very least the skipgram model). Reasons:

                                                                              • I need to use fastText-like embeddings in several Rust programs.
                                                                              • I want to use/try some other ideas from the literature (such as position-dependent context vectors).
                                                                              • It’s fun ;).

                                                                              I have training working now. Next up: saving/loading models, clean up first version for code review. Other than that running some final experiments for a paper. Preparing/giving lectures.

                                                                              1. 12

                                                                                Wow, that’s a lot of bloat, and a great demonstration of why I don’t use Gnome (or KDE).

                                                                                I’m much happier with StumpWM, which just does its job and doesn’t try to integrate with everything.

                                                                                1. 12

                                                                                  Unfortunately, if you want Wayland — and I do, as it really has made all my vsync/stuttering/tearing issues go away; Fedora is now as smooth as my mac/windows — your choices are limited. Sway is starting to look good but otherwise there’s not much at the minimal end of the spectrum.

                                                                                  If I have to choose between GNOME and KDE, I pick GNOME for the same reasons the author of this piece does. I was hoping the tips would come down to more than “uninstall tracker, evolution daemons et al. and hope for the best”. I’ve done that before on Fedora and ended up wrangling package dependancies in yum. I really wish GNOME/Fedora would take this sort of article to heart and offer a “minimal GNOME” option which is effectively just gnome-shell.

                                                                                  1. 3

                                                                                    Why is Wayland so poorly implemented? Is it because few distributions have it as default or is it because it’s harder? I see many tilling wm written in 50 different languages and it seems that sway is getting slowly it’s way to a usable wm, but it seems like a slow adoption from my point of view.

                                                                                    1. 4

                                                                                      It is a slow adoption, I’m not particularly sure why. Most (all?) of the tiling wms for X leverage Xlib or XCB, right? Perhaps it’s just needed some time for a similarly mature compositor lib to appear for Wayland (indeed, Sway is replacing their initial use of wlc with wlroots which may end up being that).

                                                                                      As for why Wayland in general isn’t more prevalent, I’d guess compatibility. X is just so well established that replacing it is inherently a lot of work in the “last mile”. Fedora/GNOME/Wayland works great for me with my in-kernel open AMD driver. Maybe it’s not as good for Intel iGPUs? Maybe it’s not so good on Nvidia systems? Maybe it doesn’t work at all on arm SoC things? I have no idea, but I can easily understand distros holding off on making it default.

                                                                                      1. 3

                                                                                        Maybe it’s not so good on Nvidia systems?

                                                                                        Exactly, the proprietary driver does not support GBM, they’ve been pushing their own thing (EGLStreams) that compositors don’t want.

                                                                                        Maybe it’s not as good for Intel iGPUs? Maybe it doesn’t work at all on arm SoC things?

                                                                                        Everything works great with any open drivers, including VC4 for the RPi.

                                                                                        1. 2

                                                                                          Maybe it’s not as good for Intel iGPUs?

                                                                                          Just a data point: I’ve got a new thinkpad recently, installed linux on it, together with gnome3. Only yesterday I’ve discovered it was running on wayland the whole time, with no apparent problems what-so-ever. And that includes working with a dock with two further displays attached, and steam games. Even the touch panel on the screen works without any further config.

                                                                                      2. 1

                                                                                        Unfortunately, if you want Wayland — and I do, as it really has made all my vsync/stuttering/tearing issues go away; Fedora is now as smooth as my mac/windows

                                                                                        And effortless support for multiple displays with different DPIs, plus better isolation of applications. I completely agree, when I switched to Wayland on Fedora 25 or 26, it was the first time I felt in a long time that the Linux desktop is on par again with macOS and Windows (minus some gnome-shell bugs that seem to have been mostly fixed now).

                                                                                        At some point, I might switch to Sway. But with Sway 0.15, X.org applications are still scaled up and blurry on a HiDPI screen (whereas they work fine in GNOME). I’ll give it another go once Sway 1.0 is out.

                                                                                        1. 1

                                                                                          not much at the minimal end of the spectrum

                                                                                          Weston! :)

                                                                                          My fork even has fractional scaling (Mac/GNOME style downscaling) and FreeBSD support.

                                                                                          1. 1

                                                                                            There’s a Wayland for FreeBSD? I thought Wayland had a lot of Linux specific stuff in it?

                                                                                            1. 3

                                                                                              Sure, there is some, but who said you can’t reimplement that stuff?

                                                                                              • libwayland, the reference implementation of client and server libraries, uses epoll. We have an epoll implementation on top of kqueue.
                                                                                              • Most compositors use libinput to read from input devices, and libinput:
                                                                                                • reads from evdev devices (via libevdev but that’s a really thin lib). We have evdev support in many drivers, including Synaptics (with TrackPoint support).
                                                                                                • uses libudev for device lookup and hotplug. We have a partial libudev implementation on top of devd.
                                                                                              • For GPU acceleration, compositors need a modern DRM/KMS/GBM stack with PRIME and whatnot. We have that.
                                                                                              • Compositors also need some way of managing a virtual terminal (vt), this is the fun part (not).
                                                                                                • direct vt manipulation / setuid wrapper (weston-launch) is pretty trivial to modify to support FreeBSD, that’s how Weston and Sway work right now
                                                                                                • I’m building a generic weston-launch clone: loginw
                                                                                                • ConsoleKit2 should work?? I think we might get KDE Plasma’s kwin_wayland to work on this??
                                                                                                • there were some projects aimed at reimplementing logind for BSD, but they didn’t go anywhere…
                                                                                              1. 1

                                                                                                For GPU acceleration, compositors need a modern DRM/KMS/GBM stack with PRIME and whatnot. We have that.

                                                                                                Do NVidia’s drivers use the same stack, or are they incompatible with the Wayland port? I’d give Wayland a try, but it seems hard to find a starting point… I’m running CURRENT with custom Poudriere-built packages, so patches or non-standard options aren’t a problem, I just can’t find any info on how to start.

                                                                                                1. 2

                                                                                                  No, proprietary nvidia drivers are not compatible. Nvidia still does not want to support GBM, so even on Linux, support is limited (you can only use compositors that implemented EGLStreams, like… sway 0.x I think?) Plus, I’m not sure about the mode setting situation (nvidia started using actual proper KMS on Linux recently I think?? But did they do it on FreeBSD?)

                                                                                                  It should be easy to import Nouveau to drm-next though, someone just has to do it :)

                                                                                                  Also, you can get it to work without hardware acceleration (there is an scfb patch for Weston), but I think software rendering is unacceptable.

                                                                                          2. 1

                                                                                            I tried to give Wayland a try twice, on both my media PC and a new Laptop. It’s still really not there yet. I use i3 on X11 and Sway is really buggy, lacks a lot of backwards compatibility stubs (notification tray icons are a big one) and just doesn’t quite match i3 yet. Weston, the reference window manager, had a lot of similar problems when using it with my media PC.

                                                                                            I want to move on to Wayland, and I might give that other i3 drop-in for Wayland a try in the future, but right now it’s still not there yet.

                                                                                        1. 7

                                                                                          Adobe should be in this list.

                                                                                          I have unsubscribed from all the Adobe spam, but now they have sent a confusing E-mail saying that GDPR means I need to review my communication preferences. There is no indication that I can simply disregard this e-mail, and clicking the link in the e-mail will take you to a page that re-enables all the spam!

                                                                                          Several companies have employed this exact scammy strategy, but Adobe is the biggest one that I remember.

                                                                                          1. 1

                                                                                            As far as I understand from reading the GDPR, you could send them an email with a request to erase all the data that they have on you, and they are obliged to remove it in a month.

                                                                                            (IANAL)

                                                                                          1. 8

                                                                                            Hah, I am just cleaning/tokenizing/tagging/parsing/lemmatizing a 27.3B word corpus (German text from the common crawl). The problem is embarrassingly parallel and just fits a single higher-end machine (40 cores, plenty of RAM). Each annotation layer is produced by a separate program (mostly in Rust, plus some Go, Python and Java). It’s all driven by some shell scripts and xargs -P for processing chunks in parallel, so there is virtually no overhead for the actual work distribution. UNIX pipelines are extremely flexible and still fit many job sizes well.

                                                                                            It makes me think back of a prior job where they would deploy Hadoop clusters for tasks that required only a tiny fraction of the processing. But hey, throwing Hadoop at the problem sells (it must be big data then).

                                                                                            (Of course, the are many scenarios where you actually want a Hadoop cluster, but I agree that people are often overestimating the computational requirements of their problem.)

                                                                                            1. 5

                                                                                              In my day job, I work with patent data. I’m part of a unix shop that is in the middle of a much larger java-oriented company. We regularly do the same job on the same data faster & easier on a single beefy workstation with awk than our colleagues with a reserved 400 node hadoop cluster. So, OP’s experience & yours resonate with me a lot.

                                                                                              I’m not sure it’s even a matter of overstating computational requirements. If you’ve got something that can easily be modeled as a pipeline, map-reduce is likely to be a poor fit, since too much emphasis will be on the reduce part. So, it’s a matter of understanding your own problem and your own algorithm for solving it, and understanding whether or not the hard bits can be parallelized.

                                                                                              1. 3

                                                                                                People seem to think Hadoop is about processing power. It’s not, it never was. Google only used MapReduce for processing datasets much too large to fit on a single machine. If your data can physically fit in the drives of one machine, and your processing pipeline’s working memory can physically fit in the RAM of one machine, you certainly should not use Hadoop.

                                                                                                1. 1

                                                                                                  Sure - but how many people does that apply to? AWS will rent you a machine with 128 cores and 4tb ram by the hour, and a truck to carry your backup tapes to it - and if that’s not enough you can buy bigger iron.

                                                                                                  1. 1

                                                                                                    Exactly, it barely applies to anyone.