Threads for zgrep

  1. 28

    I can think of no shortage of ways in which nix could be improved but it always baffles me when people complain about the nix language, especially since it seems that objections tend to be about its syntax. Perhaps it’s just the Haskeller in me, but to my eyes the syntax seems extremely natural given the language’s domain (targetting pure expressions with of attribute sets, lists, and literals). The syntax is consistent and minimal, focusing its sugar on those areas where it is most needed (namely bindings, where it offers the convenient with and inherit constructs). I can think of few syntactic decisions that I would change if given the opportunity (e.g. perhaps allowing comma-separated lists).

    I suspect the objections to syntax largely stem from poor documentation. While the nix grammar itself is quite small, it is not well explained in any of the introductory documentation. Instead, the nix manual gives page-after-page of prose, documenting each language construct (verbosely) by example. While this has its place, it would be nicely complemented by a single blurb of BNF. This would provide a nice summary of the language’s moving parts and lay bare its simplicity.

    I think the Nix’s biggest problem is not its language but rather nixpkgs and its numerous under-documented conventions and libraries (especially given that these often don’t interact well together; e.g. the override problem). Of course, this is a much harder problem to tackle.

    1. 24

      The problems with the Nix language have nothing to do with its syntax or its documentation (atrocious as that may be). The problems have to do with its semantics.

      The Nix language is lazy in the worst way possible. Evaluating (like say printing) any statement at any time can result in actual operations being carried out. So you can’t print or view anything.

      Nix also uses its laziness in a terrible way. Instead of providing proper APIs, they rely on nesting objects within one another constantly. So you can try to see what an object is, but you quickly discover it contains endless copies of other objects. It’s very hard to build a mental model of what thing you’re dealing with. That’s what makes overrides hard.

      This is all made far worse by the lack of types and definitions for objects. At least in scheme or lisp people use nested objects judiciously and printing doesn’t force promises. In Python I can see what a class contains. In Haskell, I have types, so I know what object I have. Nix is the worst of all worlds. Computations can run at any moment for any reason and you can never know what something is and everything is an unnamed map with random fields attached to it (I’m sorry, a “set”, in Nix they call maps sets).

      But the worst part of Nix the language is how Nix the system uses the concept of laziness. The idea that is that you’re constantly building up a derivation that will be forced by the system at the end. It’s like in C you have a phase where you compile to object files and then link them. But now imagine that the compiling to object files happens randomly, any time you happen to execute any statement, and you can never know if the statement you have compiles something to an object file or is say a number.

      All of this is made worse by how Nix is being used: there are no standard APIs for anything beyond the most basic derivation. Every single language has one or more extremely incompatible and “quirky” ecosystems.

      1. 14

        I used to be a Haskeller. The problem is that Nix misses a lot of the things that makes Haskell good. All the typing is dynamic, and there’s no structural data typing at all. Want to know what kind of value that set you have in the REPL is? Too bad! Look at the property names and kinda guess at it.

        There are no ADTs or even C-style enums, so you have stringly-typed enums everywhere (although the way the module system works mitigates this somewhat).

        The fact that packages are defined as, effectively, functions also makes development frustrating. In order to print out my package in the REPL, I have to call it with the entire environment it needs. And when you’re writing a module with configuration options, that becomes… effectively impossible. Although that’s not really a problem with the language per se.

        1. 10

          As a language, nix doesn’t feel markedly better or worse than various others. However, I think it’s still acceptable to say that I “dislike the language” because I find it difficult to use its de-facto standard library, nixpkgs¹. In my opinion, the standard library can be considered part of the language in most situations. So (as with most programming languages) nix cannot get by purely on its own merit, but is bolstered or dragged down by the libraries it comes with. Unfortunately more so dragged down, in my experience.

          Edit: What I mean to say is, I don’t think I see as many people complaining about the syntax when they complain about “the nix language”. But it’s possible I have a skewed perception of things.

          ¹ And flakes, once considered completed or stable, would be yet another (small) layer of confusion for this “standard library”.

          1. 10

            Space-delimited lists were probably chosen for convenience in simple cases, but they trip me up so often on anything non-trivial:

            # Artificial example
            # This constructs a list containing a derivation, then a function, then a set, then a derivation
            packages = [
                blah blah whatever
            1. 4

              Nix is very nice. I think the problems are mostly:

              • Documentation is too limited, and many popular sources are outdated.

              • The CLI interface is a bit confusing as it has both classic nix commands, nix 2.0 and flakes in the same binary.

              • There’s too much shell code in NixPkgs, which makes it hard to understand.

              • There’s not enough tooling to comprehend the effects of an invocation without running it.

              I am very positive about Nix’s future. I think despite its shortcomings it is the best distro you can run for many workflows. It just needs a bit more of manpower and funding to clean old cruft.

              This is in general a problem with many pieces of software, which evolve too quickly and not enough attention is put into documentation. In an ideal world, Nix would have several full time engineers writing docs.

              1. 3

                Oh and a really frequent misunderstanding is that Nix is implemented in Haskell, and therefore obscure and complicated. Nix certainly borrows many ideas from pure functional languages, and is an excellent package manager for Haskell, but has nothing to do with the language per se.

                1. 2
            1. 14

              Putting in my notice at my current job, then presumably working on all the ‘urgent’ things my boss was expecting me to do but hadn’t told me about….

              outside of work going to continue trying to learn haskell through doing advent of code!

              1. 3

                Congrats! If you can afford it, I hope you’re able to take some time off before the next gig.

                1. 3

                  Congratulations! Break a leg.

                  1. 1

                    continue trying to learn haskell through doing advent of code!

                    I’ve been doing this too! It’s been strange, mainly since I’m not used to Haskell’s approach to problems. I have some familiarity with thinking about things functionally, but not as much as Haskell requires. I guess that’s what practice is for!

                  1. 1

                    Meh! That is opt in by the website….

                    One click on Tranquility Reader firefox extension and splat, back to the ’90s but without the blink tag!

                    Tranquility Reader improves the readability of web articles by removing unnecessary elements like ads, images, social share widgets, and more.

                    1. 5

                      Meh! That is opt in by the website….

                      I believe that Firefox judges using a few heuristics such as content length, HTML tags, and classes used. I wouldn’t count that as opt in.

                    1. 2

                      Very slowly working through bits of The Napkin. Attempting to set up a storage server in the basement. Trying to figure out how to focus on tasks and how to generally convince myself to do things.

                      1. 1

                        Hey, cool! I’m contemplating a storage server myself. I’m considering using an old 2007-ish Mac Pro I found on the sidewalk. (Was going to use it as a PC, then found out the motherboard screws were in a proprietary arrangement. I don’t really have the time for hardware stuff…)

                      1. 8

                        Part II because it’s a sequel.

                        Get it?? “Sequel”?? Like SQL?

                        I’m here all week. Don’t forget to tip the waitstaff.

                        1. 6

                          Get it?? “Sequel”?? Like SQL?

                          Some people don’t know, but this is the name with the history behind it:

                          Chamberlin and Boyce’s first attempt at a relational database language was Square, but it was difficult to use due to subscript notation. After moving to the San Jose Research Laboratory in 1973, they began work on SEQUEL.[12] The acronym SEQUEL was later changed to SQL because “SEQUEL” was a trademark of the UK-based Hawker Siddeley Dynamics Engineering Limited company.[14]

                          On the other hand, my mom once pronounced it “squirrel” and I think that’s apt for a language which helps you store things for later.

                          1. 7

                            Someone recently mentioned to me that SQL should be pronounced “squirrel”, as in “my squirrel”, “miss squirrel”, and “post grease squirrel”. I think I can’t argue against their reasoning.

                        1. 6

                          Gonna be working on my IRC bot! I’ve never done this before, but I find it to be an excellent way to learn a language since it includes parsing text and interacting with the IRC protocol.

                          1. 5

                            A useful resource for the IRC protocol:

                            1. 4

                              I had to implement the IRC protocol for my bot in Ruby a couple months ago because I couldn’t find a single good gem for it (most were from ~2008, abandoned and didn’t support everything I wanted) and it turned out to be very easy. Just remember to respond to PING and maybe send PINGs to the server yourself every 120 seconds or so, and to only send login after the server has sent the MOTD (code 376 or 422), that caused me a lot of headaches!

                              Here’s the implementation for my bot if you wanna take a gander.

                              1. 3

                                I have already gotten those things done already, IRC was a really simple protocol to tackle :) It’s actually been working since yesterday with a few commands. Excited to add more!

                            1. 9

                     [..] much more than just a viewer. You’ll need to open either the Gimp or Xournal for any basic editing

                              LibreOffice Draw is the real PDF editor :D

                              But yeah, I kinda do wish reordering pages wasn’t “beyond the scope of Evince”.

                              the only email client with a native “conversation view” is Geary, which is in early development and still very buggy

                              Eh.. is it really that buggy? I really don’t have any problems with it, but then I’m not a heavy email user and kind of an email-hater.

                              Screen tearing with the intel driver. Come on. This was solved on xorg and now with Wayland it’s back

                              There is no screen tearing on Wayland (modulo really bad compositor bugs), it’s literally a BIG point of the whole protocol. There is no “intel driver” with Wayland, compositors use generic EGL/DRM/GBM interfaces, provided by Mesa (where the Intel drivers are iris for new stuff and i965 for old stuff, nothing is called intel). The author is definitely using xorg, considering the mentions of xdotool etc.

                              Xournal is an obscure app

                              Wait, why would you use the ancient xournal? You’re looking for xounalpp.

                              1. 2

                                Wait, why would you use the ancient xournal? You’re looking for xou[r]nalpp.

                                I, for one, had never heard of xournalpp before. Thanks for the link!

                              1. 3

                                Sorry for having the first comment you receive be a (minor) complaint about a platform you likely weren’t targeting. On Firefox on Android, attempting to type ,help makes the help text briefly appear before quickly vanishing. This makes it a tad difficult to understand what this does without resorting to finding it on your GitHub. :P

                                (Without having looked into it from a computer, and therefore meaning I don’t know what’s actually happening, my guess would be that this is something to do with changing content when the cursor position changes?)

                                1. 6

                                  Hi @zgrep, Thank you for reporting this issue with detailed information. It helped me to reproduce the issue with Firefox on Android. The issue was caused by a second input event being fired when the text input field was being changed by the tool itself. Apparently, this behaviour seems to occur only with Firefox on Android. This behaviour does not seem to occur with the other browsers I had earlier tested with like Firefox on macOS, Chrome on macOS, iOS, and Android.

                                  I have now pushed a fix to resolve this issue. The issue you faced should be resolved now. Thanks, once again, for taking a close look at this tool, reporting this issue, and helping me to improve it!

                                1. 2

                                  Teaching myself how to implement depth/breadth first searches in Python because I’m still struggling with the island counting algorithms exercise I mentioned last week. Right around rewrite #3 I realized I needed to take a giant step back and figure out what was going wrong.

                                  This is IMO a strong argument for why semi-formal algorithms training is important. I’d love to find more resources on algorithms that aren’t Comer. I owned the 1st edition of his book but found it to be incredibly hard to penetrate.

                                  I see Algorithms by Sedgewick & Wayne is part of my Safari subscription. Maybe I’ll put some time into trying to get through that.

                                  1. 3

                                    May I suggest MIT OCW as a potential resource?

                                    1. 1

                                      Introduction to Algorithms requires Mathematics for Computer Science as a pre-requisite, which requires Calculus, which I do not have.

                                      My former co-workers are telling me I could probably get by the Math for CS course and not conquer the Calculus dragon and be fairly well prepared for the course.

                                      Wish me luck :)

                                      1. 3

                                        I do believe that most of 6.042 (Math for CS) and 6.006 (Intro to Algorithms) do not require calculus. I don’t recall 6.046 (Design and Analysis of Algorithms) being particularly calculus-laden either, but my memory is fuzzy and unwilling to clarify.

                                        If at some point you do wish to slay the calculus dragon, and enjoy watching YouTube videos, there’s a wonderful set of videos by 3blue1brown that gives a very nice introduction to it. Of course, to actually learn it (or anything), the best way is always to do exercises or tasks that require calculus.

                                        If you have any questions, feel free to ask (through a message, IRC, email, etc), and I can attempt to answer. No guarantee I’ll give good answers, it’s been a little while and I’m not as adept with mathematics as I’d like to be. However, I find it’s easier for me to learn from talking to other people, than it is for me to learn from staring at a lecture or lecture notes, so I figured I’d extend the offer anyway.

                                        Good luck! :)

                                      2. 1

                                        That is awesome, thank you very much! I think the intro is probably about all I can handle right now :)

                                        I actually dream about retiring and taking some of the free open to the public summer session “for fun” courses MIT offers. I worked there for a few years and the community is downright magical - I don’t care what anyone says about institutional rot and fund raising foibles :)

                                    1. 9

                                      While not a (single) programming language, my first thought was to use uniq(1):

                                      echo 'aaaabbbcca' | grep -o '.' | uniq -c
                                      1. 1

                                        Wow, that’s elegant.

                                      1. 2

                                        This was a wonderful read, and it reminded me of Rash, a merger of Racket and shell scripting. I find it interesting the blog post is embedding a shell-like syntax within Python, while Rash is a language that embeds Racket syntax within itself.

                                        1. 1

                                          Outside work, I’m trying to reverse-engineer the IR protocol for an AC. Y’all are welcome to help!

                                          Hitting “power” on the remote yields a message like this:


                                          I’ve recorded one message for each temperature level the remote can set, the bits that change are, oddly(?) spread out across the message. H,M and ? changes with the clock in the remote. T changes with temperature. _ indicates the bit is the same in value in all messages so far.


                                          If you collapse the message and just look at the bits that change with temperature setting changes, these are the bits for each temp level (in F, on the remote anyway):

                                          61 00001100
                                          62 00001111
                                          63 10001100
                                          64 01001100
                                          65 01001111
                                          66 11001100
                                          67 11001111
                                          68 00100100
                                          69 00100111
                                          70 10100100
                                          71 10100111
                                          72 01100100
                                          73 11100100
                                          74 11100111
                                          75 00011000
                                          76 00011011
                                          77 10011000
                                          78 10011011
                                          79 01011000
                                          80 01011011

                                          If anyone has IR experience and/or likes puzzles, let me know if you see the pattern. I may just end up encoding these in a table, though that feels like cheating :p

                                          1. 6

                                            I can help with the first four bits: It adds 0.5°F, then converts it to °C, subtracts 16°C, and the bits are in reverse order.

                                            In Python:

                                            def bits(temp_F):
                                                return bin(int( (temp_F + 0.5 - 32)*5/9 ))[3:].zfill(4)[::-1]
                                            1. 1

                                              Oof! Awesome.

                                              The last four are interesting.. there’s definitely a grouping. A list of temp in C, minus 16, vs bit pattern seems to say they group incrementallyish. Wonder if it’s some sort of checksum or mask. As I’ve understood it IR protocols are prone to include redundancy or similar, given the shoddy transfer medium.

                                              bits temp_c_minus_16
                                              1100 [0.39, 1.5, 2.06, 3.17]
                                              1111 [0.94, 2.61, 3.72]
                                              0100 [4.28, 5.39, 6.5, 7.06]
                                              0111 [4.83, 5.94, 7.61]
                                              1000 [8.17, 9.28, 10.39]
                                              1011 [8.72, 9.83, 10.94]

                                              edit: staring at this, definitely something about the fraction right, it splits cleanly around .5; all the ones that are >.5 end in 11, and the ones that are <.5 end 00

                                              1. 4

                                                The next two bits seem to be the bitwise not of the {uppermost bits, division by 4} of (°C - 16). You noticed that the last two bits follow (°C - ⌊°C⌋) > ½, but they also correspond to the 4th bit to the right of the decimal point.

                                                All together in Python:

                                                for f in range(61, 81):
                                                    print(f, end=' ')
                                                    c = (f + 0.5 - 32)*5/9
                                                    c = 0xff & int(c * 2**4)
                                                    c = list(map(int, bin(c)[2:].zfill(8)))
                                                    c[0:4] = c[0:4][::-1]
                                                    c[4:6] = [1 - b for b in c[2:4]]
                                                    c[6] = c[7]
                                                    print(''.join(map(str, c)))

                                                I’m not entirely satisfied with this explanation, but it’s the best I can think of.
                                                To know for certain you could always dump the code off of the remote and reverse engineer it. :P

                                            2. 3

                                              Checkout the content at analysir. They sell a product but Chris is also an expert and his blog posts may be helpful.


                                              1. 1

                                                Oh wow, that’s super in depth, thanks for this!

                                            1. 6

                                              I’m really intrigued by these devices. I’ve heard a lot of good things about them, but the thing that’s putting me off is that you seem to have to use their cloud service to sync PDFs and notes, and the hand-written stuff seems to use a proprietary format.

                                              There’s also a reviewer on YouTube called “My Deep Guide” who loves the rM2 but he has some valid criticisms of the software not being as good as the hardware, especially compared to competitors.

                                              1. 4

                                                I’ve refused to connect my device to their cloud and have had no problem - it is a bit unfortunate that this means losing the handwriting recognition, but everything else I care about works just fine.

                                                You can sync files over usb, the remarkable pretends to be a network adapter and serves a webpage on which you can drag files onto/off of. I’ve also sshed into the device and created a script to back it up with restic (just a normal piece of backup software, not remarkable specific) ;)

                                                The hand written notes format is proprietary, but it’s simple and has been reverse engineered. It’s not hard to export the notes to another format.

                                                I’d generally recommend it, as long as you understand it as “paper + pdfs printed onto paper + bare bones linux” and not anything more.

                                                1. 2

                                                  This section is titled “jailbreak,” which is actually a bit of misnomer because the reMarkable runs Linux and you can ssh into it with ease.

                                                  1. 3

                                                    I’m not sure how that really makes it a more usable device. Instead of being able to plug it in with a USB cable and sync files as a removable drive, I have to put it on the network and use a non-standard feature?

                                                    1. 3

                                                      You can plug in a USB cable, it just shows up as a network interface instead of as a storage device. It’s probably not the most usable option, but it does make it easier to see and interact with the device as a computer instead of as a fancy flash drive.

                                                      1. 3

                                                        When you connect a reMarkable to your computer via USB, the computer sees a USB network adapter. Then use DHCP or set an IP manually for that adapter. Usually the reMarkable will use for itself and for “your side”.

                                                        So even when you use USB, you can ssh into it.

                                                        1. 1

                                                          I would check out the Awesome list, specifically the APIs and Cloud Tools sections. I also use scp fairly often to just grab the files directly, or even export them using the app.

                                                          I’ll check out “My Deep Guide,” I haven’t really read many reviews of the device myself. I’m also not really aware of any competitors?

                                                    1. 1

                                                      If I decide later to skip the numbers and use bullets instead, I tell the computer to introduce each list item with a bullet. This is one command covering the whole list.

                                                      In this respect, Markdown is a step backwards compared to directly writing HTML. I wonder if there are any other lightweight markup languages that have this feature of whatever the author is using (presumably LaTeX).

                                                      1. 2
                                                        1. Word does have this feature. It’s just that lots of people don’t know about it, or don’t like what it does to the formatting, so they don’t use it.

                                                        2. Markdown has this feature, too. If you prefix every numbered item with “1.”, then it will number them for you.

                                                        1. 1

                                                          That’s not the feature being discussed, it’s turning an ordered list into an unordered list. In HTML it’s replacing the opening and closing ol with ul. In Markdown, it’s replacing *1*. with * in every entry (and if you’re keeping different numbers, that’s a complex find & replace).

                                                          1. 1

                                                            It’s also one of the features which makes markdown less usable as a markup language for non-programmers. On Reddit, I frequently see comments starting with a “1.” because the author wanted to start their comment with a number + period and markdown interpreted it as a list entry.

                                                            1. 2

                                                              How many times have I seen this?

                                                              1. This has been a test of markdown lists.

                                                              Edit: turns out lobsters does the wrong thing here. Probably won’t dig into the css that caused this tiny mistake in an edge case. I wrote 12345.

                                                          2. 2

                                                            In CommonMark, at least, it appears that only the first number in the list matters:

                                                            1. List item 1.
                                                            1. List item 2.

                                                            Is turned into:

                                                            1. List item 1.
                                                            2. List item 2.

                                                            This also works on, it seems. So in theory swapping all of one type of bullet point for another is easier than manually changing all numbers, but you are correct in that it’s still more difficult than it would be in HTML or LaTeX.

                                                            Edit: Oops, I didn’t refresh the page before answering your comment, sorry. I was beaten to it by 20 minutes.

                                                            1. 2

                                                              Emacs’ Org-mode uses the same markup but will automatically fix up the numbering in the markup itself as you move things (e.g., when moving a line with M-up or M-down) or when you ask it to with C-c C-c.

                                                              1. 1

                                                                This behavior is in the original Markdown code/spec:


                                                              2. 2

                                                                In a programmers text editor, this is not an expensive thing to change. (In vim something like: Vip:s/^\d+\./ -/, probably need “magic” on)

                                                                Markdown is designed to be readable and presentable as source code, and it would obviously be more difficult to work out which numbered item was which if they didn’t have the number next to them.

                                                                If you want a more expressive markup language there are plenty :)

                                                              1. 4

                                                                I think that there should be a strong bias towards exposing program internals, and a way of browsing those.

                                                                For example, command line tools should provide enough metadata for its command inputs that you should be able to generate a GUI form to call something like grep based on that metadata.

                                                                In a similar sense, though a program might have a “defined entrypoint”, there should be a lot of encouragement to expose alternative entrypoints to use subsystems as needed. For example, be able to directly call a browser’s rendering layer and get some render result/paint to a screen. Maybe more simply, be able to “pry open” the GUI settings menu easily and just find some function you could call directly to toggle certain settings (think Applescript).

                                                                I do think the “life & death of javascript” talk also covers some interesting ground. If we say “let’s not support C” (or, perhaps more cleverly, “we support C through a virtualization abstraction a la emscriptem) then we could potentially build a more type-rich system for most programs.

                                                                The thought experiment: if every program’s base was written in Python, it would be fairly easy to expose Smalltalk OS-style “dictionary of commands/variables” for various programs.

                                                                1. 1

                                                                  On Linux, in theory, a program can also be a shared library. In fact, one can execute from the command line. And the same can be said for AmigaOS, a program there could also be a shared library, but I never saw one, nor did I ever get around to doing a “proof-of-concept.”

                                                                  1. 1

                                                                    While probably not going quite as far as you wish, I’d like to point out snakeware as a Python-based userspace.

                                                                    I’d also make the point that exposing additional entry points is extra work, and people could get mad at you if you ever need to change them. This could result in change being slow, and rife with backwards compatibility hacks everywhere. Emacs is probably a good example of this, but I don’t actually know first-hand if it is.

                                                                    I do very much like the idea that additional entry points should be strongly encouraged, though. Or the general idea of easing “librarification” of large applications.

                                                                  1. 5

                                                                    Hoping this can be abused (and is) to break enclaves as used by e.g. widevine, for e.g. compatibility, accessibility and archivist purposes.

                                                                    1. 3

                                                                      I see one big problem: possibility that streaming services will just stop working on all your existing hardware.

                                                                      1. 3

                                                                        How’s that a problem?

                                                                        • Intel hardware would be used to dump streams, not watch them.
                                                                        • Streams need to be dumped only once. Then they are forever unprotected and can thus be safely preserved by archivists, intact.
                                                                        • I don’t stream DRM’d content in the first place, and I’m not about to start doing so.
                                                                        • Any annoyances (such as described by you) to those who do help build antagonism to DRM.

                                                                        It’s a win-win, as I see it.

                                                                        1. 3

                                                                          You can’t dump them anymore if you’re not able to use netflix & co at all on such intel devices. All you get is that nobody with such a system can use them anymore. And I bet the first OS to be dropped entirely is linux, it’s already not full-hd supported, due to being not as “secure” as windows. So in summary you’re happy if everybody else has to suffer.

                                                                          1. 3

                                                                            What you say doesn’t quite make sense to me. Netflix and such services have an incentive to keep supporting Intel platforms—a large portion of their users use solely Intel-based platforms, and it would not make sense to lose them all as customers. x86-based Intel processors will still be used in both new and old hardware for a long while (despite ARM becoming more and more popular).

                                                                            As for dropping non-Android Linux users, I could see that as being more likely solely in terms of user base, but I still don’t think it’s a likely outcome. Breaking enclaves used by Widevine is not, as far as I’m aware, related to Linux in particular. I assume that the same technology is used by Widevine on all supported platforms (Windows, macOS, and Linux-based), and one could potentially develop an attack on any of them. Googling suggests that Netflix restricts video resolution when it’s not using hardware-based DRM on those platforms, which makes me believe there’s more motivation to do this on non-Linux platforms anyway.

                                                                            1. 4

                                                                              The issue is that Linux is designed to be an operating system which is modifiable by the user at all layers. Linux users are supposed to be able to run software in ways which violates the abstractions, for unusual situations such as sandboxing, debugging the platform, or developing novel new tooling. That is explicitly a non-goal for Windows and MacOS. To whatever extent operating systems can mitigate attacks on Widevine, Microsoft and Apple are likely to happily deploy those mitigations.

                                                                              It is of course up to the relevant video consortium whether they ever actually drop Linux support, and I don’t claim to know what they’re thinking or what they’ll do in the future. I do think it’s clear though that dropping Linux support would be consistent with their larger goals, if Linux’s market share drops far enough.

                                                                              Android involves a lot of custom kernel work for each device, much of which never gets upstreamed, so it’s absolutely possible that Android could keep Widevine support while non-Android Linux loses it.

                                                                              1. 6

                                                                                The issue is that Linux is designed to be an operating system which is modifiable by the user at all layers.

                                                                                I fail to see how that is an issue.

                                                                                DRM is.

                                                                                1. 2

                                                                                  I mean, I think DRM is a scourge and shouldn’t exist. I’m just explaining that it’s entirely conceivable that, in an escalation of that conflict, Hollywood could decide that support for Linux is less important than continuing to have DRM.

                                                                                  1. 2

                                                                                    I mean, I think DRM is a scourge and shouldn’t exist.

                                                                                    I absolutely agree.

                                                                                    Hollywood could decide that support for Linux is less important than continuing to have DRM.

                                                                                    That’d be a good thing. Because the more restrictive DRM is, the less likely people will put up with it.

                                                                                    Personally, if I am interested in something, I’ll probably buy it. But should it have DRM, I won’t. I’ll either lose my interest or find an easier way.

                                                                                    The more the annoying DRM is, the more the people there’ll be who’ll think along these lines.

                                                                                    1. 2

                                                                                      That’s certainly a fair position.

                                                                                  2. 2

                                                                                    Because I’d like to keep using netflix on my laptop without using windows in the future (it’s already degraded), thank you very much. I don’t care about demonizing DRM for the sake of having literally nothing available on linux, that’s useless.

                                                                                    Otherwise I’ll happily switch every system in my family back to windows after years, so they can keep using the things that are relevant for them. I already have a smartphone which can’t even use the app due to missing SELinux.

                                                                                  3. 2

                                                                                    I think you summarized my thoughts better than I could. Especially that android could easily still play widevine content (for example via play-services which require locked bootloaders etc), while open/normal linux systems loose their access entirely.

                                                                                2. 1

                                                                                  Oh and for example disney streaming already does not work on linux. And I personally could totally play the “I don’t care about them” card, until r/leopardsatemyface comes back when other services have to do the same, to keep their streaming contracts.

                                                                                  1. 2

                                                                                    Considering the relationship between copyright and Disney, I won’t be caught dead willingly giving up any of my money to them.

                                                                            2. 1

                                                                              The entitlement of the anti-drm movement is astounding. No one is guaranteed access to someone else’s work for free, and anyone should be able to protect their time and monetary investment.

                                                                              Everyone frames this as the People vs the Big Bad, but the reality is that quality content is expensive and requires significant investment for smaller companies/brands which is where I see DRM most often used, because all it takes is 1 bad actor to steal thousands of dollars and hundreds of hours of time.

                                                                              When those brands/individuals don’t want to allow to harvest their users information, they need a pay-wall and they need a way to secure their investment.

                                                                              We cannot delude ourselves into thinking we can have it both ways if we are to move beyond an ad revenue driven monetization model for the internet.

                                                                              DRM isn’t perfect, but it’s the best we have so far.

                                                                            1. 6

                                                                              The reasoning is that only Firefox continues developing and maintaining an independent, functioning browser engine, with an independent code base.

                                                                              There is still WebKit as a browser engine, which is a perfectly reasonable browser engine. Though Blink may have initially been a fork of WebKit, I believe that the two have diverged severely over time. WebKit is even more-used than Firefox in the linked browser usage page, if only because of Apple. There are non-Apple browsers that use the WebKit engine.

                                                                              A recent change in the add-on system has broken compatibility with a wide range of older extensions, with no compatibility layer.

                                                                              I would like to point out that there is a technological reason for this (corresponding post), one of making it harder for a malicious extension to do harm (among other things). I don’t know how much this was actually a problem in practice, if the replacement is sufficient, or if a different alternative would have been better, but I felt that it was important to at least mention.

                                                                              1. 4

                                                                                Webkit cannot be safely used on Debian-based systems as it does not receive security updates: (I believe the situation on other operating systems is similar, but as far as I know Debian is the only one to call it out in the release notes as not being suitable for using on untrusted sites.)

                                                                                There are some remediations that technically-inclined users can apply to make it somewhat-safe to use, but it’s not something you can make a blanket recommendation for people to use without a mountain of caveats.

                                                                                1. 5

                                                                                  Interesting! I was wondering what the situation is like for other operating systems, and found an overview by a Gnome developer in 2017. The gist of it seems to be that WebKit does not do long-term releases, they merely do 6-month stable releases. Therefore long-term releases of any distribution remain vulnerable, as well as Debian supposedly remains vulnerable, but most other distributions are apparently up to date.

                                                                                  Of course, instead of merely trusting a blog post from 2017, one could also check repology, which at first glance seems concerning given the amount of red visible and yellow triangles, but it appears that most latest stable releases of distributions also have a version of webkitgtk that does not have any CVE’s attached to it. Including Debian stable, for that matter.

                                                                                  The same Gnome developer’s 2016 post on the topic claims that:

                                                                                  Debian is correct that we do not provide long term support branches, as it would be very difficult to backport security fixes. But it is not correct that “library interdependencies make it impossible to update to newer upstream releases.” This might have been true in the past, but for several years now, we have avoided requiring new versions of libraries whenever it would cause problems for distributions, and — with one big exception that I will discuss below — we ensure that each release maintains both API and ABI compatibility.

                                                                                  With the big exception being the deprecation of an old API/ABI around September 2014. I don’t know if this is a reasonable exception or not, but also would sincerely hope that it doesn’t matter by now.

                                                                                  In any case, I feel like it’s still potentially sensible to say that a blanket recommendation of WebKit is a subpar idea, without additional guidance or information.

                                                                                2. 2

                                                                                  I’m not sure if WebKit is technically different enough. Google has replaced individual components, but the overall architecture would be too hard to change in a major way (ship of Theseus is still a ship after you replace everything).

                                                                                  1. 3

                                                                                    I’m not so certain, because part of the goal of forking WebKit to make Blink would be to let them make larger architectural changes of the kind that upstream WebKit would be more wary of accepting (or have various reasons not to accept). The Chromium team seems to imply as much in a (rather old) FAQ document, as does a presentation from a WebKit developer. I guess I find it hard to assume Google wouldn’t make large changes to a code base, especially if they’re aiming to improve speed as much as they can, and especially over the span of several years.
                                                                                    However, these are (relatively) old documents I found by searching for what I wanted to find, and perhaps the overall architecture is (as you said) too hard to change in a major way. I wouldn’t know, I haven’t looked at any of the code myself (and don’t plan to any time soon).

                                                                                1. 2

                                                                                  Ok I need advice. I used to exclusively use a Macbook Pro (and still use it frequently), I generally agree with this article, I have a lot of experience (and LOVE) running FreeBSD servers, I write a lot of C code and have no problem porting software, I like the MIT/BSD license over GPL, and I run Ubuntu now on my Thinkpad/desktop (which I use more frequently than my Macbook), is it worth it to switch to OpenBSD?

                                                                                  OSes I’ve used so far (clientside):

                                                                                  • OS X, easy, gets out of your way, lots of nice proprietary stuff. Quality is slowly degrading over time though. And brew sucks. Maybe the arm laptops will change my mind.
                                                                                  • Ubuntu, I love APT/dpkg and know it like the back of my hand, installing proprietary stuff is pretty easy, sane defaults, easy to Google stuff, most software with linux compatibility usually has a Ubuntu .deb or a ppa, and generally gets out of my way. Only real con is the warts that come from Linux and the occasional weird decision by Canonical
                                                                                  • Slackware, it was nice but lack of a package manager is kind of annoying
                                                                                  • Arch Linux, it was fun when I was in high school but now I don’t really care what WM I use and I hate the rolling release model
                                                                                  • Jailbroken iOS, my favorite OS by far, apt/dpkg + BSD base + Mobilesubstrate = amazing

                                                                                  And just to be clear I don’t care about Free as in Freedom™ when it comes to my personal computers. I just want a well designed OS that will make me as effective as possible, and can run proprietary Linux binaries. And I want to do VFIO and play games in a Windows VM in QEMU

                                                                                  1. 4

                                                                                    is it worth it to switch to OpenBSD

                                                                                    I should probably say this with a nixpkgs contributor hat on, but:

                                                                                    Maybe you should add NixOS and Guix to your candidates. Most of the OSes that you have used, use traditional package managers. Nix and Guix provide a lot over that: declarative system configurations, per-project virtual environments, atomic upgrades/rollbacks, and the ability to have multiple versions/build configurations of a program/library on the system. Given your requirements, you would probably like Nix more, because it also packages proprietary software (so e.g. installing CUDA is not a problem) and provides stable releases besides the rolling unstable.

                                                                                    You can also install Nix or Guix on an existing Linux distribution to test the waters (macOS too, but it has issues more regularly).

                                                                                    It may not be for you, but it’s at least worth considering.

                                                                                    1. 3

                                                                                      +1 to NixOS. :)

                                                                                      I love the culture of people putting up their entire system configurations, defined end to end, up on public repos. Here is mine:

                                                                                      When something doesn’t work, it’s very easy to find examples of other peoples’ setups as a reference. If we have the same hardware, you can use my configurations (or subsets) to replicate my experience precisely. If something breaks, it’s easy to get help on a problem by pointing to my configuration. Not to mention how easy it is to roll back!

                                                                                      I don’t think I can ever go back to a “traditional” operating system without these mechanics.

                                                                                      1. 1

                                                                                        Yah I’ve heard of Nix and it looks cool but I never really found APT/dpkg lacking or restrictive. I’ll probably try it out in a few years

                                                                                      2. 1

                                                                                        I would like to point out that choosing NixOS as a daily driver has the downside of requiring you to learn an entire new ecosystem to be able to figure out why something went wrong, or how to make it do what you want.

                                                                                        This means that you end up having to learn both nix the language, which is relatively easy, as well as nixpkgs the packaging library, which I’ve found to be ridiculously and immensely difficult. The documentation is still lacking (for example, but has improved over time), as is any easily accessible tooling for discovering what can be done that isn’t “read the entire source for this package and trace how it would get built”.
                                                                                        (There’s also the NixOS configuration library that I have not yet needed to understand the internal workings of.)

                                                                                        However, the community is pretty much always happy to help out, and asking people has always resulted in useful (and sometimes explanatory) answers.

                                                                                        So in the end NixOS is what I use. Perhaps it’s just that I’ve gotten used to digging through nixpkgs to fix my problems and don’t know how other packaging systems work, or maybe I’m hoping that one day I’ll actually rely on all of the benefits that I’ve supposedly been given, but I’m sticking with it for the time being.

                                                                                        Additionally, I would assume that Guix needs a similar commitment to use, but at first glance (from a while ago, when I last looked at it) it seemed there was more effort put into documentation. However, I’m not well versed in Guix to any extent, and so I don’t have anything knowledgeable to say about it.

                                                                                      3. 3

                                                                                        I just want a well designed OS that … run proprietary Linux binaries.

                                                                                        Then you are not a match for openbsd.

                                                                                      1. 31

                                                                                        I have recently gone down the rabbit-hole of note-taking apps, since none of them seem to meet my criteria:

                                                                                        • works offline with my local files (ideally in a human-readable format, if not compliant Markdown)
                                                                                        • excellent math support (see my prosemirror-math extension for an example)
                                                                                        • wysiwym (this is crucial for documents with lots of math)
                                                                                        • support for tags + bidirectional links, for easy categorization and interlinking between notes
                                                                                        • citations
                                                                                        • custom css themes
                                                                                        • free and open source, for extensibility

                                                                                        Here’s a summary of the different workflows I’ve tried over the years:

                                                                                        • Initially, I simply took notes on paper. Writing math was easy, but it was too difficult to stay organized or to make changes later.
                                                                                        • Next, I switched to a system where I’d take rough notes on paper, then polish them into a nice LaTeX document with overleaf that I could reference later. This works out well for some things, but when I’m writing LaTeX I feel to pressured to make everything look pretty. This also isn’t ideal because of the lack of links, difficulty inserting images, etc..
                                                                                        • For a while I used Jupyter notebooks, since they enable a nice mix of code / math / markdown. Eventually it just grew too cumbersome to start a notebook server every time I wanted to write. (however, these days I think notebooks are built into VS Code–so maybe it’s better).
                                                                                        • Next, I started using Typora for rough notes, and I’d eventually synthesize the most important ones into a LaTeX document with Overleaf. This was fine, but I had a lot of trouble with organization. Typora doesn’t support anything like tags / wikilinks.
                                                                                        • Next, I started using OneNote to keep a daily work journal, in linear order. If I’ve forgotten something, I can usually remember what else I had going on the same month/week, so having everything in a linear order really helped when I wanted to search over my past notes. It also helps remind me of my thought process when I go on a long depth-first tangent.
                                                                                        • Unfortunately, OneNote has terrible math support. So at this point, my notes were spread betweeen paper, OneNote, Typora, and Overleaf. I had no idea where to look for the most “up to date” version of anything.

                                                                                        When the pandemic started, I found myself with a lot of free time, so I decided it was time to make my own note-taking app called Noteworthy! I’ve been using it exclusively for my notes the past 3-4 months and it’s almost ready for public release!

                                                                                        In the process of making Noteworthy I’ve been inspired by all the other great note-taking apps out there. Here are just a few of my favorites:

                                                                                        • Typora, a nicely polished Markdown editor – has the best support for math input I’ve seen
                                                                                        • Obsidian, a split-pane Markdown editor focused on bidirectional linking
                                                                                        • Zettlr a Markdown editor focused on publishing / academics
                                                                                        • RemNote converts your notes into spaced-repetition flash cards, similar to Anki
                                                                                        • foambubble, a family of VS Code extensions to help search + organize your notes
                                                                                        • logseq, a GitHub-hosted alternative to Roam
                                                                                        • Neuron Notes a neat Zettelkasten system written in Haskell, based on GitHub repos
                                                                                        • R Studio includes an awesome Markdown publishing experience, similar to Jupyter Notebooks
                                                                                        • (coming soon) Athens Research, an open-source alternative to Roam
                                                                                        • (coming soon, made by me) Noteworthy, which aims to be an extensible, open-source alternative to Obsidian and Typora, with a focus on wikilinks and excellent math support

                                                                                        Some honorable mentions:

                                                                                        • Dendron, a hierarchical note-taking editor based on VS Code
                                                                                        • kb, a minimal text-oriented command-line note manager
                                                                                        • Notebag a minimal Markdown app with tag support
                                                                                        1. 4

                                                                                          I would add to your list Joplin, which I’ve had very good experiences with. I think it ticks a lot of your boxes, and it also has quite good mobile support which can come in handy.

                                                                                          1. 4

                                                                                            I can’t believe I left out Joplin! It’s multiplatform (mobile. pc, terminal) has more features than practically every other note-taking app out there.

                                                                                            It’s been a while since I last tried Joplin, but I think I remember choosing not to use it for my own notes since it uses split-pane (code+rendered) editing instead of wysiwym, which isn’t ideal for notes with lots of math. I believe there’s also no support bidirectional links, but I could be misremembering.

                                                                                            1. 4

                                                                                              I currently use Joplin, which works decently well for me. I have a few complaints about it: it’s a heavy Electron app that doesn’t exist in my distro’s package manger, so I have to build it from source. I don’t like the distinction it makes between “notes” and “notebooks” - I wish that notes could have arbitrarily-deeply-nested children, like some other notetaking software I’ve used has had. I do appreciate that it has a mobile app, but I’ve run into a few useability nits inputting text in that mobile app. And I wish there was a browser version.

                                                                                              This Noteworthy project looks interesting, and if can solve some of these problems better than Joplin, while still doing everything that I do like out of Joplin, I would consider switching to it.

                                                                                              1. 4

                                                                                                I wish that notes could have arbitrarily-deeply-nested children

                                                                                                A quick skim of the Joplin site didn’t really make it clear what a “notebook” is – are you just talking about being able to easily define hierarchies of notes? Or do you mean a full-on, infinitely-nested-list style app like Athens / Logseq / Roam, where every list bullet is considered to be a separate note? And where all the notes are connected as a big graph?

                                                                                                With Noteworthy, the goal is not to impose too much structure on your notes – you shouldn’t have to change how you think just to work with a new app. I decided that an approach based on tags would give the most freedom, similar to how Obsidian does it.

                                                                                                • (done!) Include tags anywhere using [[wikilink]] , #tag, or @[citation] syntax.
                                                                                                • (done!) easily search for all documents referencing a specific tag
                                                                                                • (done!) By default, filenames are tags and tags are filenames! Each file can additionally define a list of aliases, which allows for e.g. an abbreviation and its expansion to point to the same file.
                                                                                                • (planned) define your own tag hierarchies for easier search / disambiguation
                                                                                                • (planned) use logical operations (and/or/etc) in tag searches

                                                                                                I’d like to experiment with a Datalog-esque syntax for tag search as well. Roam and Athens both use Datalog internally to facilitate searches, and I believe it has worked out well for them. It would be super cool to expose some kind of tagging system based on predicate logic to the user.

                                                                                                This Noteworthy project looks interesting, and if can solve some of these problems better than Joplin, while still doing everything that I do like out of Joplin, I would consider switching to it.

                                                                                                What would you say are your most-used features? Regarding Electron vs native vs browser,

                                                                                                • Noteworthy is an Electron app, and I’m trying to keep it as lightweight as possible. This is mostly by necessity, since projects like KaTeX and ProseMirror have no native counterparts, afaik. I’ll happily re-write it if an alternative emerges – I’ve been following React-Native and Rust GUI for that reason. I also plan to delegate some of the heavy lifting to Rust programs like ripgrep.

                                                                                                • Browser version – I can definitely imagine Noteworthy running in a local server, accessible through a browser.

                                                                                                If all goes as planned, there will be a public beta of Noteworthy in a couple months, where I’ll try to gather feedback about what’s working / what’s missing. Keep an eye out :)

                                                                                                1. 1

                                                                                                  In Joplin a notebook is a collection of notes, which map to single markdown files. Notebooks can be arbitrarily nested, but a notebook can only contain notes, not have raw text associated with it. So there’s effectively two types of node in the tree structure it exposes to you. I would prefer it if you could have a tree of notes, all of which contain text, and may or may not have any kind of nesting under them.

                                                                                              2. 1

                                                                                                IIRC, they have an editor that lets you edit the markdown as it is rendered (one pane). I think this feature is still experimental though.

                                                                                                I’m not sure what you mean by bidirectional? In the sense that linking from node A to note B also creates a back-link in note B to note A? That’s not a thing in Joplin to my knowledge.

                                                                                                I’ve got the skeleton of a graph viewer inspired by Obsidian which talks to Joplin over it’s REST API, but it’s not currently working and I haven’t had the time to finish a PoC yet. I’m far enough into it to determine that creating such a companion app is definitely do-able – Joplin’s API is quite nice.

                                                                                            2. 1

                                                                                              This is a great resource, thank you.

                                                                                              1. 1

                                                                                                I think org-roam fill all your checkboxes.

                                                                                                The author answered in this thread here

                                                                                                1. 1

                                                                                                  I haven’t personally tried org-roam due to a phobia of emacs, but it looks like a great alternative to the other roam-likes. One thing that’s not clear – does it support some kind of instant math preview?

                                                                                                  1. 2

                                                                                                    Due to how ridiculously extensible emacs is, you can be certain that the answer will be “yes, with some elisp”.

                                                                                                    1. 1

                                                                                                      and if you want a packaged solution, org-fragtog :)

                                                                                              1. 1

                                                                                                Nice post! Where/how do you store the passphrase for automated backups?

                                                                                                1. 2

                                                                                                  Thanks! The systemd unit that does the backups talks to pass to get the passphrase. It in turn relies on gpg-agent to not have to ask to unlock the password store. This works for me because I do backups during the day and my email client keeps the gpg-agent awake.

                                                                                                  1. 1

                                                                                                    Aren’t you stuck in a chicken and egg problem ? You encrypt your backups using a password, saved in a store. If you loose your whole $HOME, how do you recover ? You need the password, which itself need a gpg key, which is backed up, but encrypted right ?

                                                                                                    Or maybe you backup your gpg keys and password store using other means ?

                                                                                                    1. 2

                                                                                                      Not certain if this is what they meant, but I assume the idea is that they both memorize the passphrase (in case recovery is needed), and also don’t want to keep typing it in for automated daily backups.

                                                                                                      1. 2

                                                                                                        Yes, except 1password has it memorized for me.