1. 6

    Still polishing off my followup to Password Generation in Ruby and Rust. Hopefully get it out over the weekend.

    I might bash some more on my new email search tool, try to massage it into something that’s at least minimally useful - it’s more a proof-of-concept right now, albeit quite a fast one.

    I’m also helping out a teacher sort out forms for quizzing students. He’s currently using 123FormBuilder and it’s a bit like the website equivalent of Kowloon Walled City.

    1. 10

      This might be a stupid question, but why not less instead of cat? Cat is not really meant to display files but to concatenate them. I’d definitely like a better pager, but neither w3m or vi in view mode worked for me, so i’m still using less

      1. 5

        Cat is still cat. Bat is like the kitchen sink that just clobbers more? Yeah, I don’t quite understand why this tool is positioning itself relative to cat.

        It is definitely not a clone. But I am all for new, more usable terminal based tools that use what we have at our disposal, more cores, more ram, ssd read/write.

        I’d really like a tool that built an n-gram inverted index of all files below my current dir and allowed me to jump to anything, that showed similar token/phrases/lines, etc. All terminal driven, with an option to load a rich GUI over a local http connection.

        1. 3

          Although I agree with you, I can see why this would be positioned as an alternative to cat.

          Quite a lot of people use cat to preview files, and that’s what bat does. I know less and more exist, but for some reason I still find myself using cat. Perhaps other people do the same.

          1. 4

            I use cat because, if I’m lucky, the output will fit in my terminal, and I’m ready for my next command; with a pager, I need to press a key to exit out of the pager before I can issue my next command, and the output disappears (in the case of less) as if the pager never ran, so I can’t keep context around.

            1. 11

              By the way, less can be configured to act well in these circumstances. I have more as an alias for less -FX. Those two options to less are:

                     -F or --quit-if-one-screen
                            Causes less to automatically exit if the entire file can be displayed on the first screen.
                     -X or --no-init
                            Disables sending the termcap initialization and deinitialization strings to the terminal.  This is sometimes desirable if the deinitialization string does something unnecessary, like clearing the screen.
              

              I also define $PAGER to be less -FX, so manpages and the like don’t clear the screen once the pager quits.

              1. 5

                I second this. -c would be helpful in $PAGER as well so that everything above on your screen stays untouched.

                Personally, I’ve been rolling with this:

                $ type le
                le is an alias for 'less -FcmNqX --follow-name'
                
              2. 2

                the output disappears (in the case of less) as if the pager never ran, so I can’t keep context around.

                If you want to get rid of this sort of behaviour globally, disable “alternate screen” in your terminal.

                In tmux, set-window-option -g alternate-screen off. In putty there’s Disable switching to alternate terminal screen under Terminal → Features.

                1. 1

                  Just tested bat, and the output doesn’t disappear. When you need a specific section from a file (not the whole thing) - using bat over cat and not less makes sense. Neat.

              3. 2

                Bat will act like cat if it determines there is no tty. Thus, bat is like less when used interactive and like cat when scripting.

                Like someone else said, people use cat to dump contents to the terminal so they can refer to it while continuing work.

                1. 2

                  Proof!

                  Oh.. you can also use it to concatenate files 😉. Whenever bat detects a non-interactive terminal, it will fall back to printing the plain file contents.

              1. 2

                I’m going to try and finish off my followup to Password Generation in Ruby and Rust, showing how easy it is to make small changes that significantly improve performance.

                I might also spend some time on Gutenberg #381, making external link checking more usable and robust.

                Or maybe I’ll just build more terrible hospitals.

                  1. 2

                    I wrote a fast cat once out of actual necessity.

                    I was working with an embedded system where, for installation/testing purposes (details forgotten), we had a service running in a VM that would help bootstrap a system. Part of this setup was a script that would run on the target, read a binary blob from a file descriptor and write it to the device. That reading was done with the target system’s implementation of cat.

                    As it happened, the sending of the binary blob to the target turned out to be a significant bottleneck. After I profiled it, it turned out that cat was taking a long time to read and write the bytes. I looked at the source and found it was using fread and fwrite. I changed it to use read/write and the transfer time went down significantly. It was great because I’m fairly certain this was part of our automated build system to create images for the devs and the result was that build times went down a lot.

                    So sometimes you really do need a faster cat.

                    1. 2

                      Why were fread/fwrite slow? Aren’t they thin veneers over read/write that do a bit of buffering?

                      1. 2

                        For “buffering”, read “copying”.

                        With read/write, it’s just copying into a buffer (read), and then copying out of that buffer (write).

                        With fread/fwrite it’s copying into a buffer (read), then copying from that buffer to another buffer (fread), then potentially copying that back (fwrite), before finally copying it back into another buffer (write). That can really add up - even the read/write loop is more than we’d want ideally, hence stuff like splice.

                    1. 1

                      I’m currently on a Matias Quiet Pro, because I wanted something fairly quiet with NKRO and a decent Alps-feeling switch. They’re not quite as nice as while Alps, but they are meaningfully tactile and the “click” is nice and high.

                      My previous keyboard was a ~30 year old NTC KB-6153EA with white clicky Alps switches, which are just lovely. They make Cherry MX Blues feel and sound like complete junk.

                      I also have a slightly worse-for-wear IBM M122 which I plan on building a converter for just for the sake of it. Who can resist the prospect of 24 F-keys?!

                      1. 2

                        Unicomp PC 122 is a modern USB-supporting version of the M122 5250 keyboard and is readily available.

                        1. 1

                          Not in the UK it’s not. And where’s the fun in that?

                          1. 1

                            Very true!

                      1. 8

                        Neat. I wrote a Rust version, which involved a bit more yak shaving.

                        1. 3

                          And it’s now a crate.

                          1. 2

                            Thanks for that. It felt weird asking for a crate here, so thanks for closing my issue!

                        1. 6

                          Elegant! The simplicity is really impressive, a real 80/20 kind of solution.

                          Maybe you could solve the pipefail thing by having a tts utility that invokes the target program as a subprocess, capturing its stdout+err and then when it stops, wait4() it and return with the same error code the child did.

                          1. 4

                            Added to rtss - stderr of the child still goes to stderr, so redirection works as you’d expect.

                            1. 2

                              Nice. ❤

                          1. 1

                            Looking forward to trying TruffleRuby with FreshBSD. Not holding my breath for a FreeBSD build, though.

                            1. 1

                              FreeBSD is stuck on Java 8 for now (the 9 porting work has started but seems kinda abandoned), and all the Graal stuff really wants Java 9/10…

                            1. 4

                              I’ve been learning Rust, with gcstool being the budding result - a utility for creating compressed databases for set membership tests. Main motivation being so you can check against leaked password databases without having to call out to a third party API or burn gobs of SSD/memory.

                              The 500 million-strong pwned passwords 2.0 list weighs in at just 1.6GB with 1 in 50 million false positives - a good improvement over bloom filters, which would need more like 2.15GB for the same error rate.

                              As part of this I also wrote LineReader, an alternative to BufReader::read_until that cut about a minute off slurping in the database on my dinky little Xeon.

                              I’ve been quite impressed with Rust so far - the language is rather nice and the tooling is excellent. I can see myself doing a lot more with it.

                              1. 3

                                I’ve been using auto_cd with tcsh and zsh for nearly 20 years now, having used it as the default behaviour on AmigaOS for a good half-dozen more. Who’s still typing cd? :P

                                1. 7

                                  Does anyone have pointers to background info on the issue with FreeBSD/NetBSD needing an instruction emulator, for something related to console I/O? Seems like they must be doing something fairly unique there if OpenBSD was able to emulate all these other OSs without needing an instruction emulator, but now needs one for those two.

                                  1. 1

                                    Wild guess: it’s the rep prefix.

                                    1. 3

                                      I think it’s just the outsb part - copies data from memory to an I/O port, which is obviously not “real” in a VM so the hypervisor needs to do something with it.

                                      For instance, see how bhyve handles it:

                                      • vmm/intel/vmx.c – EXIT_REASON_INOUT saves some state and sets exit code to VM_EXITCODE_INOUT.
                                      • vmm/vmm.c vm_run() on VM_EXITCODE_INOUT calls vm_handle_inout():
                                      • vmm/vmm_ioport.c - all about emulating IO port calls.
                                      1. 3

                                        I’m sitting next to mlarkin at AsiaBSDCon and he says “uhh, no, Ted”. :) (Note this went through several edits to come to “uhh, no” so as not to offend other lobsters.) ;)

                                    1. 2

                                      Use +"" or "".dup rather than String.new, unless you specifically want a string with binary encoding – the former will use the interpreter default, the latter will not.

                                      1. 5

                                        I gather INVPCID is the important one. PCID was added in Westmere (2010), but INVPCID, which allows for selectively invalidating TLB entries, arrived a few generations later in Haswell (2013). Without it the Linux workaround at least just defaults to the no-PCID path.

                                        Rather annoying, all my main local servers are Westmeres.

                                        1. 1

                                          I’ve seen conflicting information on this, some people claim that Linux demands INVPCID for the performance problem mitigation but some people are saying that PCID without INVPCID still takes a faster path on Linux than a processor without both features would.

                                        1. 10

                                          The Java/Microsoft mainstream is gaining new traction. Aesthetics also got diluted. Careful engineering is getting diluted. Social networks ubiquity and the easy path of quick cloud deployment is making people less and less patient. Crafting requires patience. People crave Likes, a new generation of vanity metrics arose.

                                          This is… baffling. My experience with Rails projects is that they accrue layer after layer of monkey patches, hundreds of ill-advised gem dependencies, excruciatingly slow CI builds, “surprising” bugs when monkey patches conflict with other monkey patches, poorly thought-out method_missing metaprogramming, and model classes that span thousands of lines. Aesthetics? Careful engineering? Crafting? Those are not things I associate with Rails.

                                          1. 8

                                            I’ve been building rails apps for around 7 years at different places, and only one codebase I worked on was well factored and approachable. I’d say “good” Rails only happens when you work really hard to design as much of the domain as possible outside the framework.

                                            1. 3

                                              I’d say “good” Rails only happens when you work really hard to design as much of the domain as possible outside the framework.

                                              I agree, but think that is as it should be, and that it is true for most frameworks when building beyond prototype/demo projects or scope-restricted services.

                                              I have found that framework-based projects, whose teams treat said framework as a scaffold to build within, rather than tools that collaborate with their own domain, are those that wind up a twisted mess. Rails or otherwise (though the larger the framework, the quicker this takes hold). The Ruby language also makes this easy for people who have never seen a mess before, nor introspected on how to avoid it.

                                              I sometimes suspect that the choice of the term “framework” has something to do with it. If a team was instead introduced to four “libraries” named ActiveRecord, ActionPack, ActionMailer and Rack, I wonder how that would influence their use in a project.

                                              Stemming from that is my other common thought: the architectural fragility of many long-lived Rails applications is made worse by the way it is taught (especially to beginners). I like Ruby, and most of what makes Rails useful comes from Ruby. However, Ruby never seemed to me a good first language, never mind one to learn from within the faux walls of some framework. Unless you know how Rails is doing what it does via Ruby, it is hard to see the forest for the trees, and to grasp and see how you might plant your own.

                                              Josh Susser once said something to the effect “there are three eras of building a project in a framework”, and went on to describe four, which I will roughly paraphrase from memory (I tried but could not find the talk it was from):

                                              1. The framework is a huge productivity multiplier and everything is awesome!
                                              2. The framework is still helpful, but it’s influencing your architecture too much and you’ve bumped into a few places where it just doesn’t fit.
                                              3. The framework is actively interfering with your productivity.
                                              4. The if-you-live-to-see-it fourth era: you dig yourself deeper by building a framework on top of the framework.

                                              I personally feel there is a “Good Ending” version of the fourth era: You heed the pain felt in the third era, and learn to build with the framework rather than build within it.

                                              1. 4

                                                The culture and rhetoric of Rails also works against this. Rails wants to own everything. Building alongside the thing feels wrong, and a lot of people who you hire as Rails developers will argue that it is wrong, and work against you. Ruby doesn’t help very much, either, because there’s no real namespacing and no way to contain the massive monkey-patches like ActiveSupport.

                                                If you have a lightweight framework, it’s a lot easier to treat it as the presentational layer and not let it infect your business logic. Rails actively works against you.

                                            2. 3

                                              Aesthetics? Careful engineering? Crafting? Those are not things I associate with Rails.

                                              What, you don’t like magic superglobals or using spooky action at a distance to suppress callbacks?

                                              Remember, kids, it’s not global mutable state if you make it thread-local and wrap it in a constant. Sprinkle liberally.

                                            1. 2
                                              1. 8

                                                This was one of the key points for kettles in the Ecodesign report a few years ago (alongside turning off when the water is boiling, being insulated enough that they don’t waste much of the heat they make warming the room, and having a clear indication of how much water is in them) - that they should have a reasonable design life. 7 years was the suggestion, I think.

                                                British press presented it as “EU TO BAN HIGH POWER KETTLES”. Because of course they did.

                                                I wonder how much they’ll bother to spin this negatively now we’re leaving.

                                                1. 12

                                                  If you’re on FreeBSD, you can use mac_portacl(4) to permit specific users or groups to bind to privileged ports. e.g. for a webserver you might have this in your /etc/sysctl.conf:

                                                  net.inet.ip.portrange.reservedhigh=0
                                                  security.mac.portacl.rules="uid:80:tcp:80,uid:80:tcp:443"
                                                  

                                                  pf’s also capable of enforcing rules based on a socket’s owning user and group.

                                                  1. 1

                                                    Bingo, this is the sane way to solve it. You really don’t want to kill privileged ports entirely unless you’re ok with someone taking over port 22

                                                  1. 2

                                                    filemon(4) looked neat, so I made a Ruby interface to it. Because why not.

                                                    Highlight: making a pointer to a stringified representation of an integer: ptr = [[fd].pack('I')].pack('P').unpack('j!').first. Eww.

                                                    1. 2

                                                      This looks cool, but also quite similar to HCL: https://github.com/hashicorp/hcl

                                                      They’re both basically curly-braced nginx-style config languages that evaluate to JSON.