1. 43

    I would guess that a very substantial proportion of the people who read Lobsters have heard of Timsort.

    1. 22

      That, and TimSort really only makes sense in certain situations:

      • Comparisons have to be extremely expensive. Think dynamic dispatch. This is because TimSort itself performs a bunch of branches and integer comparisons to keep track of galloping scores and the stack invariant.

      • You need a stable sort. If you don’t need a stable sort, pattern defeating quicksort will probably do better.

      1. 11

        Quicksort is the Achilles of sorting algorithms: unbeatably fast, easy to implement, in-place; but with the vulnerable heel of bad worst-case performance (the worst case being pre-sorted data in the naïve implementation) and instability.

        1. 5

          There’s a fairly easy fix to that, called introsort: start with quicksort, but bail out to a guaranteed O(n log n) sort like heapsort if it takes too long. In the bail-out case, you lose constant-factor performance compared to if you had used heapsort in the first place, but you avoid quicksort’s O(n^2) worst case, while still getting its good performance in non-pathological cases. It’s used in practice in .NET and some C++ STL implementations.

          1. 3

            Quicksort -> Heapsort is method I used. It worked fine in practice. I love solutions like that. Another, unrelated one was sklogic’s trick of using a fast, dumb parser first to see if it’s correct. If it wasn’t, he switched to one that made error messages easier.

            I bet there’s more of this stuff waiting to be found for situations where people are shoving every case into one algorithm.

        2. 4

          Comparisons have to be extremely expensive. Think dynamic dispatch.

          That explains why Python uses it as it’s standard sort.

          1. 9

            Yeah. That’s exactly why Python uses TimSort.

            More tellingly, where Rust uses an algorithm that’s related to TimSort for its stable sorting algorithm, they didn’t implement “galloping” because it’s not worth it. https://github.com/rust-lang/rust/blob/7130fc54e05e247f93c7ecc2d10f56b314c97831/src/liballoc/slice.rs#L917

        3. 10

          I consider myself relatively knowledgeable about many different topics of CS and had not heard of Timsort until this article. What’s the point of your comment? That the article is not worth posting as you presume that it is widely known?

          1. 4

            The point is that the title of the article, and of this submission, is inaccurate. I would call the title clickbait because for most readers, the article doesn’t deliver what it promises – a sorting algorithm “you’ve” never heard of. I think the article itself is fine; it’s just the title that is a lie.

            1. 5

              That seems to be a really low-value comment. For whom is the remark actually intended? For other crustaceans to look at, nod and agree, thinking, “yes, I too possess the superior knowledge”? Does every submission with a title that invokes “you” need to be correct and ‘deliver on its promise’ for all possible “you”s? C’mon.

              1. 4

                Yes, I suppose jgb could have been more explicit in why they brought up their guess. (I wrote an explanation of my interpretation of the comment, not the original comment.)

                Does every submission with a title that invokes “you” need to be correct and ‘deliver on its promise’ for all possible “you”s?

                I think every article with a title that invokes “you” needs to be correct and ‘deliver on its promise’ for the majority of possible “you”s in its audience. If a title says “you’ll love this” and most readers don’t love it, the title was wrong, and it wasted people’s time by getting them to open the article on false pretenses. It is up to article authors to adhere to that principle or not.

                As for titles of submissions of articles with clickbait titles, there can be a conflict between submission titles that reflect the author’s intent and titles that accurately describe the article. I don’t have a simple answer as to when submission titles should differ from the article title.

                1. 3

                  I think every article with a title that invokes “you” needs to be correct and ‘deliver on its promise’ for the majority of possible “you”s in its audience.

                  I think I agree with this, and I think my concern comes down to disagreeing instead with the notion that the majority(/“a very substantial proportion”) of Lobsters readers have heard of Timsort. Short of a poll there’s not an actual answer to that; I just felt particularly rankled because I hadn’t, and presumably if I had I wouldn’t have bothered or thought to comment myself.

                  I err on the side of preserving the article title in the submission, which I think is pretty common. Accordingly, I think most Lobsters are primed to see submission titles that aren’t necessarily addressing them as Lobsters readers, but in some context that might be quite removed.

          2. 2

            I thought it played a pretty big role in the Oracle vs. Google lawsuit too, making it one of the more famous algorihtms.

            However see “rangeCheck” mentioned a lot, which is a trivial part of TimSort.

            https://en.wikipedia.org/wiki/Oracle_America,_Inc._v._Google,_Inc.

            Here it seems to cite TimSort. But for some reason I can’t find a lot of sources that talk about TimSort and the lawsuit, even though at the time I remember it being a prominent thing.

            https://forums.appleinsider.com/discussion/149435/google-engineers-defend-source-code-email-in-oracle-lawsuit-over-java/p4


            edit: here’s another one mentioning TimSort and the lawsuit.

            https://majadhondt.wordpress.com/2012/05/16/googles-9-lines/

            Googling “rangeCheck timsort lawsuit” digs up some results.

          1. 3

            I like the idea of the unit testing feature, as can be seen in the time.rpl example:

            hour =  { [01] [0-9] } /
                    { "2"  [0-3] } 
            -- test hour accepts "00", "15", "20", "23"
            -- test hour rejects "", "0", "5", "24", "99"
            
            …
            
            rfc2822_time = { hour ":" minute {":" second}? }
            -- test rfc2822_time accepts "00:00:00", "12:00", "23:59:60", "16:08:33", "01:55", 
            -- test rfc2822_time rejects "", "24:10:10", "33:40", "17:28:33.0", "00:00:00.00000", "10:30:02.125"
            

            I’m not convinced that Rosie Pattern Language is worth the complexity it adds compared to using a PEG parsing library for my preferred language. But I don’t know of any language-specific parsing library that makes tests-as-documentation as easy to write as the above. It makes me wonder how to port that feature to those other libraries.

            1. 1

              This website has already been posted three months ago, though not that part in particular. Is this a new part of the site?

              1. 1

                The Sidebar page existed three months ago; it is not new. Evidence: this third-party Sidebar implementation written the day of that post.

              1. 6

                According to @pushcx two months ago, there is currently no page that lists all of your comments: https://lobste.rs/s/cgtc2w/happy_7th_anniversary_lobsters#c_hfknb5.

                Approaches for implementing this feature

                Around the time of that thread, I looked into how to edit the Lobsters source code to add this feature, though I didn’t get as far as a patch. The “Your Threads” page runs the CommentsController#threads action. That action includes a call @showing_user.recent_threads(20, …, …). In that call to #recent_threads, 20 is the amount argument, which is used as a LIMIT clause in the query. So one way to implement this would be to paginate the “Your Threads” page and change User#recent_threads to take an offset as well as a limit.

                Another way to allow seeing your old comments would be to introduce new UI to the Comments page, which is different from Your Threads in that it shows comments without the context of the whole thread, and it already supports pagination. The /comments and /comments/page/2 URLs could be made to accept a user parameter like /comments/shawn and /comments/shawn/page/2, similar to the existing /threads/:user route. The user parameter would filter the displayed comments to only those by that user. Implementing that would require changing CommentsController#index to filter the comments if that parameter exists. UI to set that URL parameter would be nice but probably not necessary for the first version.

                I stopped short of implementing one of these approaches and making a pull request because I ran into difficulties setting up a local version of Lobsters to test changes on. If I get Lobsters running locally, there is another, smaller code change I would contribute first, to make sure I have the process down. So if anyone wants to try implementing one of the approaches above, please, go ahead.

                Can any Lobsters maintainer comment on whether the “paginated full comment history” feature would be accepted if a pull request were made? Right now other users’ comment history is limited to their last 20 comments – that isn’t a deliberate design decision, right?

                1. 3

                  Can any Lobsters maintainer comment on whether the “paginated full comment history” feature would be accepted if a pull request were made? Right now other users’ comment history is limited to their last 20 comments – that isn’t a deliberate design decision, right?

                  I do not understand limiting comment history to the last 20 comments to be a deliberate design decision, and I have from time to time wanted to look further back in my own comment history. Consistent with the PR being of sufficient quality to merge I don’t see a prima facie reason this specific feature would be rejected.

                  1. 2

                    IIRC the issue stems from a desire to Do Pagination In Lobsters Correctly And Consistently Errywhere, but my data is at best many months old.

                  2. 2

                    Thanks for the detailed and correct post. The page has always had the last 20 threads a user has commented on, and the lack of pagination is not considered a feature.

                    @friendlysock is correct below that this is a “perfect is the enemy of good” thing. I attempted a fix with pagy (seemed appropriately small with a straightforward API) but it had a bug with bad user input the maintainer wontfixed saying it’s OK if a site 500s because a user edited a URL. I’d also like the pagination to be stable, so oldest stuff has the lowest page number and doesn’t change page number as new stories/comments come in. I dunno how to handle the latest pages like /comments when they aren’t a “full page” of 20 comments, though, as having that page list one or two comments is lousy and having it list 20-39 is weird. So I figured I’d like it simmer in the background and see if I had any better idea in a few more months.

                    1. 1

                      I have discovered an existing GitHub issue about this feature: #394 “Feature: paginate all comments”

                    1. 8

                      There are arguments made that social media causes harm to others (e.g. polarization, isolation, and anxiety). Futhermore, many believe they are harmed by a loss of privacy from the advertising strategies of Google, Facebook, and others. The EU has taken these concerns very seriously and the US has been in a heated debate on the role of social media companies in society. A fair amount of open-source contributors are employed by such companies.

                      The point I’m making is not whether social media and advertising causes harm, but that what causes harm is highly politicized and subjective. Software development is not the same as practicing medicine.

                      1. 0

                        It is true that harm is subjective, but that doesn’t seem to be a problem for this license as currently written. Subjectivity isn’t a problem for the license because it implicitly uses the software user’s definition of harm (not the licensor’s or anyone else’s).

                        How I came to that conclusion: the restriction on use of the software only applies to “systems or activities that actively and knowingly” harm someone’s well-being. In my interpretation, “knowingly” means that the user of the software must think they are causing harm for that rule to apply. So even if most people think that, for example, social media software is harmful, the license wouldn’t forbid a Facebook employee from using the licensed software as long as that employee personally doesn’t “know” that social media is harmful.

                        Of course, enforcing that could be difficult, because if you accuse the software user of causing harm and they respond that they didn’t think they were causing harm, there is no way to know whether they are telling the truth about their thoughts. But that’s a separate problem.

                        1. 2

                          In my interpretation, “knowingly” means that the user of the software must think they are causing harm for that rule to apply.

                          What you said may be textually correct, but it is easily weaponized. It can be argued that Google and Facebook are lying, like Big Tobacco. Sean Parker, a Facebook founder, said they knowingly created an addictive platform. Other executives and developers have made similar statements about ‘knowing’. There are also studies that can be cited. Therefore, it could be argued that such employees knowingly cause harm.

                          Again, I’m not making this argument. I’m just making a point.

                          If the open-source community advocates this license, many could argue that is not Hippocratic but instead hypocritical. The open-source community is sponsored to a great extent by social media and advertisers.

                          EDIT: ‘many could argue that this is not Hippocratic but instead hypocritical.’

                      1. 9

                        There was an argument on Twitter about whether this actually protects people: if I decide an oil company that’s harming the environment is using my software, can I yank it from them under this license? I’d be harming their economic well-being.

                        1. 12

                          What happens when this gets weaponizes by someone who doesn’t want to support homosexuality or companies supporting it. Same for gun control or health insurance or the issue of the day. I don’t like putting kids in cages but I worry that this carve out idea has major flaws.

                          What if someone decides that my Judaism and its Zionism should exclude me? I feel personally threatened by this.

                          1. 1

                            Sadly I think this is more likely than say, a human rights violating military giving a shit about it.

                            Strangely enough complying with licences is not high on the agenda for torturers.

                            I know from The Bad Old days of Apartheid the sanctions only applied to innocent governmental activities like water supply… the military honestly didn’t give them a seconds thought.. if they needed software or a machine or whatever they’d lie, steal, put up fronts conceal whatever.

                            1. 2

                              Many kinds of entity are able to ignore these clauses.

                              However, there’s at least one growing group that can’t: companies who are selling to foreign governments (eg US companies selling tech to the Chinese government).

                              The US government isn’t going out of its way protect them, and the Chinese government can’t offer much protection from US courts.

                              1. 1

                                US companies selling tech to the Chinese government

                                You mean US companies openly selling tech to the evil entity that is openly using it.

                                In the Bad Old Days, US arms dealers quite liked sanctions…

                                US government regulations don’t mean “you cant do a deal”.

                                They mean the deals have to be done “at arms length” with a bit of forgery in the middle.

                                Meaning the salesdroids get to clip the ticket several times instead of just once.

                                Unsurprisingly, arms dealers are not very nice people.

                                1. 1

                                  It sounds like you are saying (in a very roundabout way) “Yes, using this licence will increase the price paid by Bad Guys for technology, since they’ll have to do their deals at arms length”.

                                  1. 1

                                    That is an optimistic interpretation of what I’m saying, yes.

                                    In practice since they are breaking (or bending to near breaking) some laws…. they tend to break any that will cost money. (eg. Getting export permits, paying tax… etc.)

                          2. 3

                            The most relevant part of the license for that question would be this:

                            The software may not be used by individuals … for … activities … that actively and knowingly … harm … the … economic … well-being of groups in violation of the United Nations Universal Declaration of Human Rights.

                            But revoking a license is not using the software. The restriction doesn’t apply to the license holder’s activities of licensing. So my answer is yes, you could revoke the license in that situation.

                            (I am not a lawyer.)

                          1. 1

                            C++: I like the elegance of the idea that you can wrap { } around any group of statements to be able to write it where a single statement is syntactically required.

                            1. 13

                              I use the Fish shell’s abbr (abbreviation) feature for my typing-saving Git aliases. Because Fish replaces the abbreviation with the full command before the command is logged and run, I can more easily spot when I ran certain Git commands when I browse my shell session or search through my command history.

                              For example, after defining this abbreviation:

                              abbr --add gitst "git status"
                              

                              I can type gitst in my prompt:

                              $ gitst
                              

                              But after pressing Enter, my session looks like this:

                              $ git status
                              …
                              nothing to commit, working tree clean
                              

                              Then, if I leave my shell and go back to it later, I can see that I just ran git status. I don’t have to remember what gitst stands for.

                              Along the same lines, I can see the long form of flags in my history while spending as much time typing as if I had written their short forms. My gitspu abbreviation expands to git stash push --include-untracked, which is easier to read than git stash push -u.

                              1. 7

                                I recommend against implementing a drop-down as described in the article, even for simple use-cases. The resulting drop-down doesn’t support keyboard shortcuts such as pressing Tab to focus the drop-down, or pressing Space to open the drop-down and select a value, up/down arrows to select a value, or typing the first letter of a value to select it. This lack of functionality will slow advanced users down, and will prevent users with certain disabilities from using the page.

                                If the styling available for the built-in select and option is not enough, I recommend one of those existing “javascript libraries and css frameworks, which have own implementation for custom select dropdowns” mentioned by the article.

                                I would expect an article about implementing custom form elements to at least mention the technique of using a label to make your custom elements control an actual select element behind the scenes. With how useless this article is and how many ads are on the linked page, this submission is dangerously close to spam.

                                1. 1

                                  The work-around I found is console.log("x: " + JSON.stringify(x));

                                  1. 1

                                    If you need to explicitly convert the object to a string like this, the output will be easier to read if you pretty-print it. You can do this by passing JSON.stringify a value such as 2 or '\t' for its third argument, space:

                                    console.log("x: " + JSON.stringify(x, null, 2));
                                    
                                  1. 1

                                    Nice tip ! Still I don’t get what differs between the two ways of calling console.log. Does someone has a deeper explanation ?

                                    1. 3

                                      This console.log call logs a string:

                                      console.log("user: " + user);
                                      

                                      This string is defined by "user: " + user. The + operator with a string on the left implicitly calls .toString() on its right argument, user in this case. But the default implementation of toString for all objects, such as this user object, returns only the string [object Object]`.

                                      Both of the following console.log calls log an object:

                                      console.log(user);
                                      console.log({user});
                                      

                                      No strings are involved here. console.log doesn’t call toString to show user; it uses the JavaScript runtime to reflect on the object keys and values directly and shows a special UI for browsing them.

                                      As others have mentioned, you can also call console.log like this, which logs a string and then separately logs an object on the same line, with no + operator causing .toString() to be called:

                                      console.log('user:', user)
                                      
                                      1. 1

                                        That makes perfect sens to me, thank you for the anwser :)

                                    1. 6

                                      This is really interesting, thank you. For a long time I have been a user and sometime-contributor to IkiWiki, a static wiki site generator. The original/principal author decided that the default theme for the wiki should be basically no theme, an “anti-theme”. For new wikis I can understand (you need to configure a lot of things, so you might as well configure a theme too), but he extended this decision to the site that forms the “advert” and demonstration of the software itself, namely its homepage. But the anti-theme is so hostile to legibility I really felt that it was a bad decision and could be putting of potential contributors before they’d got started. Unfortunately nobody is sufficiently actively contributing to IkIWiki now and my appeal to revisit that decision, 18 months ago, has gone unanswered. I just posted Typesetter.css there as I felt it’s relevant to that discussion.

                                      1. 2

                                        I’ve just had a look and legibility for me was very high. This is on an iPhone 8 with Safari. It’s great to not have to press the ‘reading mode’ button on every site.

                                        What kind of browsers / screen sizes make it show legibility issues?

                                        1. 3

                                          I have a wide monitor and my eyes really can’t track a line of text that well over such a long distance. It also causes me to “miss” the next line when I scan back to the left.

                                          1. 1

                                            I see. I don’t make my browser full screen on wide displays - not because of this, because it’s rare, but because most sites do their own thing to keep the text to a reasonable width - and therefore there is a lot of wasted space.

                                            Back in the days of Netscape Navigator, I remember setting up the details so that sites without styling would look good to my eyes. If browsers support a default style sheet then it would be good if it had a max width to cater for this. I’d like to publish without CSS and let the user decide how they want content to appear (surely this is a good way to hell with accessibility?) but I’ve seen this problem before.

                                          2. 3

                                            If you’re talking about ikiwiki.info, then I find it illegible on my iPhone 7 with Safari, which (I think) is the exact same size and ppi as your 8. So to a certain extent it must be in the eye of the beholder.

                                            1. 1

                                              Your link goes to this Lobsters discussion page. Fixed link: ikiwiki.info

                                              1. 1

                                                Thanks!

                                        1. 17

                                          Another fun feature of ruby is the __END__ keyword. If that keyword appears alone on a line, ruby will stop parsing the file at that line. In the same file you can use the magic DATA variable to read the rest of the file. Eg:

                                          #!/usr/bin/env ruby
                                          puts DATA.read
                                          __END__
                                          Hello
                                          

                                          This program will output Hello\n when executed.

                                          This allows to easily create self-unpacking scripts for example, or to join data with code.

                                          1. 6

                                            This feature is present in Perl too.

                                            1. 3

                                              Perl’s documentation of its __END__ and __DATA__ special literals: https://perldoc.perl.org/perldata.html#Special-Literals

                                          1. 4

                                            When I worked with Clojure in Vim, I preferred vim-sexp over paredit.vim. I remember that Paredit forced that parentheses always be balanced. It clashed with my workflow of deleting a snippet of code (sometimes unbalancing parentheses) and then pasting it somewhere else in the file (making the parentheses balanced again). vim-sexp provides the same slurp, barf, etc. commands as Paredit without imposing that restriction.

                                            If you use vim-sexp, you will probably want vim-sexp-mappings-for-regular-people as well.

                                            1. 3

                                              I use both vim-sexp and vim-sexp-mappings-for-regular-people but I also use parinfer-rust. https://github.com/eraserhd/parinfer-rust It is by far the best application of parinfer I’ve come across. If you have tried other versions before and didn’t care for it I would suggest giving this one a go. It uses “smart mode” I think they call it, and it just works. I couldn’t stand the emacs version but this one made my transition back to vim so fun and simple.

                                              1. 3

                                                At least in Emacs, Paredit also provides bindings for deleting whole sexps, so you don’t end up with that issue. For instance, in

                                                (let [x 5
                                                      foo (fn [y]
                                                            (+ y x))]
                                                  (foo 8))
                                                

                                                if you position your cursor after the 5, you can do C-M-k (kill-sexp) twice and pull out both the foo binding and the fn expression onto the clipboard. It’s a bit of a different approach, but it means never having to do that cleanup work of balancing parens.

                                                1. 2

                                                  Yeah paredit is the thing I’m most likely to change. I’ve heard of vim-sexp but haven’t used it yet!

                                                1. 8

                                                  The comment field there doesn’t permit editting and correcting typos…..

                                                  So let me try again here…

                                                  In a galaxy far far away….

                                                  Larry Wall wondered why he needed to learn 3 pretty bad languages, sh, awk, sed…., and devised perl as the Grand Unifying Language.

                                                  Perl sadly borrowed too much from it’s inspirations, and wasn’t much more readable.

                                                  Then Matz came along and resolved to borrow the best from perl and scheme and ….. and make something more powerful than them all, yet more readable.

                                                  It’s called Ruby.

                                                  And yes, you can do everything in Ruby, in one line if you must, that you can do in bash, awk, sed, jq, perl…. but in a more powerful and maintainable form.

                                                  All this has been available for decades, why are we (still) bashing (pun intended) our heads against the Lowest Common Denominator?

                                                  1. 8

                                                    serious question: what does “doing some awk in Ruby” look like? This might be a pretty big motivator for me to finally figure out Ruby for scripting (I’m more of a Python guy myself but awk works nicely for small scripts on line-oriented stuff when I want a one-liner)

                                                    1. 8

                                                      Compare:

                                                      # Official way of naming Go-related things:
                                                      $ grep -i ^go /usr/share/dict/* | cut -d: -f2 | sort -R | head -n1
                                                      goldfish
                                                      

                                                      Versus Ruby:

                                                      puts(Dir['/usr/share/dict/*-english'].map do |f|
                                                        File.open(f)
                                                          .readlines
                                                          .select { |l| l[0..1].downcase == 'go' }
                                                      end.flatten.sample.chomp)
                                                      

                                                      Simple example, but I think it demonstrates that doing various basic and common tasks are quite a bit more complex to do in Ruby than in the shell.

                                                      That doesn’t mean I’m always in favour of shell scripts – I got that example from an article I wrote saying you shouldn’t use shell scripts – but there are definitely reasons shell scripting persists, even though we have things like Perl and Ruby.

                                                      In that article I wrote “I regret writing most shell scripts [..] and my 2018 new year’s pledge will be to not write any more”. I’ve mostly failed at that new years’ pledge, and have happily continued shelling about. I have started rewritting shell script prototypes to other languages at the first sign of getting hairy though, and that seems like a middle ground that is working well for me (I should update/ammend that article).

                                                      1. 5

                                                        To be fair, it looks like most of the additional complexity in the Ruby code comes from reading files: the first command in the pipeline, grep -i ^re glob, is what becomes

                                                        Dir[glob].map do |f|
                                                          File.open(f)
                                                            .readlines
                                                            .select { |l| l[0..1].downcase == re }
                                                        end.flatten
                                                        

                                                        The rest of the script contributes very little to the Ruby code.

                                                        I suspect this is a recurring theme when trying to replace shell pipelines with programs. Only Perl avoids some of this additional complexity for reading files, I think.

                                                        1. 5
                                                          puts Dir['/usr/share/dict/*-english'].
                                                            flat_map { |f| File.readlines(f).grep(/^go/i) }.
                                                            sample
                                                          
                                                          1. 6

                                                            At least with Ruby I don’t have to constantly cross-reference the man page and my cargo-culted knowledge of Unix’s multitude text manipulation DSLs, all unlike. It’s pretty obvious what it’s doing.

                                                            1. 1

                                                              Actually you used very little shell there in your first example.

                                                              You also used grep, cut, sort and head.

                                                              Why do you assume the backtick operator and the | operator for io doesn’t exist in ruby? In fact why do people assume shell and jq do not exist if you use ruby?

                                                              Personally I tend to reduce the number of tools involved to reduce the cognitive load of needing to understand each tool to understand the one liner.

                                                              I balance that against considerations like going IO.read(”|sort -u fileName”) can be a huge performance boost

                                                              Anyhoo… some examples of ruby onliners

                                                              http://reference.jumpingmonkey.org/programming_languages/ruby/ruby-one-liners.html

                                                            2. 7

                                                              Because code in sed or awk that worked a decade ago (or, hell, two years) still works. Ruby code seems to bit rot faster than any other language I’ve use for nontrivial work.

                                                              Also, with awk, I could put it down for a year, then use it again, and everything I’d need to be productive fits in a small man page. (The same seems to be true of sed, though I don’t use it much.) The Ruby ecosystem moves a lot faster, and if you haven’t been following it closely, catching up will add extra work. (Whether it’s actually going anywhere is neither here nor there.)

                                                              Yes, awk is a more limited language, but that’s a trade-off – there are upsides, and I know which I’d prefer.

                                                              1. 1

                                                                Not true.

                                                                The awk scripts I wrote decades ago with in Solaris awk which is not quite the same thing as gnu awk.

                                                                Well thought out growth in a language is good.

                                                                I find the maintenance burden in ruby rolling forward with language versions is very low.

                                                                Doubly so since rubocop will often autocorrect stuff.

                                                              2. 6

                                                                I don’t know Ruby. But for me these are the reasons why I am writing more and more bash programs:

                                                                • Bash is my command line. So I am doing a lot of small steps, file modifications, comparing, searching analysing. At some point I can see that some of the steps can be composed and I pull them out of the history, try them out on the console and at some point put them into a script. If Ruby would have a REPL in which I can do all the operations that I am doing on the command line with less typing and more comfort, I would maybe give it a try.

                                                                • Bash is on every Linux box. Ruby is not.

                                                                1. 4

                                                                  Ruby does have a REPL. It’s called IRB and it comes with every Ruby installation. I use it exactly as you describe, for composing small programs iteratively.

                                                                  1. 1

                                                                    Are you using the Ruby REPL as your daily all-time console or just when you have in mind to create a program? I am asking honestly because I do not know anything about Ruby or their REPL and I am quite interested how good this REPL is as a replacement for the daily life?

                                                                    My point is that shell scripts are a by-product of using the shell for doing manual tasks. And I get better and better at my shell usage, and even after 20 years of shell usage I am still discovering new features or ways to do something in a more efficient way. While the shell language is really ugly, but being very succinct plus the composition of unix commands, the history, the prompt customization, the possibility to have vi mode for editing (and I probably forgot a lot of features), all this makes using shell such an efficient tool.

                                                                    1. 2

                                                                      Well, no, not as my daily shell. I dislike shell scripting enough that I switch to Ruby pretty quickly if I’m having to spend any amount of time or effort on a task, but it’s not meant to be a replacement for bash/zsh/fish.

                                                                  2. 3

                                                                    Bash is on every Linux box. Ruby is not.

                                                                    Let’s not limit ourselves here. For those not using Bash and/or Linux, how about this:

                                                                    • Bourne-compatible $SHELL is on every Unix box. Ruby is not.
                                                                    1. 2

                                                                      Bash is on every Linux box. Ruby is not.

                                                                      So is ed.

                                                                      However sudo apt install ruby solves that problem.

                                                                      And yes, ruby does have a REPL.

                                                                      1. 2

                                                                        apt: command not found.

                                                                        sudo: permission denied

                                                                        $

                                                                        1. 2

                                                                          Have fun with ed then, it’s the Standard!

                                                                          https://www.gnu.org/fun/jokes/ed-msg.html

                                                                          1. 1

                                                                            I have written scripts in ed before to do some sufficiently tricky text manipulation. It’s a good tool.

                                                                    2. 5

                                                                      Mostly, because picking up enough jq, awk and sed to be useful is faster than learning the ins and outs of Ruby?

                                                                      I suppose you could make a similar argument about learning Ruby one-liners, but by the time I’m writing a very long bash script, I’m probably writing a larger program anyway, either in Go or Python. Ruby as a language doesn’t have much appeal to me, at least at the moment.

                                                                      Awk, at least, fits very nicely into a small space right next to regex. jq is a bit fiddilier to pick up, but very nice for basic stuff. Sed, I still don’t have down very well, but also is nicely regex adjacent.

                                                                      1. 3

                                                                        I regularly write sed one liners to do refactorings on my Ruby code. Usually the sed call is fed by the result of grep or find. I could write a Ruby one liner to do the same, but it would be a much longer line and escaping would be much more difficult. Ruby is simply not a replacement for the convenience of sed.

                                                                        And maintainability is a red herring here: the whole point of something like sed is that you use it for one-off commands.

                                                                        1. 2

                                                                          I’m not that experienced with jq, but when it comes to awk (and sed), one of their benefits is that you can easily write a program in the shell, since they act as glue between pipe operations.

                                                                          For example, to filter out all lines that have less than 4 characters, all you have to write is

                                                                          ... | awk 'length >= 5' | ...
                                                                          

                                                                          no imports or types required. It was made for stuff like this, which makes it easy to use. I’ve only read a book about Ruby a few years ago, but to process stdin/out this was should require a bit more overhead, shouldn’t it?

                                                                          1. 1

                                                                            One part of your history lesson is missing: Paul McCarthy and Steve Russell saw what was going to happen and pre-emptively invented Lisp. And yes, you can do everything in Lisp, in one line if you must, that you can do in bash, awk, sed, jq, perl… but in a more powerful and maintainable form.

                                                                            ;)

                                                                            1. 2

                                                                              s/Paul/John/

                                                                              This gotta be one of my most common brainarts…

                                                                              1. 2

                                                                                It was Yoko’s fault.

                                                                              2. 1

                                                                                Ruby equivalents of the basic awk and sed examples from the article, as examples of Ruby one-liner structure:

                                                                                • AWK: awk '{print $1}' logs.txt
                                                                                  • Ruby: cat logs.txt | ruby -ne 'puts $_.split[0]'
                                                                                  • Ruby: cat logs.txt | ruby -ane 'puts $F[0]'
                                                                                • sed: sed 's/^[^ ]*//' logs.txt |sed 's/"[^"]*"$//'
                                                                                  • Ruby: cat logs.txt | ruby -ne 'puts $_.gsub(/^[^ ]*/, "").gsub(/"[^"]*"$/, "")'
                                                                              1. 2

                                                                                What seems to be missing, in my opinion, is a book that is written for beginners in programming, that teach you programming while teaching you the language.

                                                                                There are many such books out there, at least for other languages than Clojure. When I took a programming class in high school, I learned a lot from the free online textbook we were using, Introduction to Computer Science using Java by Bradley Kjell. I think that book would be good for self-learners, too. I liked its style of asking questions at the end of each page to keep you thinking. The downside is that the book teaches Java, a language that is often cumbersome to write in – certainly more cumbersome than Clojure. But learning any second programming language is relatively easy after you have learned your first.

                                                                                Though it may be easier to find beginners’ books for languages other than Clojure, I acknowledge that switching focuses from your original choice of language may be a bad idea. It is easy to get caught in a trap of always hearing about a better way of doing something and switching to it before you have learned much, causing you to spend all your time reading introductory materials and no time actually doing the exercises and learning things.

                                                                                Given that you want to learn Clojure, maybe there are more beginner-oriented books you haven’t considered. I noticed there’s a book Getting Clojure by Russ Olsen that was published a year ago. Reading the chapter excerpts linked on that page and the reviews on Goodreads, it looks like that book might be basic enough to be suitable for beginners. You can read the excerpts yourself and decide.

                                                                                1. 1

                                                                                  Thanks for the time you took to write a post thorough like this. Yes, I agree that second language is relatively easy when you’ve learned your first, but I’m still struggling with my first :) Not for long, I hope.

                                                                                  The thing about switching languages and keep getting stuck reading introductory materials is what I got trapped in, I think. But that’s why I’m going to stick with Clojure and not get distracted by trying to do something in other languages, at least for now.

                                                                                  Getting Clojure is a book that several people recommended, alongside Clojure for the brave and true. I’ll definitely check it out.

                                                                                1. 2

                                                                                  Speaking of pages that list comments, is there any existing page that shows all of one’s own comments? I would like to be able to save all of my comments to my computer, and to search within my comments to find one I remember making about a certain topic.

                                                                                  Your Threads contains one’s comments inline with other comments, but the page ends with threads from a few months ago. Comments contains all comments, paginated, but I don’t see a way to filter it to a specific user’s comments.

                                                                                  1. 1

                                                                                    No, there’s currently no page for all comments by a user.

                                                                                  1. 4

                                                                                    I gave up on the captcha. Will this end at some point? I think I filled 4 or 5 of the captcha screens before I wanted to throw my PC out of the window.

                                                                                    The funny thing is how you can relate many components to modern web pages: Cookie banners, message boxes popping up on the bottom right, fullscreen hover popups about “Please subscribe to our newsletter”…

                                                                                    1. 3

                                                                                      The captcha ended for me after three screens – checking pictures containing “glasses”, “checks”, and “bows”.

                                                                                      Maybe the form checks if you marked the pictures accurately. Did you notice that the checkboxes aren’t associated with the picture close above them, but the picture far below them? To see the checkboxes for the top row of pictures, you have to scroll up. I’m not sure if it mattered, but I didn’t select the pictures of glass windows since I would call them “glass”, not “glasses”, and I didn’t select the single picture of chess pieces that were not in check.

                                                                                      1. 3

                                                                                        I noticed the checkboxes being above the pictures (but also well done that you have to scroll up a bit to see this). My mistake might have been the glasses. There was one image with multiple panes(?) of glass, which I marked as “glasses”.

                                                                                        But having to repeat this over and over was also a nice feeling of surfing the web with a VPN and getting Google captchas. I feel that when you use a VPN you get the most difficult ones and have to do like 5 or more to reach a website.

                                                                                    1. 4

                                                                                      I would like to know what kind of drama has ensued over human moderation of tagging at AO3 before jumping ton the conclusion that their particular system of tag curation is better than either top-down Dewey-decimal style classification or twitter-style freeform tagging. I personally rather like the ability to freely create tags, twitter- or tumblr-style, and like the rhetorical effect of writers introducing new information in the form of of a hashtag, which human moderation for the purpose of searchability would get in the way of.

                                                                                      1. 6

                                                                                        You can still freely create tags. Like, I just saw a story that has a “don’t read this if you’re having a bad night okay” tag, which is clearly not meant to help you search or browse, it’s just an informational tag. And I can still click on that tag to see other stories (none yet) with the same tag, I just can’t use it combined with other tags in AO3’s filtering engine.

                                                                                        There’s definitely been drama over the tag system, and there was a ton of debate involved in implementing it. You can read about it on Fanlore. But this is a very large and diverse community, so there’d be drama about any policy made. I don’t think “drama” is a point for or against any policy.

                                                                                        1. 2

                                                                                          This criticism quoted on that Fanlore page resonates with me:

                                                                                          But that’s the problem with the freeform tags; they are simultaneously presented as personal expression for the authors that cannot be touched or questioned even if it’s unwranglable or obviously a typo, and a rational structure to find fic with. It’s an incoherently thought out system.

                                                                                          If AO3 were at all sane and sensible, they’d have a canonical set of tags for indexing and then a freeform area.

                                                                                          After reading all the comments on that Fanlore page, I think there are not two, but three types of tags that people write:

                                                                                          • tags that could be pre-defined officially by the tag-wrangler team, like “Harry Potter” or “slash”
                                                                                          • unusual tags that are still useful for categorization, like “high school AU” or “angry sex”
                                                                                          • one-off tags that are just author’s notes that the author doesn’t want to emphasize, like “don’t read this if you’re having a bad night okay” or “Sirry is a creamy cupcake”

                                                                                          Inspired by the Proposed Suggestions for Improvement section of the Fanlore page, here is a tagging system that gives readers more information and still gives authors freedom of expression. The idea is to store tags of each of the above three types separately, instead of in the same list, as AO3 currently does. The tagging user interface for authors would probably consist of one combined tag field for the first two types, in which the autocomplete suggests official tags before unofficial tags, and one field for “mini-notes” or whatever you call them. The mini-notes would be displayed under the tags field, and could be hidden by users who only like seeing informational tags.

                                                                                          Adding more categories of tags increases the difficulty of communicating to the users how to use the tag system. I think that giving the fields different names than “tags”, such as “mini-notes”, would help a lot, as would putting one-sentence descriptions next to the fields. More ideas for names:

                                                                                          … I’d rename freeform tags as ‘fic labels’ or something, and then create a new set of not-freeform, wrangler-created-and-curated navigational tags. (Or the same thing, but leave the current thing called ‘tags,’ and call the new navigational thing ‘search keywords’ or something.)

                                                                                      1. 2

                                                                                        I remember when I first started using Ruby, I occasionally used my text editor’s shortcut for “wrap selected lines in block comment” and was confused that the resulting code was always highlighted as invalid. I eventually gave up trying to use block comments. It’s good to finally learn why that suggested =begin =end syntax didn’t work.