1.  

    Because it actually believes you meant what you said. Because a programmer is a kind of professional who tells computers what to do, not the other way around.

    1. 10

      Software flaws have real material costs which often manifest as sprawling negative externalities, and C makes writing complex software without these flaws basically impossible. Consider the costs of heartbleed, the Cloudflare memory disclosure bug, or every bug in SMS/MMS parsing that can brick users’ phones.

      Professionals in other lines of work use tools that prevent mistakes all the time, refusing to use tools that let you work safely because you need to stroke your ego is amateur hour.

      1. 6

        With that argumentation you can throw out all of the PLT advancements of the last decades (type systems, formal verification, contracts, …) because why would anyone need that if the computer already does what I say? I vehemently disagree with you, making the computer “do what I say” is not enough to produce working and correct software in any non-trivial capacity.

        1. 5

          which is why we all write assembly, instead of using languages that might rewrite our code behind our back or check whether we made one mistake in thousands of lines of code, being professionals who by definition are all knowing and error-free.

          1.  

            Only when it comes in handy :)

          2.  

            Totally underrated view.

            I personally didn’t sign up for this mindset that everyone just writes buggy code (read:low quality code) and the we throw linters, static analysis tools, compiler warnings and whatnot at it. Then quality should mysticaly spring to life like ghosts separating from dead bodies in horror movies, I suppose.

            There must be a limit to how many errors are reasonable. “There’s no bug free code” doesn’t mean that anything goes.

          1. 3

            I used to work for a company that had settled on a very similar setup (I think using OSPF rather than BGP but likewise using anycast and expensive Juniper routers to avoid breaking TCP). This was meant to save us from getting bottlenecked by frontend load balancers. We had two of those fancy Juniper routers in an active-active configuration, and two warm spares constantly synchronizing state, ready to take over if one of the primary routers died.

            Naturally once it was all installed and brought up, the ops group decided to test out the spare routers by forcing a failover. So they hit the button and… everything died. The routers started spewing some kind of error about invalid state that couldn’t be found in the docs or in any web search. They hit the button to fail back to the unit that had just been failed out… nope, that one wasn’t working now either. The network core was completely dead, and everything internal and external was down. Juniper support couldn’t tell us what was going on. Eventually they managed to hunt down an engineer who could tell us how to reset everything to a clean slate where it would be willing to come up again. And they told us the meaning of that message.

            Y’see, these are the kinds of machines where you pay $X for the hardware, but then you pay $Y, $Z, and $Ω per year for the licenses to use the cool features the hardware has. We had two “active” routers at any time, so we paid for two copies of that “don’t let anycast break my TCP sessions” feature. Meaning that the two active routers claimed those licenses and synced just fine, but the cold spares were getting the state stream and saying “nope! I’m not even licensed to think about that!”. There was a message about it somewhere, but it didn’t trigger any red flags. As soon as one of those spares tried to go active, everything fell apart. And I guess the spare did eventually claim a license, which wasn’t enough to clear its “invalid state” error, but was enough to sabotage the formerly-working unit.

            I just call it the “insert more money” error.

            1. 7

              I’m not sure if there are 10x developers, but there sure are 0.001x developers.

              1. 5

                This is actually a key source of disagreement about the whole “10X” idea. As far as I know, it was supposed to be about the difference between the worst developers and the best ones, not between average developers and the best ones.

                1. 2

                  I’d say it applies somewhat to both, because like most talented jobs, the distribution is right-skewed.

                  1. 1

                    There’s definitely something to that, but I think professional baseball is the wrong analogy for the software development profession as a whole.

                    Major-league baseball would maybe be more of an analogy for big-N tech companies (where the highly-skewed distribution definitely exists, based on my experience working for one of them) and the bush leagues are all the anonymous development teams in big companies that need people to churn out CRUD apps for low-volume internal use and can’t afford to (nor have any reason to) reject 95% of their job applicants.

              1. 7

                Had to read half of the article before it finally sunk in that the headline is about elimination of stores (memory writes) by computer hardware, and not some kind of metaphor for the place down the street where you buy nuts and botls and paint.

                1. 2

                  I had to read your comment before realizing that eliminating memory writes is not the only possible interpretation ;)

                1. 4

                  Ultimately, suffering from Ubuntu’s decisions is a choice.

                  There’s plenty of options, when it comes to Linux distributions. I find most of them are not Ubuntu, and thus do not irritate me in a regular basis.

                  1. 2

                    Or you could just ignore snap, Ubuntu is a fine distro.

                    1. 7

                      Doesn’t look like that will remain an option. It doesn’t make sense for them to maintain snap and deb of a given app indefinitely, and they definitely seem convinced that snap is the easier option, so you should be expecting debs of popular apps to disappear. Ubuntu-maintained ones, at least.

                      1. 4

                        I prefer the path of least resistance. For me that’s Arch. YMMV.

                    1. 3

                      Nice idea :)

                      The trick you did to have the test command declare the test description before the actual test as well as having it flush out the result of the previous test is a bit scary, though, and makes it hard to implement more advanced features. How about this for API?

                      • test "Description" — just declares the test description
                      • ok — pass/fail based on $?
                      • pass and fail — unconditionally pass or fail (for use inside more complex shell logic)
                      • skip — emits a skipped test (ok N - description # SKIP)
                      • todo — mark the next test as a TODO (not ok N - description # TODO)
                      • diag — Print a diagnostic line (starts with a # , then anything after, will be printed by prove -v but can be suppressed when you just want test results).

                      The change does add one line in the simplest case, but in my opinion it would be worth it for the sake of avoiding confusion.

                      A TODO test is “expected to fail”, which means that when it fails it doesn’t increment the failure count, and when it passes, the test harness will tell you about it so that you can remove the TODO annotation. It’s a very nice way to “write failing tests” without actually making the test run fail.

                      1. 1

                        Generally speaking, I’m not a big fan of advancing for the sake of advancing and rather keep things as simple as possible. Actually, not a big fan of doing anything for the sake of doing it.

                        Also, in general, if something feels scary, I’d rather not change it but understand it.

                        Regarding confusion, why avoid confusion?

                        Now, regarding the technical ideas, I think they’re good but for now I’m going to leave this version unchanged, however you can always download the code and add the chances yourself for your own use cases. I’ll keep your ideas in mind.

                        You can see my use case here: https://adi.tilde.institute/linters/.

                      1. 2

                        jq is a really lovely language, especially with version 1.6. I’ve contributed to a set of jq bindings for Go and have a few $WORK projects that make use of it. One is a pubsub library that has jq support in the client (any consumer can filter or transform messages using a jq expression), and one is a sort of rules engine for selecting video transcoding profiles based on the metadata of the source video stream. Its performance is surprisingly decent, too.

                        1. 9

                          This is a great idea for a post that I’ve wanted to write myself. Leaving aside trees, hash tables, and stacks, I probably used less than one “interesting” data structure/algorithm PER YEAR of programming. Some notable counterexamples are two kinds of reservoir sampling, regular languages/automata, random number generation, some crypto, and some info retrieval algorithms.

                          One thing that sparked it is a obscure but long-running debate over whether dynamic programming interview questions are valid.

                          I don’t think they’re valid. It’s mostly a proxy for “I got a CS degree at a certain school” (which I did, I learned dynamic programming in my algorithms class and never used it again in ~20 years.)

                          I challenge anyone to name ANY piece of open source software that uses dynamic programming. Or to name an example in your own work – open source or not.

                          I’ve tried this in the past and nobody has been able to point to a concrete instance. I think the best I’ve heard is someone heard about a professor who heard about some proprietary software once that used it.


                          Related: algorithms used in real software (although this is certainly not representative, since compiler engineering is a subfield with its own body of knowledge):

                          https://old.reddit.com/r/ProgrammingLanguages/comments/b22tw6/papers_and_algorithms_in_llvms_source_code/

                          https://github.com/oilshell/blog-code/blob/master/grep-for-papers/llvm.txt

                          Linux kernel algorithms:

                          https://github.com/oilshell/blog-code/blob/master/grep-for-papers/linux.txt

                          1. 10

                            I challenge anyone to name ANY piece of open source software that uses dynamic programming.

                            Git, or most reasonable implementations of “diff”, will contain an implementation of the Myers Algorithm for longest-common-subsequence, which is very dynamic-programmy.

                            No concrete example for this one, but I know that bioinformatics code is full of dynamic programming algorithms for the task of sequence alignment, which is similar to diff — identifying a way to align two or more base sequences so that they coincide with the minimal number of changes/additions/deletions required to make them identical.

                            1. 1

                              Hm I’m familiar with that algorithm but I never thought of it as dynamic programming.

                              Wikipedia does say it’s an instance of dynamic programming. Although when I click on the paper, it appears to contrast itself with “the basic O(MN) dynamic programming algorithm” (section 4).

                            2. 8

                              Since you mentioned dynamic programming, it’s worth pointing out that the name “dynamic programming” was chosen for political reasons, as pointed out in the history section of the Wikipedia article on dynamic programming. So I think it’s a really bad name.

                              1. 1

                                That’s funny, I remembered xkcd’s “dynamic entropy” comic, and it quotes the same passage:

                                https://xkcd.com/2318/

                                It also has a very interesting property as an adjective, and that is it’s impossible to use the word dynamic in a pejorative sense. Try thinking of some combination that will possibly give it a pejorative meaning. It’s impossible.

                                LOL

                                Agree it’s a terrible name… I would say it was chosen for “marketing” reasons

                              2. 7

                                I have thought about whether dynamic programming questions are fair to ask, and I ended up where you are: they are not.

                                Dynamic programming was the one I struggled most in understanding and implementing correctly. And while there are semi-practical examples (like the backpack problem), I have not found any practical, day-to-day use cases on this.

                                I had an argument with my colleague who asked this kind of problem, saying it’s basic knowledge. Turns out he did competitive programming and there, it is table stakes. But in practice, it just filtered for anyone who has learned and practiced this approach.

                                I stay away from asking this, problems that need dynamic programming to solve.

                                1. 4

                                  I’m familiar with dynamic programming mostly from high-school competitive programming as well. Otherwise I can’t say I’ve encountered real-life problems where it occurred to me to use the technique.

                                2. 8

                                  I challenge anyone to name ANY piece of open source software that uses dynamic programming. Or to name an example in your own work – open source or not.

                                  I’m literally about to implement something that could be classified as dynamic programming at work, which can be sumarrized as “computing a few simple statistics such as number of bytes changed for each directory in a large filesystem”. Dynamic programming is such a general term that it applies regularly if you want to use it.

                                  1. 4

                                    I’d like to see it. I don’t think dynamic programming is a general term.

                                    In fact one time I participated in this argument (which was years ago so my memory is fuzzy), a former CS professor I worked with explained the difference between memoization and dynamic programming. A bunch of 10-20+ year programmers like myself went “ah OK”. Unfortunately I don’t even remember the difference, but the point is that most programmers don’t, because dynamic programming is very uncommon.

                                    What you’re describing sounds like an obvious algorithm that anyone could implement, which is not the same as dynamic programming interview questions, or dynamic programming in competitive programing.

                                    As other posters mentioned, competitive programming is the main place you see it outside of a CS class.

                                    1. 2

                                      It’s absolutely an obvious algorithm, so is most dynamic programming. That was sort of my point.

                                      Can’t share the code unfortunately, but it’s just iterate over sorted list of file changes in reverse order and collect statistics as we go. Dynamic part comes from the fact that we can just look at the subdirectories of a dir (that we already have numbers for) instead of recursing into it.

                                      1. 2

                                        What you’re talking about could be called memoization, or it probably doesn’t even deserve that name. It’s just what a “normal” programmer would come up with.

                                        That’s not the type of thing that’s asked in interview questions or competitive programming. The wikipedia page gives some examples.

                                        Dynamic programming usually changes the computational complexity of an algorithm in some non-obvious way. There’s very little you can do recursing over a directory tree that doesn’t have a clear O(n) way to code it (e.g. computing statistics).

                                        1. 7

                                          I like Erik Demaine’s explanation, that problems where dynamic programming can be applied are ones where their subproblems and their dependencies can be modeled as a directed acyclic graph [1]. Up to you if you’d like to tackle that with a top down approach where you look at a node and calculate its solution based on the solutions of its ancestors, or a bottom up approach starting from the nodes in the DAG with no dependencies and propagate the solutions in topological order.

                                          My colleague and I used it for a generalization of matrix chain multiplication (for tensors) [2].

                                          [1] https://youtu.be/OQ5jsbhAv_M?t=2736

                                          [2] https://github.com/TensorCon/ratcon/blob/release/opt/gencon/pathopt.py#L198

                                          edit: by the definition above, even naive memoization can count as DP, if you’re caching solutions to subproblems of that structure. Doesn’t have to be at the difficulty level of competition to count as DP in that case.

                                          1. 1

                                            Hm interesting, searching through my personal wiki, I found a related definition which is different. I haven’t really thought about it enough to form an opinion.

                                            Either way, it doesn’t change my overall point: that there are certain kinds of algorithms problems that appear on coding interviews, and in competititve programming, that do not show up in 99% of programming jobs. They are easy to pose and have cute solutions, but aren’t testing very much.

                                            I think the “DP tables” part is key but again I haven’t thought about it enough …

                                            https://blog.racket-lang.org/2012/08/dynamic-programming-versus-memoization.html

                                            Memoization is fundamentally a top-down computation and DP is fundamentally bottom-up. In memoization, we observe that a computational tree can actually be represented as a computational DAG

                                            In DP, we make the same observation, but construct the DAG from the bottom-up. That means we have to rewrite the computation to express the delta from each computational tree/DAG node to its parents. We also need a means for addressing/naming those parents (which we did not need in the top-down case, since this was implicit in the recursive call stack). This leads to inventions like DP tables, but people often fail to understand why they exist: it’s primarily as a naming mechanism (and while we’re at it, why not make it efficient to find a named element, ergo arrays and matrices).

                                            This bottom-up / top-down distinction might have been the same as what the aforementioned professor said 5+ years ago, but I don’t remember exactly.

                                            1. 1

                                              So, is memorization of factorial top-down, or bottom-up?

                                              1. 1

                                                I would probably say neither… Either factorial or Fibonacci is so trivial that it doesn’t help to be thinking about it that way.

                                                Though I think the quote hints at a clear test for whether it’s top-down or bottom-up: if you need extra state outside the call stack. I’m getting curious enough to try this out, but right now all I can do is quote what other people say.

                                                In any case it’s clear to me that there’s some controversy over what dynamic programming really is. I think the issue is that a lot of algorithms could be framed that way but were not discovered that way, and not taught and learned that way.

                                                1. 1

                                                  I would probably say neither… Either factorial or Fibonacci is so trivial that it doesn’t help to be thinking about it that way.

                                                  I think that the triviality is actually helpful here. If it’s actually true that memoization and dynamic programming are different (and there’s clearly debate on this), can 2 implementations of a trivial function, that everyone can understand highlight the differences?

                                          2. 1

                                            On the contrary the stupidly naive way (recurse on every directory) is O(n^2).

                                            Dynamic programming is just solving a series of problems while using the answers of shared subproblems multiple times. Memoization is a common way to implement this.

                                            Yes there are some very clever algorithms that use dynamic programming, this doesn’t make obvious algorithms that use dynamic programming not also fit under the definition.

                                            1. 3

                                              Why would recursing into every directory be O(n^2)? You’re still only visiting every directory/file once? It seems like something missing?

                                              1. 1

                                                Say you have a directory structure with a single file in it, /a/b/c/d/e

                                                To get the number of bytes changed in e you need to visit e, then to get the number of bytes changed in d you need to visit d and then e, then for c you need to visit c, d, and e, and so on.

                                                Like I said, it takes a really naive solution, but if you don’t remember the values you calculate anywhere for some reason it’s sum over inodes (depth of inode)… which is O(n^2) (for bad directory structures).

                                                Note that I need these values for every directory, not just for one particular directory.

                                                1. 2

                                                  That’s still linear complexity space. Unless you’re hardlinking directories (which you then have to deal with potential recursion), it’s still O(n). If you add a file at /a/b/c/file you only visit 1 more file and no more dirs, not an exponential. O(n + n + n) or O(n + 3) still simplifies to O(n).

                                                  1. 1

                                                    If you add /a/b/c/file you add 4 more visits, not 1. Going from n= 3 /a/b/file to n=4 /a/b/c/file adds 4 more visits. In other words this worst case example takes time O(sum from 1 to n of i) = O(n(n-1)) = O(n^2).

                                                    N is the number of inodes in a arbitrary tree not a number of files in a fixed tree.

                                                    1. 1

                                                      That’s still adding a linear number of operations for each file, the depth could technically be considered a different variable, say m. So for each file (n+1) you add, you also add the number of directory traversals (m) resulting in O(m+n), which simplifies again to O(n), but in reality folders are files too, so are part of n in the first place, so again O(n). Ultimately your n space is the total number of inodes, which both files and folders have.

                                                      Abstractly, you’re just traversing a tree structure (or a directed graph if using links), which is well understood to be O(n) (maybe O(n^2) worst case if all folders are links, resulting in a fully connected graph), because you only visit each node once. If it were O(n^2), you would visit each node n times.

                                                      Remember, Big O notation is about scaling, not the actual concrete number of operations, which is why you drop any constants or variables other than n.

                                                      1. 1

                                                        It’s O(mn) not O(m+n) (in the insanely naive algorithm that recalculate things every time).

                                                        It’s not a single tree traversal but #internal nodes tree traversals.

                                                        1. 1

                                                          Even if it was O(mn) (it’s not), that still simplifies to O(n). An ‘internal nodes’ tree traversal is still O(n), n is just smaller, but again, your problem is not an internal nodes traversal, it’s a full traversal because you have to look at the blocks attached to the file (leaf) inodes, which means you need to read all inodes of all files and of all folders one time each. n = # of files + # of folders = O(n)

                                                          1. 1

                                                            I supposed an extremely naive solution could be to fully traverse each sub tree for every folder visited, which would be… O(log n)? But even that isn’t O(n^2), as the total repeated space shrinks the deeper you get.

                                                            1. 1

                                                              You’re assuming a balanced tree, which is not guaranteed. Depth of tree is O(n) in pathological cases (and average case O(sqrt(n)) is typical for randomly generated trees)

                                                              1. 1

                                                                Ah yeah, I think it would be O(n log n) not O(log n), because you traverse the tree once for each node, and a subset of of the tree for almost every n (except leafs), at least in the worst case. Still not O(n^2), and the solution for a O(n) is almost easier to conceptualize than the completely naive solution :)

                                                                1. 1

                                                                  and the solution for a O(n) is almost easier to conceptualize than the completely naive solution :)

                                                                  No argument here…

                                                                  I think it would be O(n log n)

                                                                  We agree it’s O(n) * O(time tree search) now right? And you’re trying to call the tree search time log(n)? Because trees are height log(n)? Then see the post you replied to, that’s true in a balanced tree, it’s not true in a random tree (where it is sqrt(n)) and it’s definitely not tree in a pathological worst case (where a tree is just a n length linked list).

                                                                  1. 2

                                                                    Yeah, the part I was hung up on before was that you’re naive solution traverses the entire subtree below a node for each node visit, I was stuck in the simple optimal solution. For the pathological case, basically just a bunch of folders in folders with a single file at the bottom, the depth of the tree is n, and the file inode at the bottom would be accessed n times, so O(n^2). For the common case it would be about O(n log n) where you can skip traversing larger and larger parts of the tree the deeper you get on each ‘path.’ Thanks for the discussion, I enjoyed it :)

                                          3. 1

                                            I think comparing memoization to dynamic programming is a category mistake: they are different kinds of things.

                                            ‘Dynamic programming’ is a description of a style of algorithm. It’s divide-and-conquer, usually with overlapping subproblems, making it possible to reuse intermediate results.

                                            Memoization is a programming technique to remember intermediate results, by remembering the results of function calls. You can e.g. also store the intermediate results somewhere explicitly, usually in a matrix, in which case you don’t memoize the result ‘transparently inside the function’, but use a lookup table ‘external to the function that computed the result’.

                                            1. 1

                                              I dunno I find that in addition to the Racket language resource I gave elsewhere in the thread, lots of people compare them:

                                              https://medium.com/@afshanf/cracking-the-coding-interview-questions-part-7-a7c8f834d63d

                                              A note on terminology: Some people call top-down dynamic programming “memoization” and only use “dynamic programming” to refer to bottom-up work. We do not make such a distinction here. We call both dynamic programming.

                                              There does seem to be disagreement on what dynamic programming is. And many algorithms that were not derived with dynamic programming techniques could be described as dynamic programming.

                                              But it seems that most people agree it’s related to memoization.

                                        2. 4

                                          GCC uses dynamic programming to split IA-64 instructions into bundles.

                                          1. 2

                                            Thanks, nice example and link! Still I would say it’s a niche skill, especially to come up with from scratch in an interview.

                                          2. 4

                                            I challenge anyone to name ANY piece of open source software that uses dynamic programming. Or to name an example in your own work – open source or not.

                                            Ever do video encoding or transcoding with anything built on FFmpeg or x264? Encode images with MozJPEG? Encode an AV1 video or AVIF image with libaom? Trellis quantization in advanced lossy compression encoders is a dynamic programming algorithm.

                                            1. 3

                                              Hm very interesting! I was not aware of that algorithm. Paper I found:

                                              https://www.mp3-tech.org/programmer/docs/e00_9.pdf

                                              I would still say it’s a bad interview topic, but it’s cool to see real world usages of it.

                                              1. 2

                                                Oh, no disagreement there! Even after coding it up myself, I’d hate to have someone ask me to whiteboard a working implementation of trellis quantization in 40 minutes or whatever (though I’m pretty sure I could sketch out an explanation now).

                                                In general I’m not a fan of whiteboard coding exercises at all. Whenever I’ve interviewed candidates I’ve always preferred the old-fashioned method of just reading their resume well ahead of time, looking up what ever piques my interest on it, and then having a friendly conversation about that. Usually that provides plenty of esoteric material for me to quiz them on and it also lets them show me their strengths and enthusiasm.

                                                1. 1

                                                  My current company doesn’t do a whiteboard exercise, but my previous one did… but the thing is, the usual task was to implement a basic “grep”. That is, read a file and print all of the lines that contain a user-specified string, in a language of your choice, with whatever libraries make you happy (it’s not a trick, you’re not supposed to implement Boyer-Moore on your own). Assuming you succeeded at that, we would ask you to implement a few more features, like a -v flag (only print lines that don’t match), and -A and -B flags (print context lines before and after the matching line), until you got stuck or the time for that segment was up. It wasn’t graded on minutiae like correct semicolon placement, it was just an evaluation of whether a candidate could do a trivial task, how they handled additional requirements, whether they asked sensible questions and got clarification when needed, etc. I found it pretty reasonable.

                                            2. 4

                                              I challenge anyone to name ANY piece of open source software that uses dynamic programming. Or to name an example in your own work – open source or not.

                                              I used Warshall’s algorithm (which is dynamic programming) to compute the transitive closure of a graph for a typechecker. This is, in my experience, a very common algorithm.

                                              In high school, I wrote a program for my professor that places students into groups of 4 such that their meyers-briggs personalities are as different as possible. This used dynamic programming.

                                              A professor of mine (who taught the subject to me) used dynamic programming for some kind of RNA sequencing problem in a paper he published. One of our homework assignments had us arrive at a watered down version of his (and his co-authors’) algorithm.

                                              I’m fairly certain that at least some fuzzy string matching algorithms use string distance, which is also solved using dynamic programming.

                                              These are all diverse applications of DP. In my personal, subjective experience, the idea that DP is in any way obscure or dated is absurd.

                                              Edit:

                                              To be more concrete, the “transitive closure of a graph” is for the graph of dependencies, computing the set of all functions that a particular function depends on. This is as described in the Haskell Report.

                                              For fuzzy string matching, I have in mind something like fzf, though I cannot say with certainty that it uses string distance (I’m unfamiliar with its implementation).

                                              Here’s the paper that I think I’m referencing: Statistical Mechanics of Helix Bundles using a Dynamic Programming Approach

                                              1. 2

                                                Thanks for the examples. The claim is definitely not that it’s outdated or obscure; the claim is that it’s not a good interview question because it doesn’t show up much at work. Although there were lots of people here who pointed out interesting uses of dynamic programming, that’s not incompatible with the idea that you could have a 10 or 20 year programming career and never use it.

                                                Side note: I’m familiar with the Floyd Warshall algorithm but I never thought of it as dynamic programming. I think part of the issue is that I may have a more narrow definition of it than others. (I think people even say the linear time fibonacci is an example of dynamic programming, which I find silly. I just call that the obvious algorithm. I guess it can be used to illustrate a principle.)

                                                Even so, I definitely think it’s more popular in universities, and certain domains like bioinformatics. In contrast to what people on this site typically do “for work”.

                                              2. 3

                                                I challenge anyone to name ANY piece of open source software that uses dynamic programming. Or to name an example in your own work – open source or not.

                                                I do a lot of work with text processing – computing the edit distance between two strings is something I do very often. That’s a classic dynamic programming algorithm. There are probably hundreds of open source packages that do this or some variation thereof.

                                                1. 3

                                                  Just to add to the list of responses clearly demonstrating Cunningham’s Law:

                                                  I believe the Knuth-Plass line-breaking algorithm used in LaTeX to lay out text “optimally” uses dynamic programming. This was done for efficiency, as opposed to using some kind of general global optimization routine. It’s also the reason why LaTeX doesn’t support “backtracking”.

                                                  1. 2

                                                    It’s also the reason why LaTeX doesn’t support “backtracking”.

                                                    Sile uses a variant of the same dynamic programming algorithm to lay out paragraphs on a page. The original paper describing the algorithm says that TeX wanted to use it like that, but it would require more than one entire megabyte of state for a large document, which was infeasible.

                                                    1. 1

                                                      Definitely an instance of Cunningham’s law at work :) I should make another go for my pet problems:

                                                      • it’s impossible to make a zsh-like interactive interface on top of GNU readline
                                                      • you can’t make a constant-space linear-time model of computation that’s more powerful than regular languages, and that can parse shell/JS/Python/C++
                                                      • you can’t make an extended glob to regex translator in less than a week (https://github.com/oilshell/oil/issues/192)

                                                      Thanks for the example. If there were more specific links I would make a blog post out of this :)

                                                      And although my comment was a tangent, it did motivate me to get out the “Algorithm Design Manual” and flip to the part on dynamic programming. Though I remember the applications in that book being interesting but seemingly far removed from what programmers do day-to-day. It seemed to be by a professor who consulted on algorithms for various companies, which is an interesting job!

                                                    2. 1

                                                      The Grasshopper routing library uses contraction hierarchies, which are implemented using Dijkstra’s shortest path algorithm and A* search, which are special cases of dynamic programming.

                                                      I have to agree it’s not something most people will use every day, but it never hurts to have a general idea how it works.

                                                      1. 1

                                                        Here is a concrete example of Dynamic Programming that you use every day: Word Wrap. Knuth has an algorithm that is often used for maximizing the number of words per line.

                                                        Also the field of Bioinformatics often uses the Levenshtein distance when matching two dna strands.

                                                        Also I would like to mention the single most important thing I learned from Dynamic Progrmming: Start at the end case, and figure out what constraints can work from there. For example, think about the last recursion call, and what constraints it needs, and go backwards from there.

                                                      1. 5

                                                        Been using it for about three years at work, mostly in the context of GKE.

                                                        It’s… okay. It’s super complex and it’s front-loaded with a whole ton of terminology you have to absorb before you can do anything with it, and then it’s still super complex and you have to configure five different parts of the layer cake just to get the simplest service online, not to mention probably learning at least one tool that generates config files from other config files.

                                                        But on the other hand, if you actually need a thing that does what it does, which is run containerized stuff on a cluster, with scaling and high availability and semi-intelligent placement and service discovery and whatever else, then you’re pretty much either using k8s or inventing your own — I don’t know of anything that sucks less. And if you invent your own, pretty much the best-case scenario is that you create something that works… but not as well as k8s, but in the process you gain enough understanding to throw your thing away and use k8s effectively, because they’ve already solved problems you didn’t know you had going in.

                                                        In short, the sheer meta-ness of it all breaks my brain and makes me cry and wish for the days when I could point at a gray box with blinkenlights on a rack and say “the application runs right there”, but kubernetes didn’t create that problem, the exigencies of modern software design did. Once you get used to it, it’s tolerable.

                                                        On a semi-related note, for a personal project of mine I’ve been making use of podman plus some systemd scripts to coordinate a bunch of interacting services. For the most part, podman is best thought of as “daemonless docker” (which means it interacts better with process managers because the container is actually a child of the command that started it, and you can actually tell when a container is up, and not just when the daemon has acknowledged a request to start it, and also means that it interacts better with security and accounting because running as an unprivileged user actually runs as an unprivileged user, instead of handing off work to a privileged daemon), but one place where it differs from docker is it makes it super easy to create shared network contexts like k8s pods… it’s even in the name :) Anyway, because it is a personal project, it runs on a single server, has no SLA, and can tolerate a certain amount of manual intervention if I ever have to move it somewhere else or recreate from scratch. So that’s why I can do without the complexity of k8s.

                                                        1. 4

                                                          This is an outright fun paper. It’s serious (obviously Knuth cares about the subject) but not too serious. Instead of being laser-focused, it’s happy to just sit the reader down and say “here, let me show you some reasons why I think this works out well”, and isn’t afraid to make a few digressions along the way. I don’t mind a bit of formalism, but a holistic style is good for making math accessible to people who aren’t too deep into any particular rabbit-hole.

                                                          1. 29

                                                            I cannot help myself but to imagine a open source ideas incubator governed by:

                                                            Antirez, Linus, Mike Pall (LuaJit), Rich Hickey (Clojure)

                                                            Perhaps, they would find it difficult to work together, but they all have something in common:

                                                            1. They cannot be easily manipulated
                                                            2. they do not want to manipulate others
                                                            3. They are visionaries
                                                            4. They express themselves through software (on both technical and conceptual levels)
                                                            5. They work very hard, for very long time, based on a believe and passion alone
                                                            6. They are not afraid to challenge ‘status quo’

                                                            The combination of the above, perhaps is rare, and finding a way to explore their combined wisdoms, quirks, and talents – would be very neat and beneficial to society as a whole.

                                                            If such a cooperative is successful, I would also hope that they would figure out a way to find the next generation of folks so that the process continues.

                                                            1. 5

                                                              I like it, but maybe sub Mike Pall with Roberto Ierusalimschy because he seems to meet all the same criteria except has a wider range of projects he’s involved in and presents lots of white papers and research around his projects. [1]

                                                              No offence to Mike Pall, of course, LuaJIT is still alien software in my book and “he” still could be an anonymous group of genius computer scientists sharing a pseudonym afaik.

                                                              [1] - http://www.inf.puc-rio.br/%7Eroberto/cvpub.html

                                                              1. 3

                                                                Why are there no women in this list?

                                                                1. 47

                                                                  While I agree with the sentiment, I think the short question comes off as aggressive. Instead, you can make a suggestion to avoid putting all of the onus on the previous commenter.

                                                                  For example, “I noticed your list doesn’t have any women. What about ___ or ___? They have a proven track record on $PROJECT.”

                                                                  It brings attention to talented women in the industry and provides people with a jumping off point to learn about their accomplishments. Asking questions calls attention to biases, but it doesn’t always inform.

                                                                  1. 10

                                                                    Some random posters list if 4 of their favorite open source project leaders doesn’t need to add a token woman. Asking politely would have made it harder to see how completely ridiculous this is.

                                                                    1. -1

                                                                      You mistakenly assumed the sentiment is to call out a user instead of pointing out the systems that have led to there not being a lot of women for them to include.

                                                                      The question is about why there aren’t comparably a lot of women noted in open source and not meant to call out OP directly.

                                                                      Y’all are being defensive where it isn’t needed and taking this personally instead of as a time to reflect on why open source culture is how it is and how behaviors in the current system are the problem.

                                                                      Honestly, with 23 upvotes and 22 downvoting and having to share your defense, all we’ve done instead is identify the problem even more concretely than would have happened if y’all would have taken the comment for what it was and not for what it wasn’t.

                                                                      For those of you labeling this as “off-topic” specifically, I would argue that it isn’t. I’m asking about a list in response to a comment about a list. The list is the topic in context of this thread.

                                                                      Hopefully this comes to fruition, because it’s unfortunately too relevant here. https://lobste.rs/s/lpvcsm/proposal_for_moderation_policies_no_tone

                                                                      1. 5

                                                                        Y’all are being defensive where it isn’t needed and taking this personally instead of as a time to reflect on why open source culture is how it is and how behaviors in the current system are the problem.

                                                                        If you wanted people to reflect on it, then posting a pointed question is not the way to go. Something similar to this would have been waaay better at getting people to think:

                                                                        I’d love to see a few women contributors on that list too. Do you know any who would fit that team?

                                                                        I’m not here to tone police comments because I actually agree with you that we should encourage women to join us in these endeavours, but you are not helping that cause. You are creating division by using bad wording and being more focused on being “correct”. Take this loss and let the issue die down. The trenches have already been dug here.

                                                                        1. 1

                                                                          If you wanted people to reflect on it, then posting a pointed question is not the way to go. Something similar to this…:

                                                                          I’d love to see a few women contributors on that list too. Do you know any who would fit that team?

                                                                          Your new question here has absolutely nothing to do with the one that was posted. monokrome was not looking for someone to offer up the name of a woman in tech, but to discuss the reasons why there might not be one on the list in the first place.

                                                                          I’m not here to tone police comments

                                                                          Seems to me that not only are you here to tone police, but also to police the content of the comment; to divert away from a meaningful and difficult discussion of under-representation (which you have no obligation to participate in), and instead re-frame it as a pedestrian question of “could someone please identify a women in tech”.

                                                                          1. 1

                                                                            discuss the reasons why there might not be [a female programmer] on the list in the first place.

                                                                            Why should this discussion happen on lobste.rs? I thought we only allow content related to programming?

                                                                        2. 3

                                                                          Well, it seems that I misunderstood your comment here. I imagine that many others did too. I think that is the key issue here. Having a comment that explicitly calls out the system instead of the list would have made a difference for me. Calling out just the list makes it seem like the problem is with the person who created the list.

                                                                          As for tone policing, I do believe systemic biases exist and need to be discussed. However, I think the way we discuss them is important.

                                                                          I like lobsters a lot and I don’t like seeing discussions devolve into a locked in flame war. My comment was meant to provide constructive feedback and to take air away from destructive comments. My goal isn’t to prevent you from speaking, but to make it easier for your to be heard, if that makes sense.

                                                                      2. 32

                                                                        Cynical answer: building rep in the open source world takes a lot of time, and it’s been historically so woman-hostile that women haven’t been given a chance to build serious rep.

                                                                        Hopeful answer: Jessie Frazelle, maybe? My niche is dominated by academics so I don’t know a lot of FLOSS names outside of formal methods.

                                                                        1. 6

                                                                          Speaking of Jessie Frazelle, I’ve recently learnt that she’s teaming up with a few other brilliant people at https://oxide.computer/, where you’ll find (among others) Bryan Cantrill and Steve Klabnik.

                                                                          1. 3

                                                                            Well, that’s cool :)

                                                                          2. 12

                                                                            Even the women who have built names for themselves in tech, are less likely to be known by men. So there’s that. I can think of a bunch of amazing people but not specifically in open source…

                                                                            1. 8

                                                                              Strange assumption. What are open-source projects run by women that people should know about?

                                                                              1. 30

                                                                                Cynical comment: Not identifying as a woman, but having “seen some things” over the years, I can’t help but think that some of the reasons we don’t find so many woman leading open source (surely there are many, though!) is because it’s dangerous to put yourself out there as a woman. How many sexual violence threats do you think a woman would receive if they were the maintainer of Redis, very opinionated, and shutdown ideas because they didn’t fit the direction she wanted to take? I bet the number is not 0.

                                                                                1. 11

                                                                                  You-ain’t-seen-cynical-yet comment: How many death threats do you think a man receives if they’re the maintainer of Redis, very opinionated, etc? The number is very not 0. Anyone who has ever done anything significant has been on the receiving side of some harassment. Today’s effortless communication magnifies the effect, but I’m sure there were people writing “Ο Ευριπίδης τρώει σκατά” on the walls in 420 BC. The difference is whether you let it derail you. The successful ones are the ones who ignore that shit. So why do those who persevere tend to be “privileged”? A very simple psychological reason. Perceived-privilege means having no other face-saving option. If you bail out from a visible position after receiving some harassment and you say “I quit because I felt threatened as a woman” or “I quit because I felt threatened as a person of color” or whatever, there’s a fair chance you’ll be the subject of articles praising your heroism in exposing the harsh reality faced by members of $GROUP who dare to lead. Do the same thing as a straight white male, and, well, you’re a quitter. The narrative expects success from you, it expects you to keep your problems to yourself, and you’d better not have any insecurities. Underdogs are expected to battle with self-doubt; on you, it’s childish and absurd, so forge ahead or be a failure forever.

                                                                                  1. 9

                                                                                    Or maybe it’s because those groups are often at risk in their every day life and take threats on themselves seriously due to past experience?

                                                                                    1. 2

                                                                                      That would be a pretty big mistake. Regardless of who you are, the supply of people willing to make threats still outnumbers the supply of people capable of carrying them out a million to one. The harm here comes from labels themselves.

                                                                                      1. 15

                                                                                        I think you’re arguing in good faith, so I’ll provide a good faith response. The problem isn’t one of existence but degree. I’ve been the target of internet outrage mobs and serial harassers, and both are terrifying. I lived through them, yes, but it’s absolutely awful to have someone constantly email you screeds from new, unblocked email addresses about how you’re awful and deserve to die. I’m also a straight (((white))) male and have only really gotten attacked over things I wrote, not my identity. Would I have muscled through it and continued writing if I had gotten 10x the amount of online harassment? I don’t know, maybe I would have managed, maybe it would have been too much for me. But there’s certainly some magnitude of abuse that would have gotten me to stop drawing attention to myself. Women and minorities get more abuse online, so it’s more likely to cross that line.

                                                                                        Summary: everybody has tolerance level, there’s probably no significant difference in what the tolerance level is for men and women, but women get so much more abuse online it’s more likely to cross that tolerance level.

                                                                                        1. 3

                                                                                          Different people do have different tolerance levels; it’s easy to see that some people withstand massive abuse, and some don’t, and the differentiating factor is self-esteem, or confidence, or arrogance, however you want to interpret it. So why do the ones with that confidence tend to look alike? Freely granted, one reason is because there is injustice in the world, and a track record of success builds confidence.

                                                                                          But that can’t be all either, and here is where my dark fear lies: by labeling people as disadvantaged, we disadvantage them. Psychology is a terribly powerful force. If you believe that the problems you face are the same problems shared by all human beings, trying to make something of our lives in an indifferent universe, then any successful human being can be a role model to draw strength from. If you believe that your problems are special, that only a few share your circumstances enough to understand, then you’re reliant on a role model who is “like you”. That mode of thinking used to be the province of moody teenagers, and growing past it was the sign of an adult — but now it’s just normal, and labeling strongly encourages it. All I want is a world where people tell themselves “I am a human being. I have been blessed with intelligence that can overcome any problem. I cannot ask for any more.”

                                                                                          1. 1

                                                                                            So, is your solution is to just ignore the problem and not talk about it? If so, I think that there has been enough proof to show that only makes the problem worse.

                                                                                            1. 2

                                                                                              No, just the opposite. If I just wanted to ignore problems and not talk about things, I think I could do that perfectly well on my own, without the aid of the internet. But because I have a foolish compulsion to try to make a positive difference in the world and help other human beings, you see me here. Thankfully for all involved, mostly that compulsion expresses itself as tens of thousands of hours of technical volunteering, but occasionally a bit of freelance philosophizing bubbles up. Agree with me or disagree with me, but kindly don’t misrepresent me.

                                                                                    2. 5

                                                                                      Less death threats than a woman receives. I’ve had multiple people threaten to murder me just because I was streaming video games on Twitch, and I’ve never had more than a dozen or so viewers. If a woman was maintaining something as big as Redis, they’d be getting more death threats than Antirez.

                                                                                      Nobody should be getting death threats, but that’s just the truth.

                                                                                      1. 2

                                                                                        I think it’s much more likely that people quit silently all the time, whether they’re URMs or not. It just doesn’t get publicized so much, so it’s not as visible. But when people quit publicly, it’s much more visible, so it’s what we tend to focus on.

                                                                                    3. 3

                                                                                      I am choosing not to answer this because (1) it’s beside the point, and (2) as I said in the remark you’re replying to, nothing comes to mind. I’m not clear why you felt the need to ask for something that I already said I don’t have.

                                                                                    4. 2

                                                                                      Yeah, the way women are treated in open source doesn’t help the situation.

                                                                                    5. 1

                                                                                      I was hoping people would see it this way but it seems everybody takes it defensively instead of identifying problems. Thanks for not being one of them :)

                                                                                    6. 11

                                                                                      What do you hope to gain by asking such a pointed question that comes across as very aggressive?

                                                                                        1. 1

                                                                                          To be clear: This is not trolling or off topic. This is a reply to a post that directly mentions the comment which it was replying to, and the content that this comment represents is exactly the kind of content that the link represents. I am not trolling, but want you to be aware that this behavior is being noticed.

                                                                                          The constant flagging of comments only proves that flags can’t be used to do anything like banning people, because people are too immature, defensive, and unreasonable to flag properly.

                                                                                      1. 33

                                                                                        And here you see why Antirez wants out.

                                                                                        1. 13

                                                                                          Exactly, too many people in the industry spend their time stirring up controversy. Those folk do not provide solutions and just cause people like Antirez problems and a heacache. They complain and paint people that provide tremendous benefit to the company in a bad light.

                                                                                        2. 18

                                                                                          Go ahead, make your nomination.

                                                                                          1. 11

                                                                                            Why are there no Uzbekistanis on this list?

                                                                                            1. 28

                                                                                              Алекса́ндра Аса́новна Элбакя́н isn’t Uzbekistani but Kazakhstani (so not part of the usual US/Europe community), but definitely matches the criteria in the list, through Sci Hub challenges the status quo in ways that - in the long term - may exceed even Linus’ contributions and is even a woman (see the question up-thread for why that matters). So, there’s my nomination.

                                                                                              1. 3

                                                                                                I don’t know, in the long run, if Sci Hub will be more influential than Linux. But as of 2020 it’s absolutely more influential than Redis, LuaJIT, and Clojure.

                                                                                                1. 2

                                                                                                  Hahaha, thank you for this answer :)

                                                                                                2. 4

                                                                                                  Uzbekistanis make up 0.4% of the world population. Women make up ~50% of it.

                                                                                                  1. -2

                                                                                                    Women make up ~50% of it.

                                                                                                    What are you implying exactly?

                                                                                                3. 1

                                                                                                  Because Redis, Linux, LuaJit, nor Clojure were made by women.

                                                                                                  1. -9

                                                                                                    Because none of them has come out as trans yet.

                                                                                                    1. 6

                                                                                                      Per my sister comment, I don’t think a short, quipy comment helps further this discussion. This comment thread is potentially sensitive and I do not think we should escalate it.

                                                                                                      1. 6

                                                                                                        There is nothing we as users can do. This site is just going to continue going down the shitter unless the admins/mods start enforcing rules, or replacing the terrible, terrible, upvote-comment-tree comment format with a traditional forum layout. If I was running this site, I would ban this person (and everyone who wrote a snarky reply) for 7 days, leave the comment in-tact, and prevent people from replying to it. It’s simple and gets the message across.

                                                                                                        1. 2

                                                                                                          Could you link to the comment you’re referring to? I can see a couple candidates and don’t want to misunderstand you by guessing.

                                                                                                          Also, what specific rule would you like to see added and enforced?

                                                                                                          1. 1

                                                                                                            “Why are there no women in this list?” is rather passive aggressive and in no way constructive, I think it warrants a temp-ban. It’s a zero effort cheap shot. Look at the comments that spawned from it. Would be a tragedy if lobste.rs became like HN, Reddit, or, godforbid, Twitter. </2 cents>

                                                                                                            1. 2

                                                                                                              Not to mention that the poster’s Twitter account is suspended – that takes some effort.

                                                                                                              1. 1

                                                                                                                moved reply to meta thread for the sake of conversation there: https://lobste.rs/s/lpvcsm/proposal_for_moderation_policies_no_tone#c_fu0xzl

                                                                                                              2. 1

                                                                                                                Sorry I shouldn’t have used the word “rule”. I don’t think specific rules are strictly necessary. All I’m saying is, you’re admin, it’s your site, you have all the power and can do basically whatever you want. I feel like punishing the bad actors in a public way would fix a lot of things.

                                                                                                                SomethingAwful and Facepunch had pretty good mod teams and tooling if you want something to go off of.

                                                                                                              3. 1

                                                                                                                Well, now we know why N64N64 isn’t a mod.

                                                                                                            2. 2

                                                                                                              What is your problem?

                                                                                                          2. 2

                                                                                                            RMS fails the second criteria? If not, why did you choose not to include him?

                                                                                                            1. 25

                                                                                                              I can imagine those four working well together. I cannot imagine them working well with RMS.

                                                                                                              1. 1

                                                                                                                This comment is especially funny when juxtaposed with https://lobste.rs/s/gigoo8/end_redis_adventure#c_tf95fm - as RMS is well known for his ‘gross’ approach to sexual advances.

                                                                                                              2. 1

                                                                                                                While I understand the idea, I wonder whether such a thing is in general a good idea. You know, when someone is good and successful at X, it doesn’t necessarily mean that they are also good at making others good and successful at X.

                                                                                                                This is based on a complete assumption. What if these people got successful because they went their own road. I think that’s what these people have in common. Following the steps of one of them closely (which might be the main thing they can provide) might be the complete opposite.

                                                                                                                Without wanting to go too far from that thought, I also see that in day to day work, that is people assuming they do the right thing based on someone else having done the same thing. While this of course might be true and is a good approach in many situations, it might not be.

                                                                                                              1. 12

                                                                                                                Note this is a rough draft: I’m working on a more polished version as well as a series of articles. But people seem to like it so far so I figured I’d share the current version.

                                                                                                                1. 4

                                                                                                                  Interesting video.

                                                                                                                  Though I don’t understand why people want to be engineers so badly. Is it an ego thing?

                                                                                                                  Software development has more similarities with pigs wallowing in their own shit than with any serious engineering discipline from my perspective.

                                                                                                                  No quality standards, no punishment for gross misconduct, no loss of privileges for incompetence.

                                                                                                                  1. 3

                                                                                                                    Software development has more similarities with pigs wallowing in their own shit than with any serious engineering discipline from my perspective.

                                                                                                                    Just to be clear here, you’re saying you disagree with the entire video, right?

                                                                                                                    1. 2

                                                                                                                      Yes.

                                                                                                                    2. 2

                                                                                                                      If the only reason you can think of and think is constructive to discuss here is one you already know no one will endorse then I don’t think you are behaving virtuously.

                                                                                                                      1. 1

                                                                                                                        Oh, I forgot the comment section only exists to voice agreement.

                                                                                                                        1. 5

                                                                                                                          No, it exists for thoughtful discussion, which you aren’t making even a token attempt to provide.

                                                                                                                          1. 2

                                                                                                                            Yes, of course, the charitable, empathetic, imaginative and otherwise totally reasonable and only possible interpretation of my comment is that that is what I am saying.

                                                                                                                            I could also just have repeated the comment you are responding to, because you’re just doing it again.

                                                                                                                        2. 1

                                                                                                                          I share that suspicion. Giving workers a fancy job title seems to be a common way to motivate them (or at least avoid demotivating them).

                                                                                                                          On the other hand, I also think that construction of some software systems should be done in accordance with an engineering discipline. Control software in airplanes and cars for example. People who practice such discipline deserve to be called engineers. Even if they don’t want to be, they should be because it makes them aware of their accountability.

                                                                                                                          If you’re just making a video game, I don’t care how you do it (putting ethical questions about addictive game mechanics aside for a moment).

                                                                                                                          1. 0

                                                                                                                            I don’t want programming to be a regulated engineering field, for the specific reason that I’m worried about regulatory authorities defining “quality standards”, “gross misconduct” and “incompetence” in politically-biased ways in order to use “loss of privileges” as a stick to prevent types of software that they dislike from being written. Imagine if there was a regulatory body that had the power to strip a license from a programmer if they contributed to social media software that didn’t censor speech that body judged to be harmful to marginalized people (an inherently political judgment); or that kept selectively enforcing code quality standards on cryptocurrency software because a lot of the individual members of the regulatory body thought that cryptocurrency was bad for society.

                                                                                                                            1. 13

                                                                                                                              Just because there are bodies that certify (and de-certify) electronic engineers doesn’t mean that I can’t build circuits in my own time, or even do so professionally. It just means that if Boeing were to hire me, and then a system I built caused an airplane to crash, the people affected would have significantly better grounds to sue Boeing.

                                                                                                                              Engineering certifications are not nearly as effective an instrument of state power as you suggest here.

                                                                                                                              1. 2

                                                                                                                                Isn’t this more about “professional engineers”? The type of person that has a “stamp of approval”.

                                                                                                                                Anyone can work in an engineering field (with training, schooling) but not many are the final approvers who have a professional liability far beyond mere mortals.

                                                                                                                          1. 12

                                                                                                                            I’ve got a friend with a Mech. Eng. degree who spent some time in aerospace (he now does something less glamorous that blows fewer people up). At one point I was telling him that I wasn’t sure exactly what my job role was shaping up to be at a new company — I wasn’t tasked with writing anything in particular, and I wasn’t “architecting”, but I was given a system that was just past proof-of-concept stage and told to get it ready for production, so I was instrumenting it, stress-testing it, eliminating crashes, improving performance, and ensuring it could scale to massively more traffic — and it was turning out to be very gratifying. There were lots of problems, some of them weren’t easy to solve, and every bottleneck eliminated would end up revealing another one hiding just beyond. But as they fell, one by one, you could start to see the system just humming along and doing what it needed to do more and more efficiently, with fewer and fewer glitches.

                                                                                                                            As I was explaining this to him, he said “well, I would call that engineering.” Given the usual reaction of engineers to anyone appropriating that label, I took that as a bit of praise that’s stuck with me ever since, and reminded me that there’s a place for applied science in what we do.

                                                                                                                            1. 3

                                                                                                                              I wonder if this would work better with a real serial port.

                                                                                                                              1. 4

                                                                                                                                Yes.

                                                                                                                                According to docs I found, the VT102 has a 128-character input buffer, and it sends an XOFF (flow control stop) when there are 32 characters in the buffer, meaning that the host can send 96 additional characters before the buffer overflows (or more, if the VT102 processes some of the buffer in the meantime, but I don’t know how fast it processes and won’t make any assumptions). At 9600 baud, 8N1 or 7E1, it takes 100ms for the host to transmit 96 characters. That should be ample time to respond to XOFF.

                                                                                                                              1. 3

                                                                                                                                Field Day.

                                                                                                                                The club I usually go with isn’t operating this year, and I’m pretty sure that their field day tent (which is a big blue triage tent) is on loan to the hospital that they have an agreement with, and being used as a COVID testing tent, so I was planning on doing solo portable. But I’ve been too busy to actually get my gear together for that, and a couple friends told me that their local AUXCOMM has a small operation going, so I’m going to swing by tomorrow, help them get set up, and maybe grab a little time on the air and eat a few burgers.

                                                                                                                                1. 3

                                                                                                                                  It’s going to be operated well. The expertise that the cloud providers can provide and the automation they can afford to implement will likely surpass what you can do, especially across multiple services.

                                                                                                                                  Not… really. They’re operated by people just like us, they make mistakes, they make stupid decisions, and they’re driven by goals that are never in exact alignment with your own.

                                                                                                                                  It’s going to be cheaper.

                                                                                                                                  No it’s not. If it’s something core to your business, managed is going to be way more expensive.

                                                                                                                                  The most expensive AWS RDS instance is approximately $100k/year

                                                                                                                                  Not really fair, one huge DB instance isn’t the right answer for anyone, and one of the biggest thing RDS has is nowhere near the upper limit on growth. My medium-size employer has 40x that many cores running relational DBs. I’m sure we can find a couple people to do care-and-feeding if it means reducing a $4M bill.

                                                                                                                                  It’s going to be faster for development. Developers can focus on connecting these pieces of infrastructure rather than learning how to set them up and run them.

                                                                                                                                  Or rather, they can focus on creating a layer that makes the third-party thing they were given resemble the thing they actually need, instead of building they actual thing. Then they hope and pray that the cloud thing keeps operating the same way in the future, because all of their work can be made worthless at any time.

                                                                                                                                  This is the “build vs. buy” argument and the fundamentals haven’t changed. If you have a requirement that isn’t actually your core business that you don’t want to hire for, buy. If you’re a startup with no budget trying to prove out a concept, buy for everything except that tiny nugget. If it’s something that actually matters, build.

                                                                                                                                  1. 5

                                                                                                                                    I like the article, I think it’s valuable, but please…

                                                                                                                                    Since time immemorial PostgreSQL supports JSON fields and can even index them. By immemorial I mean …

                                                                                                                                    I perfectly remember the world where PostgreSQL had no JSON support

                                                                                                                                    why yes, that is exactly the opposite of “immemorial”! And yes, I know that the phrase “since time immemorial” is almost always used hyperbolically, but what’s the use of a hyperbole that you unceremoniously shoot down just one sentence later?

                                                                                                                                    1. 2

                                                                                                                                      I interpreted that comment as a joke

                                                                                                                                    1. 15

                                                                                                                                      SawyerX’s keynote making this announcement laid out more of the rationale, how Perl 5.32 today begins reading a file as the perl of 20 years ago, then you have all the boilerplate bringing your into the modern world, and what burden that puts on newcomers.

                                                                                                                                      Most things today will run okay under an interpreter that has all this on by default, but if you really, really want to stick to “no strict” by default it will be a one line change to get back. I don’t think this is going to be that burdensome for things like distros that have perl in the base for support scripts - I mean, hopefully they’re already strict safe.

                                                                                                                                      1. 9

                                                                                                                                        A point I found interesting is that with this Perl (5) finally has a roadmap other than “let’s keep things as-is”. There was a hint that Cor, the new object system, may get included in one of the new releases as the object system to be used.

                                                                                                                                        Exciting times. Raku is cool and all, but Perl (5) has this peculiar elegance to it that I haven’t quite seen anywhere else. I’m happy to see the edges smoothed out.

                                                                                                                                        1. 7

                                                                                                                                          A point I found interesting is that with this Perl (5) finally has a roadmap other than “let’s keep things as-is”.

                                                                                                                                          There’s been one of those for about ten years, to be fair. 2000-2010 was kind of stagnant, development-wise, but with the 5.12 series in 2010 we got a yearly release cadence, a deprecation policy, a mechanism for ‘experimental’ features, a new policy on what goes in core, and more focus on new features to make the language nicer to use. Sure, the pace of development hasn’t been blazing fast, and we do still care a lot about back-compatibility (being realistic, if new users aren’t flocking in by the millions, you should at least take care of the ones you’ve got, and their legacy apps), but it’s been a real thing.

                                                                                                                                          Another thing that might not be immediately obvious to outsiders, but that’s been going on for ten years, is the outright encouragement of hacking the language from modules. Perl has always been flexible in that way, but since around 5.12 or 5.14 that’s been ramped up even more with features like keyword plugins, meaning that it’s been possible to create stuff like sub signatures and async/await as completely optional features that anyone can play with, and that can be developed outside of core, and then propose them for adoption as core features down the line.

                                                                                                                                          1. 3

                                                                                                                                            There’s been one of those for about ten years, to be fair

                                                                                                                                            Yes, good point. However, from my perspective it always felt like there was no clear vision in there, other than “let’s see what we can introduce without breaking anything”, and apart from sub signatures (which are still experimental, and I remember being excited about them on YAPC::NA 2013) and the syntax extension ops you mentioned there hasn’t really been anything major on the horizon. All the stuff prototyped in module space can, technically, end up in core one day – but when’s the last time that has happened? Even the aforementioned sub signatures happened only because of one dedicated guy who wasn’t allowed to use CPAN at work so he needed to put them in core.

                                                                                                                                            As 5.32 was approaching release, I had no idea if there’ll even be anything to be excited about. It’s comforting to see that there’s now a bold vision for the future. Sawyer’s talk really spoke to me – I’d like my Perl to by nice by default, rather than “I have enough years of experience to bend it into something nice, most of the time”.

                                                                                                                                            1. 3

                                                                                                                                              Yeah, no doubt. I like it too, I don’t mind seeing a little more boldness. It’s either do that or fade out entirely. I just want to give a little credit to all the cool folks who haven’t been sitting doing nothing since 2000. This is a culmination, not a bolt from the blue.

                                                                                                                                        2. 1

                                                                                                                                          The video is marked as private? Will it be publicly available later?

                                                                                                                                          1. 1

                                                                                                                                            Oh, they must have edited and reposted, I’ve updated the link.

                                                                                                                                            1. 1

                                                                                                                                              Thanks!

                                                                                                                                        1. 2

                                                                                                                                          Getting by, here. I live outside the city, and haven’t gone in since early March, which doesn’t bother me one bit. Still working the same job, doing the remote thing, which is also not a huge change — just that before 25% of meetings were Zoom calls and now 100% are.

                                                                                                                                          Getting out of the overcrowded open-plan and skipping the commute is nice, sharing a small house with my wife during the workday (not to mention pretty much 24/7) is challenging.

                                                                                                                                          Been low on motivation at work but I think that mostly comes from having gotten put onto a project in Feb that sounded interesting but has actually been totally dull, with not a ton for me to do. I’m working on fixing that, but in the meantime it leaves me time for some of those personal projects I never have enough time for.

                                                                                                                                          I ignore news of the outside world to the maximum extent possible. It does nothing good for me.

                                                                                                                                          1. 1

                                                                                                                                            I’ve been fighting with this lately. I’m writing a client for an amateur radio transceiver that sends audio as a stream of UDP packets, and I have to deal with sample rate drift between the sender and the receiver to make sure that buffers don’t overrun or underrun. On Ethernet I’ve got things working pretty well. On WiFi… wow. I generally thought my network was pretty fast and worked pretty well. But if I run the client on my laptop on WiFi, the amount of jitter, dropped packets, and reordered packets I get… Well, it’s been a real good test of the quality of my sync algorithms, I can say that.