1. 4

    Are there other types that are always on the heap? It turns out there are!! Here they are:

    • Vec (an array on the heap)
    • String (a string on the heap)

    That sounds wrong to me. Both Vec<T> and String­—as far as I know—are triples: a length (usize), a capacity (usize), and a pointer into the heap. Those three values can be stored in the stack, however the pointer will always point to a memory block in the heap.

    1. 8

      Yes, you’re correct here; the author is being a bit less precise than you. It’s how many people think about it most of the time, though.

    1. 4

      I’ve settled for now on Tilix after trying or re-trying various including:

      • urxvt
      • terminator
      • terminology (in the article)
      • tilda (in the article)

      I want something like the OSX iTerm which just seemed to do what I want without a lot of friction. I’d prefer tabs and panes over windows but as long as I get fast rendering, 24-bit color, and can remap the keystrokes I’m happy. Tilix seems to fit the bill. I’d like to take advantage of triggers but haven’t gotten around to rebuilding vte.

      1. 1

        I’ve also been using tilix for a few weeks now. I was a user of urxvt before, but I wanted to have tabs (otherwise I end up with 15-20 terms opened) and I wanted better font rendering than urxvt. I tried alacritty, but its lack of features was problematic (no true underlining, no scrollback, can’t dim the terminal when it’s not the active window, etc.)

        Tilix is a pretty good terminal emulator. I have one problem with it, when I launch it, sometimes it fails to raise to the top in Openbox, so I need to Alt-Tab it in.

      1. 3

        I like this book, and I like Wirth’s approach to compilers in general (i.e., keeping them very simple), but the typesetting in his documents is so bad that it makes reading them harder than it needs to be.

        1. 3

          ISTR the printed copy I had was much more readable than this PDF, although it was once I learned that of course all his works were prepared on the same OS, computer, et cetera he built, it made sense why they all had kind of lousy formatting. Which is really too bad, I agree.

        1. 1

          I needed to implement a simple (though not small) database, and I was unsure how to persist data to disk as it was my first such project. I was thrilled to find this paper: it is short, to the point, and explains a simple and easy-to-implement technique; the only thing I’m sad about is that I did not think of it myself :)

          1. 1

            What is it about inference rules, regular expressions, and BNF grammars that allow us to understand them perfectly well, even with the syntactical variance? If simple syntactic differences can cause confusion with overlines and substitution, shouldn’t their expression be re-thought entirely? For instance, would the example that mixes overlines and substitution not be clearer if two for-all quantifiers (∀) were used?

            1. 1

              Presumably, if you used a quantifier, you’d still have the problem of delimiting the extent of the quantified variables. As discussed in the talk, you would need to signal grouping with an overline, parens, binding dots, or similar.

              1. 1

                What is it about inference rules, regular expressions, and BNF grammars that allow us to understand them perfectly well, even with the syntactical variance?

                Regular expressions and BNF grammars are easy to understand because they’re just powerful enough for their intended purpose - specifying the syntax of programming languages, which is an easy task provided you don’t purposefully make it hard.

                On the other hand, I’m not sure we understand inference rules that well.

                For instance, would the example that mixes overlines and substitution not be clearer if two for-all quantifiers (∀) were used?

                Not really. The trouble with a substitution like t[xs -> ts] is that it doesn’t really mean what it literally says. You don’t actually have two separate lists xs of variables and ts of terms to substitute them with. You have a single list ss of pairs of the form (x,t), where x is a variable and t is a term to substitute it with. (Alternatively, you may insist on using two separate lists, but then you would need to maintain an invariant that the lists have the same length. IMO, this makes life unnecessarily hard.)

              1. 2

                I just want to say, good luck, we’re all counting on you.

                1. 1

                  It’s his ship now, his command; he’s in charge, he’s the boss, the head man, the top dog, the big cheese, the head honcho, number one…

                1. 4

                  Yes, yes, yes! When I did literate programming, I realized that the real value was not the good-looking PDF, but rather the narrative for understanding the solution (though I must admit I sometimes missed the mark and the text helped not at all). I don’t know about other people, but I typically find it difficult to jump into a code base, read it, and get a good sense of why things are the way they are, e.g., choice of data structures and algorithms, seemingly premature extension points, etc. It’s doubly true with business logic: a complex piece of code might exist only because a client 5 years ago needed that feature and it was never phased out or because of an obscure law that only exists in the province of Quebec.

                  One question I’ve been pondering lately is how to encourage good story telling in software. I agree with the article that git history is not it. I thought literate programming might be a way, but currently there are difficulties with the existing tools that make LP unpractical—for example, most editors don’t deal well with the mix of LaTeX and the unordered chunks that only make sense when tangled together. In addition, I found that it was more difficult to “refactor” prose than code, so I was more reluctant to make changes in my software because I had no mechanical way—either tests or a type checker—to check that I updated everything relevant.

                  @akkartik’s layered software approach is interesting: each individual layer could have a README file that describes why it exists. I haven’t tried it myself (maybe I should for this year’s Advent of Code in December?), so I can’t comment on how it is in practice, but I certainly like the idea and love that someone is pushing in that direction.

                  If people have good tips for including a narrative in a software project, I’d love to hear them. I even wonder if including a video of the author drawing a diagram might be a good idea.

                  1. 2

                    I’d love to hear about your experiences if you try this out. Layers rely on tests to gain confidence that you caught all the places that need to be changed. They’re harder to refactor, yes, but I’d say they’re just exposing you to the full difficulty of the refactoring (explaining rationale) rather than letting you shove it under a carpet somewhere like conventional approaches do. They make you face up to the essential difficulty of the operation.

                    Since each layer is a single file, I think of the comment at the top as the README for the layer. That’s one of the surprising benefits I found from using layers: they provide a Schelling Point where readers can find the big-picture rationale for a feature. Conventional organizations often scatter the code for a feature too much to admit a strong Schelling Point. At work I often find the most useful comment about a feature in some header somewhere after weeks of wrong turns trying to understand it. In contrast, here are a few layers with useful comments up top (copying from a recent comment).

                    @smalina and I have been discussing this story quasi-privately at https://www.reddit.com/r/BarbarianProgramming/comments/74coqq/software_is_about_storytelling.

                    I’ll also be interested to hear your take on my critique of classic LP.

                    1. 1

                      I had always wondered why this wasn’t done at all when I first started programming. Context is for kings and we containerise everything BUT context. Thanks for linking that subreddit!

                      Slight tangent: @litonico_ had a cool strangeloop talk on context.

                      1. 2

                        Interesting. That actually reminds me of a talk from a previous Strange Loop. Submitted.

                        Edit: still watching your talk. It’s a really nice format, stringing pre-recorded text editor sessions together.

                    2. 1

                      Literate programming is a very good idea that is unfortunately made impractical by our existing tools. Most technical books don’t contain just text. They contain formulas, diagrams, pictures and whatever it might take to get the important ideas across. Literate programs ought to contain formulas, diagrams and pictures too, if those happen to be the easiest way to explain their inner workings.

                      1. 1

                        That’s why literate programs are often in LaTeX :)

                        1. 1

                          As much as I love LaTeX for typesetting mathematics, it is impractical for annotating programs the way I want to. I design my programs using paper and pen, which gives me the freedom to use whatever notation I want, without the inconvenience of explicitly importing such and such LaTeX package just to support the notation. Ideally, my program annotations would be (selected excerpts from) the design papers themselves. Maybe I’m willing to put some effort to redraw the diagrams more cleanly. But I’m not willing to put the effort to typeset those diagrams.

                          1. 1

                            I’ll be interested to see these programs you’re writing today, and the diagrams you want to include with them that are hand-drawn cleanly but not typeset. Are you sure the diagrams are actually useful? To anybody but your self?

                            Writing actually went through a lengthy period when every writer had his own unique-snowflake style of writing. Outgrowing this phase was a major advance.

                            To summarize my argument: maybe diagrams would be useful to include with software. However:

                            • Literate Programming would be plenty useful without them as well, so it’s not worth blocking on integrating diagrams with IDEs before we start using it, and

                            • Diagrams will always be a lot of work. Think about blueprints in other fields today. Anything not typeset that you’re coming up with is likely to be more of a liability than an asset.

                            1. 1

                              Are you sure the diagrams are actually useful? To anybody but yourself?

                              Completely sure. For a mundane example, if you are implementing self-balancing search trees, the easiest way to convince yourself that you have considered all possible tree rotations is to draw them all (up to symmetry).

                              Writing actually went through a lengthy period when every writer had his own unique-snowflake style of writing. Outgrowing this phase was a major advance.

                              I’m talking about expressing ideas using formulas and drawings, not about using this or that font.

                              Diagrams will always be a lot of work.

                              Diagrams can be drawn almost instantaneously using paper and pen. Of course, using LaTeX, they require a lot more work.

                              1. 1

                                Ok, I understand better what sort of diagrams you mean. But the slight change in cost of diagramming based on the tools you choose is irrelevant if the software you write will be read many many times. At its root, Literate Programming is about making that aspirational assumption, that software will be read orders of magnitude more times than it’ll be written. It’s not true today, but I’d like to live in a world where it is true. And that’ll never happen without first making things easy to read. It’s a chicken-and-egg problem.

                                It’s fine if you’re not on board with this agenda. But this is why Literate Programming is as it is. It does not care that diagrams take a little extra effort. You just have incommensurate goals, both equally ‘impractical’ by today’s standards.

                                1. 1

                                  And that’ll never happen without first making things easy to read. (…) It’s fine if you’re not on board with this agenda.

                                  I’m certainly on board. However, typesetting is neither necessary nor sufficient for documentation to be easy to read. What is actually needed is clarity of ideas and clarity of exposition.

                                  1. 2

                                    Absolutely, 110% agreed! I think I understand your position now. You want to prioritize convenient diagramming support before typesetting. I’m totally down with that. But it seems really hard given that competing with pen and paper – while also quickly bouncing to and from a keyboard – will require significant hardware support.

                        2. 1

                          Given that all our programs today don’t rely on non-text media (except for the odd bit of ASCII art), I think this new constraint you’re introducing is actually more impractical than the limitation you’re complaining about. Pure-text literate programs (with zero LaTeX or pdf) would be strictly superior to the current status quo.

                      1. 39

                        The argument seems to rely on the cost of static typing, which is stated in the following four points that I challenge:

                        It requires more upfront investment in thinking about the correct types.

                        I don’t buy this argument at all. If you don’t think about correct types in a dynamic languages, you will run into trouble during testing (and production, when your tests aren’t perfect). You really have to get your types right in a dynamic language too. Arguably, with a statically typed language, you have to think less because compiler will catch your error. I think that’s the whole point of statically typed languages (performance concerns aside).

                        It increases compile times and thus the change-compile-test-repeat cycle.

                        I’d have to see some proof to show that static typing plays a significant role in compile time. I’ll buy that you can make a very complicated (perhaps turing complete) type system and that would have a big impact. But there are statically typed languages that compile really fast, and most of the compile time is probably not spent on types. I’d argue that it is likely for the compiler to catch your error with types faster than you could compile, run, and test to find the same error with no types.

                        It makes for a steeper learning curve.

                        That may or may not be true. Sure, a type system can be very complicated. It doesn’t have to be. On the other hand, a dynamic language will still have types, which you need to learn and understand. Then, instead of learning to annotate the types in code, you learn to figure out type errors at run time. Is that so much easier?

                        Either way, I don’t believe type systems are an insurmountable barrier. And I think some people give the learning curve way too much weight. Maybe they are working on throwaway software on a constantly changing faddy tech stack, in a place with high employee turnover. It’ll matter more. I suppose there’s a niche for that kind of software. But I’m more into tech that is designed for software that is developed, used, and maintained for years if not decades. A little bit of extra learning up front is no big deal and the professionals working on it will reap the benefits ever after.

                        And more often than we like to admit, the error messages a compiler will give us will decline in usefulness as the power of a type system increases.

                        That might be the case with the current crop of languages with clever type system, though I don’t know if it’s inherent. Do static type system need to be so powerful (read: complicated) however? A simpler system can get you just as much type safety, at the expense of some repetition in code.

                        I think there are dimnishing returns, but not due to the cost of static typing as such, but rather due to the fact that types just don’t catch all errors. Once the low hanging fruit is out, there’ll be proportionally more and more logic errors and other problems that aren’t generally prevented with types. Or you could catch these with extensive type annotation, but the likelihood of preventing a real problem becomes small compared to the amount of annotation required. And then there’s the usual question: who checks the proof?

                        There have been some famous bugs that resulted from systems using different units. So if these numeric quantities were properly typed, these bugs would have been prevented. However, what if we change the scenario a little, and suppose we’re measuring fuel or pressure, in the right units. But we read the wrong quantity – spent fuel instead of stored fuel, or exterior pressure instead of interior pressure? Sure you can add more types to prevent such misuse, but it gets more and more verbose, and then you’re moving closer and closer to re-expressing (and thus enforcing) the program logic in the language of the type system; we could consider that to be a language of its own.

                        Now you have two programs, and one can prevent bugs in the other, but both could still be buggy. And the other program starts to grow because you start needing explicit conversions to enable the code to actually perform a computation on internal-pressure-in-pascal. Of course you are subverting the type system when you say you really want to convert internal-pressure-in-pascal to just pressure-in-pascal or whatever. Bugs ahoy?

                        1. 18

                          A simpler system can get you just as much type safety, at the expense of some repetition in code.

                          I agree with most of the rest of your comment, but this part is untrue. Stronger type systems do allow you to enforce more powerful laws at compile time. At one end of the curve we have a type system like C’s, which barely buys you anything, and then at the other end we have full dependent types where you can prove arbitrary invariants about your code (this function always terminates, this value is always even, etc.) that you cannot prove in a weaker type system. In between is a huge spectrum of safety checking power.

                          1. 7

                            The C type system can actually be quite powerful if you wrap basic types in one element structs. struct meter { int v; } and struct foot { int v; } can’t be added by mistake, but can still be worked with using one line inline functions with no performance penalty. It’s just work (which nobody likes).

                            1. 5

                              I would not describe that as “quite powerful” at all. That’s one of the most basic things a type system can give you.

                              You can’t really prove any interesting properties until you at least have proper polymorphism. Java doesn’t, for example, because every object can be inspected at runtime in certain ways. In a sensible type system, there are no properties of objects except those which are explicitly stated in the type of the object.

                              In such a type system, you can prove interesting properties like that a data structure does not “depend” in any way on the objects in contains. For example, if you could implement a function

                              fmap :: (a -> b) -> f a -> f b
                              

                              Which “mapped over” the contents of your object with some function, this would prove that your object never inspects its contents and therefore its structure does not depend on the values of its contents (because this function is universally quantified over ‘b’, and therefore you could map every ‘a’ to a constructed type which cannot be inspected in any way).

                              You can prove all sorts of useful properties like this (often without even realize you’re doing it) once you have proper quantification in your type system. One of the coolest quantification-based proofs I know of is that Haskell’s ST monad is extrinsically pure.

                              As you add more power to your type system (up to full dependent types, linear types, etc.) you can prove more and more useful things.

                              1. 2

                                As long as you like all your types disjoint, sure. But I’ll pass.

                                1. 2

                                  So what’s wrong with disjoint types?

                                  1. 2

                                    It doesn’t you have rationals and floats that are both numbers for example.

                                    1. 2

                                      In Ocaml ints and floats are different types and operators like (+) only apply to ints, one has to use (+.) for floats. It’s not a problem IME.

                                      1. 1

                                        I think automatic type conversion of ints to reals was the original sin of FORTRAN.

                                      2. 1

                                        In mathematics the system Z of integers and the system R of reals are different. The number 3 has different properties depending on system context - for example 3x = 1 has a solution in the second context.

                                  2. 0

                                    But it lacks a keyword connection to category theory.

                                2. 12

                                  I don’t buy this argument at all. If you don’t think about correct types in a dynamic languages, you will run into trouble during testing (and production, when your tests aren’t perfect). You really have to get your types right in a dynamic language too. Arguably, with a statically typed language, you have to think less because compiler will catch your error. I think that’s the whole point of statically typed languages (performance concerns aside).

                                  That’s a good point and one that took me a long time to learn: if a concept cannot be expressed in a language, it doesn’t magically disappear and absolve the programmer from thinking about it. Types are one example as you mention; similarly, in many discussions about Rust, some people mention that the borrow checker is an impediment to writing code. It’s true that some programs are rejected by the compiler, but lifetime and ownership are also concerns in C programs as well. The main differences are that in Rust you have rules and language constructs to talk about those concerns while in C it’s left to documentation and convention.

                                  1. 7

                                    But there are statically typed languages that compile really fast, and most of the compile time is probably not spent on types.

                                    OCaml is a good example of this.

                                    1. 3

                                      I don’t buy this argument at all. If you don’t think about correct types in a dynamic languages, you will run into trouble during testing (and production, when your tests aren’t perfect). You really have to get your types right in a dynamic language too. Arguably, with a statically typed language, you have to think less because compiler will catch your error.

                                      It’s true that you have to get the types right in a dynamic language, but the appeal of dynamic languages isn’t that you don’t have to think about types. It’s that you don’t have to think about the shape of types. For example:

                                      def make_horror_array(depth: int):
                                          arr = []
                                          deepest_arr = arr
                                          
                                          for i in range(depth):
                                              deepest_arr.append([])
                                              deepest_arr = deepest_arr[0]
                                          deepest_arr.append(depth)
                                          return arr
                                      

                                      What type should that return? Contrived, but it’s not the gnarliest type problem I’ve run into. Sometimes it’s nice to have a language where I can give up on getting the types right and rely on tests and contracts to check it.

                                      1. 4

                                        It’s that you don’t have to think about the shape of types.

                                        How do you use a value without thinking about it’s type or shape?

                                        In your example, you can’t just blindly apply numerical operation to first element of that horror array since it might be another array. So, if you wanted to get to the value inside of those nested arrays, you’d need to think about how you would “strip” them off, wouldn’t you? And wouldn’t it mean that layers of nesting have some special meaning for us?

                                         

                                        Taking your implementation as reference:

                                        >>> make_horror_array(0)
                                        [0]
                                        >>> make_horror_array(1)
                                        [[1]]
                                        >>> make_horror_array(5)
                                        [[[[[[5]]]]]]
                                        >>> make_horror_array(10)
                                        [[[[[[[[[[[10]]]]]]]]]]]
                                        

                                        we can write a Haskell version that distinguishes between a value nested in a “layer” and a value by itself:

                                        λ> :{
                                        λ> data Nested a = Value a | Layer (Nested a)
                                        λ>
                                        λ> -- just for presentation purposes
                                        λ> instance Show a => Show (Nested a) where
                                        λ>   show (Value a) = "[" ++ show a ++ "]"
                                        λ>   show (Layer a) = "[" ++ show a ++ "]"
                                        λ> :}
                                        λ>
                                        λ> mkHorror n = foldr (.) id (replicate n Layer) $ Value n
                                        λ> :type mkHorror
                                        mkHorror :: Int -> Nested Int
                                        λ>
                                        λ> mkHorror 0
                                        [0]
                                        λ> mkHorror 1
                                        [[1]]
                                        λ> mkHorror 5
                                        [[[[[[5]]]]]]
                                        λ> mkHorror 10
                                        [[[[[[[[[[[10]]]]]]]]]]]
                                        

                                        and if we don’t need layers anymore, we can get value out pretty easily:

                                        λ> :{
                                        λ> fromNested :: Nested a -> a
                                        λ> fromNested (Value a) = a
                                        λ> fromNested (Layer a) = fromNested a
                                        λ> :}
                                        λ>
                                        λ> fromNested (mkHorror 0)
                                        0
                                        λ> fromNested (mkHorror 5)
                                        5
                                        
                                        1. 4

                                          Assuming it’s correct, it should return whatever the type inference engine chooses for you :)

                                          1. 1

                                            This is because type theory is confused about what programs do - which is operate on bit sequences (or, these days, byte sequences). These sequences may be representations of mathematical objects or representations of things that are not mathematical objects, but they remain representations, not actual ideal mathematical objects.

                                          2. 1

                                            Now you have two programs, and one can prevent bugs in the other, but both could still be buggy.

                                            Aren’t a lot of type systems proven type-correct nowadays?

                                            1. 3

                                              The type system can be fine but the rules you define with the types could be flawed. Thus, you can still write flawed programs that the type system can’t prevent because the types were defined incorrectly.

                                              1. 1

                                                Can you give an example? I’m not sure exactly what breed of incorrect types you’re referring to.

                                            2. 1

                                              It requires more upfront investment in thinking about the correct types.

                                              I don’t buy this argument at all. If you don’t think about correct types in a dynamic languages, you will run into trouble during testing (and production, when your tests aren’t perfect).

                                              One thing I am learning from working with inexperienced developers is that even thinking about which container type you are using is a challenge. Should your function return a Seq? An Array? An Iterator? A Generator? And what if your new library returns a Generator and the old one returned an Iterator and now you have to rewrite all your declarations for seemingly no reason at all? Some kind of “most general type” restriction/requirement/tool would help with this…

                                              1. 2

                                                This is one of the things I think Go does really well (in spite of having a generally quite weak type system) - thanks to implicit interfaces, you can just return the concrete type you’re using and the caller will automatically pick up that it ‘fits’.

                                                1. 1

                                                  This sort of works – but even with that system, it’s easy to declare one’s types too tightly.

                                                  It depends in part on how granular the collection library’s interfaces are (ditto for numeric tower, effects tracking, monad wizard tool).

                                                2. 1

                                                  I’m unclear what you mean. Many languages offer two solutions. You can declare the variable IEnumerable or whatever as appropriate. Or you declare the variable as “whatever type the initializer has”.

                                                  1. 3

                                                    When in doubt, use the inference!

                                                    1. 1

                                                      It is sometimes easy to choose wrong. Iterable vs Iterator vs Enumerable

                                                1. 1

                                                  But why? What does this change?

                                                  1. 3

                                                    He used to represent a cell with a btree but changed to pack the info (which of these 9 values is still possible) into a binary field. He says 32 bits, so presumably he ignores 21 bits and stores the 9 true/false possibilities in the remaining 9 bits. Big wins for allocating less, fitting in caches, following fewer pointers.

                                                    Solving Sudoku is all about finding which single possibility remains. Popcount is an opcode for counting the number of 1 bits in a register, so it directly optimizes a key operation.

                                                    1. 1

                                                      When I specify the architecture, the translation of count_ones() becomes the instruction popcnt; I haven’t explored the issue in much detail, but since I use the population count to know if a cell is solved or not and do this a lot, I suspect that using this native processor instruction contributes to the speed-up of the program.

                                                      1. 0

                                                        This is equivalent to -mcpu=native in GCC.

                                                      1. 22

                                                        I thought the point of pair programming was not to increase efficiency, but to allow a junior programmer to understand how a senior works and improve their own skills.

                                                        1. 6

                                                          This is incorrect. To quote from the Extreme Programming website:

                                                          One thing pair programming is not is mentoring. A teacher-stundent relationship feels very different from two people working together as equals even if one has significantly more experience.

                                                          An organization wanting to improve their junior developers through pairing will be very different from one that strives to practice pair programming proper. A few differences:

                                                          • Pairing seniors with juniors vs pairing equal partners.
                                                          • Pairing some of the time vs pairing all the time.
                                                          • Pairing to improve skills vs pairing to improve the code.
                                                          • Giving senior more authority vs partnering as equals.
                                                          1. 2

                                                            This is incorrect; look at u/gregnavis’s post.

                                                            Aside from that, efficiency != efficacy. https://english.stackexchange.com/questions/50662/is-there-any-difference-in-meaning-between-efficacy-and-efficiency

                                                            1. 0

                                                              Yeah, I think that’s the way experienced coders need to understand it. At least since 2010, new coders have outnumbered us by at least 5 to 1. They will either learn on their own, the hard way, probably hurting your company’s viability in the process, or they will need good mentoring. For the few years I did pairing, it was really an excuse to get us to mentor, and not just write solo code all week. And I think in the long term, it was worth it in time/quality tradeoffs.

                                                              1. 1

                                                                Since the sixties (as far as I’m aware), roughly half of programmers have had less than 5 years experience, with half the remainder having fewer than 10. That’s how fast people are entering the profession!

                                                            1. 41

                                                              I don’t think syntax highlighting is a sign of weakness or not understanding a language.

                                                              1. 7

                                                                I wonder if someone would see it as a weakness to use proper formatting or to use obviously named variables. It’s just a preference like tabbed spacing or anything else.

                                                                1. 6

                                                                  On one hand, humans have color perception for a reason, so not using syntax highlighting is consciously handicapping yourselves.

                                                                  On your other hand, the article argues that syntax highlighting distracts you from semantics, so maybe what we should really use is semantic highlighting. But even then you’d still be using colors, so either way.

                                                                  1. 14

                                                                    What I’ve found is that, especially in Vim, because just about everything is highlighted, it ends up just looking like a big jumble of colours and nothing stands out (it’s also very inconsistent and the syntax files are a mess). And then you’ve got issues like native types being highlighted, while custom types aren’t recognized which is just confusing. I’m a fan of minimal highlighting, and I’ve gradually dialed my personal colour scheme right back to only highlighting comments and ‘TODO:’, ‘NOTE:’ (so they really catch my attention), and a few very specific things like function/method definitions to make it easier to visually scan a file.

                                                                    1. 7

                                                                      I think colors are best reserved for marking important things. Splashing the code with a rainbow of colors prevents anything from standing out.

                                                                      1. 6

                                                                        I wrote on this topic; I agree that going without syntax highlighting is making it harder on yourself (e.g., not noticing that the code you are looking at is in a huge comment block or that you used an incorrect escape sequence), but too much highlighting and nothing particularly stands out. I made my own theme where most text is in one color, and I highlight comments, string literals, function definitions and a couple of other constructs. I particularly like the highlighting of function definitions, it helps my to quickly see where one function starts and where another begins.

                                                                        I don’t have any particular talents in art, color theory, design, UX, and all that, and I’m sure that a competent theme designer could take only 3-4 colors and create a theme that really highlights the value of syntax highlighting.

                                                                        1. 3

                                                                          not using syntax highlighting is consciously handicapping yourselves

                                                                          While understanding that syntax highlighting, like editor and programming language choice, is highly subjective, I disagree with this statement.

                                                                          I personally find that disabling syntax highlighting, and all colors in my terminal, helps me to focus on the actual semantics. I find that I actually read the code more carefully and retain more of the substance than when using highlighting.

                                                                          I also find that disabling highlighting is particularly useful for viewing files written in programming languages I’m less familiar with. While yellow may mean parameter in one language, it may mean class declaration in another. Disabling colors completely removes any chance of information bias based solely on a first glance.

                                                                          I would love to see any studies you have to support the theory that not using syntax highlighting when reading code is an intentional handicap. I would also love to know in what ways you think it is handicap. Is my understanding of the code compromised? Am I slower and less productive? How do you measure how handicapped I am?

                                                                          1. 3

                                                                            I personally find that disabling syntax highlighting, and all colors in my terminal, helps me to focus on the actual semantics. I find that I actually read the code more carefully and retain more of the substance than when using highlighting.

                                                                            “On your other hand, the article argues that syntax highlighting distracts you from semantics, so maybe what we should really use is semantic highlighting.” :P

                                                                            I would love to see any studies you have to support the theory that not using syntax highlighting when reading code is an intentional handicap.

                                                                            Here you go!

                                                                            1. 2

                                                                              “On your other hand, the article argues that syntax highlighting distracts you from semantics, so maybe what we should really use is semantic highlighting.” :P

                                                                              Yes, that’s why my statement was prefaced with “I personally”. I interpreted your use of “on your other hand” (emphasis mine), to mean you were incredulous of that argument. I was adding my own personal experience and preferences to the discussion.

                                                                              I can’t speak to the papers, as I’m reading them now, but I appreciate the sources. You did not, however, answer how I am handicapped. Simply trying to authoritatively state that forgoing highlighting makes me perform at a lesser degree than someone using highlighting is a broad and vague statement.

                                                                              1. 2

                                                                                That’s fair. My position is more philosophical: colors are very information dense for humans, so we should leverage that for parsing code. That doesn’t mean any particular syntax highlighting scheme Is Good, or even that our current position on syntax highlighting Is Good- I think it’s helpful, but there’s a lot of room for improvement.

                                                                                One thing I haven’t really seen, but am really into the idea of, is semantic highlighting. That would be things like “color any function that’s imported somewhere else in the codebase” or “highlight any variable I later mutate.” Those would potentially be a lot more powerful than just coloring keywords and such, but would also be trickier to write a highlighter for, which might be why nobody’s done it yet.

                                                                                Edit: “your other hand” was a typo :/

                                                                                1. 2

                                                                                  That’s also fair. I was mostly just wanted to dig into the theory that anyone not using highlighting is intentionally handicapped. I, clearly, disagree there but I definitely accept that feelings around highlighting, like most programming-related meta-things, are highly subjective.

                                                                                  While I do not use highlighting, I’m also interested in seeing how tools can improve to help people perform their tasks to the the best of their ability. I’m not opposed to highlighting existing and would love to see improvement made to make highlighting more useful. Semantic highlighting would be very interesting and I would definitely give it a try.

                                                                                  I just don’t don’t buy that I’m at a disadvantage by not using normal syntax highlighting.

                                                                                2. 2

                                                                                  Neither paper actually says I am at a handicap. What they do say is that syntax highlighting is useful among certain portions of the programming population for quickly identifying certain characteristics about a program. Neither explores whether the users already used syntax highlighting, or the same tools and same colors used in the study, or how the highlighting affects the understanding of a program in someone who normally does not use highlighting,.

                                                                                  I believe you are misrepresenting the data in those papers as “not using syntax highlighting is intentionally handicapping yourself” when in fact the first paper says syntax highlighting can be beneficial for identifying certain program constructs (first paper) and the second paper clearly states that “the magnitude of the effect [syntax highlighting has] decreases as programming experience increase”, though it does say it can reduce context switching.

                                                                                  So, my question is still, why do you think I’m at a disadvantage and how does this manifest?

                                                                                3. 1

                                                                                  Very interesting links, thanks for sharing. When I was debating a similar subject, I was looking for similar documents but could never find any.

                                                                              2. 2

                                                                                On one hand, humans have color perception for a reason, so not using syntax highlighting is consciously handicapping yourselves.

                                                                                I’ve never seen this brought up about IDE’s. That’s a great point. It’s reinforced in many other areas such as UX and marketing materials. Hell, even the milk I buy is always red since many generic, grocery items are categorized by color for instant recognition. It’s proven to be a useful psychological effect in general. So, we should leverage color for categorization in IDE’s. How to best do that is obviously still an open topic but keywords vs functions vs type vs etc have been useful so far.

                                                                                1. 1

                                                                                  keywords vs functions vs type vs etc have been useful so far

                                                                                  I don’t agree with this at all. I appreciate that people like it, but I bet many orders of magnitude more time have been wasted trying to tell the difference between $accountCount and $accountCⲟunt than between private and personal.

                                                                                  Maybe even more when you consider colour fatigue tricking the programmer into thinking there’s no difference between the lines…

                                                                                  1. 1

                                                                                    Hmm. We could make the keywords all one color with the usesr-supplied identifiers on a given page being different colors. How about that?

                                                                                    1. 1

                                                                                      Maybe. It certainly sounds more useful than what vim and sublime editor do. There’s some risk though, and it’s unclear how common purposeful variable shadowing is:

                                                                                      let accountCount=get(); 
                                                                                      do_something(function() {
                                                                                        let accountCount=get();
                                                                                        ...
                                                                                      });
                                                                                      

                                                                                      The two “accountCount” variables above should be different colours, and while shadowing occurs frequently in my programs, in other programs it might be a bug.

                                                                              3. 2

                                                                                Obligatory quote: http://aiju.de/rant/syntax-highlighting

                                                                                That is what overly highlighted code looks like to me.

                                                                                For me, proper indentation / spacing and code layout is way enough for reading.

                                                                                For writing, highlighting strings reveals to be appreciated to me, but still, in languages such as shell script, you can quote every single word or let it unquoted. Syntax highlighting for strings then become totally pointless.

                                                                                When I use vim, I switch between :syntax off for less rainbow reading and writing and :syntax on when I have a doubt about a string quote in a shell script or such.

                                                                                Proper color theme is also a good compromise.

                                                                                1. 2

                                                                                  What I am often looking for with a syntax highlighter is a good linter instead. A syntax highlighter is also a linter that display the result as colours on the text…

                                                                              1. 10

                                                                                Sweet, a research paper that tackles something I’ve been talking about anecdotally for years when asked why I carry a notebook and mechanical pencil with me around the office.

                                                                                1. 7

                                                                                  I posted it sort of to vindicate my comments in The Return of the Hipster PDA thread, and because it actually came up just today on a site I read, The Imaginative Conservative. I decided to find and link the study to avoid discussion devolving into politics, but the original article that lead me to the survey may be worth reading for some.

                                                                                  1. 3

                                                                                    So when I read “The Imaginative Conservative” I had no idea what that could even mean, but I feared the worst (which probably says something about our times). I must say, however, that despite being pretty damn far to the left myself I enjoyed some of the articles.

                                                                                  2. 6

                                                                                    This is something I really appreciate about about my current workplace. We’ve got laptops and insane amounts of compute power around the office, but the unspoken culture is to prefer paper notebooks. It’s nice not feel out of place going scritch-scritch instead of click-clack during meetings.

                                                                                    (It does get a little odd meeting groups of external people, however.)

                                                                                    1. 3

                                                                                      Many years ago, when taking my degree in physics, I noticed that for those subjects where I had made longhand notes during lectures I had a much better grasp of the details of the subject immediately afterwards - revision consisted of a quick scan over my notes. In the subjects with handouts, I was having to go over the handouts after the lectures in order to make sure that I understood everything. As a result I made a point of trying to keep notes even during lectures that came with comprehensive handouts.

                                                                                      I’m not sure whether the note taking itself was helpful, or whether it was simply that it absolutely forced me to engage with all of the material during the lectures: Without having to make notes, I could convince myself that I was paying attention when in fact I might have been missing bits and pieces of the material that could significantly effect my comprehension later & mean I had to spend extra time going over material that, had I taken notes, I would have been familiar with already.

                                                                                      1. 2

                                                                                        Lately, I have found that it is extremely hard for me to think without having pencil and paper; I write down what I want to do do, I sketch a few data structures, jot dot some pseudo code I think may be relevant, etc. I find that I cannot do the same kind of exploration at a keyboard, even using a great program like org-mode.

                                                                                      1. 2

                                                                                        If you want a great “starter pack” for Emacs, I’ve looked at a bunch, and Prelude is by far my favorite one.

                                                                                        Emacs Redux is also a great blog where you can learn and pick up neat tricks.

                                                                                        1. 7

                                                                                          I would avoid heavy starter kits; they add quite a lot and customize Emacs significantly, and it can be more work to undo the choices they made than to add the configurations that you need.

                                                                                          Rather, I encourage people to scour for emacs.d repositories on github: many are very well commented, and you can easily start grabbing things that interest you.

                                                                                          1. 2

                                                                                            I dunno. I agree with the sentiment, but some of the default Emacs settings are really quite off-putting, and could give beginners a poor first impression. (Littering your disk with backup files & the shitty way buffer names for files with identical names are uniqified comes to mind) so I think it could make sense to use starter kits when you start out and then ditch it and grab just the bits you need / want from the kit. Of course I’m biased, because it’s what I did :-)

                                                                                            For what it’s worth, I used the “better defaults” package.

                                                                                            1. 1

                                                                                              That’s the neat thing about Prelude, unlike other kits it’s not heavy at all.

                                                                                          1. 52

                                                                                            Just as important as Firefox being a distinct product is that it uses its own rendering engine. There are only three major browser rendering engines: Trident/WhateverTheyCallTheEdgeEngine (Microsoft), WebKit/Blink (Chrome, Chromium, Safari, Opera, most one-off browsers like Web in GNOME, WebPositive on Haiku, etc), and Gecko/Servo (Firefox).

                                                                                            It’s not enough to have a million different browsers if they’re all using the same rendering engine. We need variety in implementations on that front too…

                                                                                            1. 11

                                                                                              Very good point; I spoke of this issue with a colleague a couple of weeks ago, and I asked: “what happens if Microsoft decides to get out of the web browser market and Mozilla collapses?” Then, for all intents and purposes, Google would literally control the web. Now, I don’t know how likely it is that Microsoft would stop its Edge effort or that Mozilla would go under—probably not very likely in the short term. Still, it seems to me that the complexity of the modern web (HTML 5, CSS, JavaScript with a state-of-the-art JIT compiler, DRM, etc.) makes it unlikely that we’ll see another web engine—much less a free web engine—that can compete with the current trio, and practically impossible to see enough new web engines to actually create competition.

                                                                                              1. 7

                                                                                                Agreed, the only way you web standards can realistically be called standards is if there’s a variety of implementations, and a variety of vendors with enough influence to shape those standards.

                                                                                                1. 5

                                                                                                  EdgeHTML. No really, that’s the name of the rendering engine. Microsoft continues to amaze with their horrendous naming :)

                                                                                                  1. 9

                                                                                                    Could be worse. Could be EdgeTML.

                                                                                                    1. 1

                                                                                                      Older versions of IE could be called “FML”, which is exactly how you feel when tracking those IE only bugs.

                                                                                                1. 3

                                                                                                  The Nature of Code, by Dan Shiffman. Very different from what I typically do, I’m having a lot of fun creating pretty images in Processing.

                                                                                                  1. 9

                                                                                                    First of all, I hate angle brackets, so that’s why I use square ones.

                                                                                                    Not sure that’s actually a great reason to be different from every other language in the same family.

                                                                                                    1. 3

                                                                                                      Angle brackets need shift, square brackets don’t. Some languages like Scala use square brackets. I miss them when I’m writing Rust.

                                                                                                      1. 5

                                                                                                        This is a very anglo-centric way of thinking about a design issue, no? I use a French Canadian keyboard layout, and I need to use shift for angle brackets and AltGr for square brackets.

                                                                                                        1. 1

                                                                                                          Most programming languages are written in English, which is very unfortunate, now that I think about it.

                                                                                                          1. 1

                                                                                                            I don’t really think it’s unfortunate. Compared to other fields we are lucky that it uses the (current) “world language”. We don’t have to deal with French, Ancient Greek, Latin, etc.

                                                                                                            Also having programming languages written in a random natural language means that they will have a hard time gaining popularity and probably means that the already stuffed, missing out on important things[1] study of computer science would require classes in natural languages.

                                                                                                            However, I agree that everything involving Alt Gr is horrible and a reason why programmers and system administrators sometimes get English language keyboards. Examples of characters requiring Alt Gr for many people are:

                                                                                                            @ [ ] { } | \ ~
                                                                                                            

                                                                                                            So if you care about these (based on German keyboards), you might want to avoid excessive use. There is also shift, but that’s way easier and probably very hard to entirely avoid.

                                                                                                            [1] I think computer science needs something along the lines of history class as mistakes get repeated or hacks that have been overcome get introduced into new designs. But that’s off topic, so not going to elaborate on this.

                                                                                                            1. 1

                                                                                                              I don’t really think it’s unfortunate. Compared to other fields we are lucky that it uses the (current) “world language”. We don’t have to deal with French, Ancient Greek, Latin, etc.

                                                                                                              It’s unfortunate for the French and Greek, not for us.

                                                                                                      2. 2

                                                                                                        While not identified in the OP, there is actually a good reason for it.

                                                                                                        See the first point on https://github.com/golang/proposal/blob/master/design/15292/2013-12-type-params.md#a-note-on-syntax

                                                                                                        1. 2

                                                                                                          This is Go we’re talking about. They’re intentionally needlessly different on many fronts already.

                                                                                                        1. 24

                                                                                                          Has anyone else found that making many small helper methods tends to make reading the code much more difficult though? This is something that I spend a lot of time musing about, and writing half-assed tools about (I have a bad scheme clone of imatix’s gsl).

                                                                                                          When I see a function that is just a list of helper functions, I have to zip around look up the implementation for all of them! Especially when they’re not re-used across the code base, you’re learning a whole set of semantics that are unnecessary for your problem domain.

                                                                                                          If the top level function was fully documented, then I’d only need to do that when I change something. Which is an improvement, but then it means that changing something is the slowest part, which is the opposite of maintainable code.

                                                                                                          A function name is only better than a line comment in that it delineates the start and the end of the code it’s referring to, unless it’s reused, but YAGNI if it’s not reused already. You could mark sections within comments like pre-function programming…

                                                                                                          Ah shit. I think I’m reverse engineering literate programming again.

                                                                                                          1. 10

                                                                                                            Has anyone else found that making many small helper methods tends to make reading the code much more difficult though?

                                                                                                            Depends on the length and complexity of the function.

                                                                                                            I think it’s much cleaner to have a function calling a list of well named functions each with a specific purpose than many lines of if else switch for loops etc.

                                                                                                            As far as maintenance, I suspect you know what you’re trying to modify and can go directly to which function is requiring the modification. Why would you need to zip around looking at all of them?

                                                                                                              1. 8

                                                                                                                I agree. My understanding is that this is partly based on people applying generally good rules in a way too draconian way. So while it’s great that code quality is something that people look at nowadays and build tools to improve it I think they arrived at a point where they might be either counter productive or too limited.

                                                                                                                Yeah, it’s good to have short functions that aren’t too complex, but I think everyone has seen code that became clearer when increasing it’s size.

                                                                                                                My favorite example is cyclomatic complexity, which is a great concept to quantify complexity and give a clue on when it’s worth to separate, however it sometimes can lead to the complexity becoming even more of a nuisance when that complexity is split up, to make the tool checking your code quality happy.

                                                                                                                I don’t know what the solution is and have seen different bad outcomes of bringing code quality (tools) into companies. Next to justifying insanity by throwing around some quote or some tool, I’ve seen the other side as well, which is starting to use such a tool and instead of taking its input simply configuring it so that the code just passes checks. Other than that I have seen extremely smart ways, just as silly patterns to work around code quality checkers, leading in the code being hard to read, rather than more easy.

                                                                                                                Something where these tools do very badly is that they are often not made with the programming language in mind and for obvious reasons not with the programming task at hand. So the outcome is usually that writing something like parsers or converters in a readable and efficient way makes most of these tool think code is horribly complex.

                                                                                                                I think general statements are hard to make. Sometimes I prefer to split up stuff into helpers, sometimes it is way clearer to kind of write “paragraphs” of code with some comment above them, but only when it cannot be reused as it is code that requires what happens before and after and the context/the amount of variables on the stack is both big and specific, but even here I wouldn’t consider that a general rule.

                                                                                                                A while ago I read an article about testing. If I remember correctly, it was linked here, but I can’t find it right now. It was about how not every test needs to be written or every single line of code needs to be tested and that despite testing being extremely important and something people should do. It still is nonsense to not test code, just like it being nonsense to have big programs in only a few functions.

                                                                                                                In other words: In my opinion, part of what make a programmer a good/experienced programmer is knowing what the right thing to do is.

                                                                                                                1. 3

                                                                                                                  My favorite example is cyclomatic complexity

                                                                                                                  So I decided to play with oclint and wrote essentially one function in three different styles. I also snuck a typo in one of them to see if the tool would give a shit (and it wouldn’t). oclint complains about fun2, which is arguably the most straightforward and a reasonably idiomatic way to write this kind of code in C. It does not complain about complexity in the other two functions…

                                                                                                                  It also nags about all the variable names because they are too short. How useless. Here’s the code:

                                                                                                                  int
                                                                                                                  fun1(int op, int l, int r)
                                                                                                                  {
                                                                                                                      return
                                                                                                                        op == 1 ? l + r
                                                                                                                      : op == 2 ? l - r
                                                                                                                      : op == 3 ? l * r
                                                                                                                      : op == 4 ? l / r
                                                                                                                      : op == 4 ? l % r
                                                                                                                      : op == 6 ? l << r
                                                                                                                      : op == 7 ? l >> r
                                                                                                                      : op == 8 ? l == r
                                                                                                                      : op == 9 ? l != r
                                                                                                                      : 0;
                                                                                                                  }
                                                                                                                  
                                                                                                                  int /* line 17 */
                                                                                                                  fun2(int op, int l, int r)
                                                                                                                  {
                                                                                                                      if (op == 1) return l + r;
                                                                                                                      if (op == 2) return l - r;
                                                                                                                      if (op == 3) return l * r;
                                                                                                                      if (op == 4) return l / r;
                                                                                                                      if (op == 5) return l % r;
                                                                                                                      if (op == 6) return l << r;
                                                                                                                      if (op == 7) return l >> r;
                                                                                                                      if (op == 8) return l == r;
                                                                                                                      return 0;
                                                                                                                  }
                                                                                                                  
                                                                                                                  int
                                                                                                                  fun3(int op, int l, int r)
                                                                                                                  {
                                                                                                                      switch (op) {
                                                                                                                      case 1: return l + r;
                                                                                                                      case 2: return l - r;
                                                                                                                      case 3: return l * r;
                                                                                                                      case 4: return l / r;
                                                                                                                      case 5: return l % r;
                                                                                                                      case 6: return l << r;
                                                                                                                      case 7: return l >> r;
                                                                                                                      case 8: return l == r;
                                                                                                                      default: return 0;
                                                                                                                      }
                                                                                                                  }
                                                                                                                  

                                                                                                                  Summary: TotalFiles=1 FilesWithViolations=1 P1=0 P2=1 P3=9

                                                                                                                  /tmp/ocl/oclint-0.12/bin/cat.c:17:1: high npath complexity [size|P2] NPath Compexity Number 256 exceeds limit of 200

                                                                                                                  1. 1

                                                                                                                    Now suppose our poor programmer had started with fun2 and thought that there was no way to cheat the tool and reduce complexity without breaking the function down into smaller pieces… tada, more code, a layer of indirection, and an extra pair of checks to maintain behavior with the old numbering scheme! And now the tool will happily accept it (well it still nags about names that are shorter than three letters).

                                                                                                                    Only now it’s much harder for somebody reading the code to determine which number matches which operation. Of course one could introduce yet another layer of indirection by replacing the numbers with identifiers, and then you also need to make sure the table & defines stay in sync. If that’s too much work, we can introduce yet more complexity by having another tool generate the table at build time.. anything goes as long as the tool is happy!

                                                                                                                    int op_add(int l, int r) { return l + r; }
                                                                                                                    int op_sub(int l, int r) { return l - r; }
                                                                                                                    int op_mul(int l, int r) { return l * r; }
                                                                                                                    int op_div(int l, int r) { return l / r; }
                                                                                                                    int op_mod(int l, int r) { return l % r; }
                                                                                                                    int op_shl(int l, int r) { return l << r; }
                                                                                                                    int op_shr(int l, int r) { return l >> r; }
                                                                                                                    int op_eq(int l, int r) { return l == r; }
                                                                                                                    
                                                                                                                    int (* const optab[])(int, int) = {
                                                                                                                        NULL,
                                                                                                                        op_add,
                                                                                                                        op_sub,
                                                                                                                        op_mul,
                                                                                                                        op_div,
                                                                                                                        op_mod,
                                                                                                                        op_shl,
                                                                                                                        op_shr,
                                                                                                                        op_eq,
                                                                                                                    };
                                                                                                                    
                                                                                                                    int
                                                                                                                    fun4(int op, int l, int r)
                                                                                                                    {
                                                                                                                        if (op < 1 || op >= sizeof optab / sizeof *optab )
                                                                                                                                return 0;
                                                                                                                        return optab[op](l, r);
                                                                                                                    }
                                                                                                                    
                                                                                                                    1. 1

                                                                                                                      I think the tool’s right. With fun2 the flow is much less obvious; in fun1 and fun3 it’s immediately clear that only one path will be taken.

                                                                                                                      1. 2

                                                                                                                        What in fun2 would even suggest that more than one path may be taken?

                                                                                                                        And then why does that not apply to fun3? Both are reliant on returns, nicely lined up.

                                                                                                                        fun1 is the least idiomatic of them all, and while the way I lay it out should give a strong clue about its behavior, it might still take a moment longer for the average C programmer to determine that it does indeed do what it should (the sneaky typo aside).

                                                                                                                        1. 2

                                                                                                                          What in fun2 would even suggest that more than one path may be taken?

                                                                                                                          There are a bunch of different if statements. A priori, any combination of them is possible.

                                                                                                                          And then why does that not apply to fun3? Both are reliant on returns, nicely lined up.

                                                                                                                          It does to a certain extent, but less so, because a switch is an immediate indication that only be one path will be taken - fall-through is so rarely desirable that the reader is inclined to assume it won’t happen.

                                                                                                                          fun1 is the least idiomatic of them all

                                                                                                                          Interesting, because I found it the clearest / most readable.

                                                                                                                  2. 4

                                                                                                                    I like style C with blocks

                                                                                                                    var x int
                                                                                                                    { // minor function 1
                                                                                                                        ...code..
                                                                                                                        x = whatever
                                                                                                                    }
                                                                                                                    

                                                                                                                    my coworker hates it though

                                                                                                                    1. 4

                                                                                                                      I like to use the same pattern. It’s nice because you don’t have to go around looking for tiny helper functions that only get called once while still properly scoping the lifetime of your variables so readers can forget about them when leaving a scope.

                                                                                                                  3. 8

                                                                                                                    I find the “many small helpers” style unbearable when any of the helpers mutate shared state.

                                                                                                                    Having many short pure functions can still be confusing, but it’s vastly more manageable for me.

                                                                                                                    1. 7

                                                                                                                      Has anyone else found that making many small helper methods tends to make reading the code much more difficult though?

                                                                                                                      Yes, I’ve been saying this for a while.

                                                                                                                      Show me your half-assed tools and I’ll show you mine.

                                                                                                                      1. 3

                                                                                                                        NOW THAT’S A WHOLE ASS, not half. That’s awesome!

                                                                                                                        Notes on yours: Reminds me a lot of the tanglec program that the axiom guy daly came up with. I like your notation a lot, but I’m not sure I like the before/after notation… I’ll take a deeper look at this and try to think more clearly about it. The testing part is fascinating, totally different ideas than I would ever think of.

                                                                                                                        The iMatix guys take it a different way and say that your literate programs have “models” within them that you can pull out and give some meaning to, by encoding them in well documented XML, which is then expanded into code.

                                                                                                                        Mine doesn’t support looping correctly yet in the template language, because I don’t want to re-invent monads. but here it is. It doesn’t cover what you’re doing quite, as I’m trying to go with a higher order concept of “models” that the gsl folks came up with.

                                                                                                                        I have some work locally that’s about joining these two ideas. Scheme as the meta-language, SXML/XML “models” to contain program models, and tangle/weave (with TeX or HTML, not sure yet) for source style. It’s extremely half-baked as there are a lot of missing parts.

                                                                                                                        1. 4

                                                                                                                          Thanks! Yes, I’m aware of Tim Daly’s work and we’ve exchanged comments in various places on the internet. I tend to think his emphasis on typography – shared with classic Literate Programming – is a tarpit. But it’s working for him, and I don’t really have complete confidence in my approach yet ^_^

                                                                                                                          I hadn’t come across the iMatix/gsl approach, do you have a pointer to learn more about it? That’ll help me gain context on your tool. Thanks.

                                                                                                                          1. 2

                                                                                                                            I tend to think his emphasis on typography – shared with classic Literate Programming – is a tarpit.

                                                                                                                            Having tried it, I am inclined to agree – at least for myself, who is susceptible to typography twiddling. On the other hand, are you familiar with Docco, the “quick-and-dirty, hundred-line-long, literate-programming-style documentation generator”? I have had good experiences with it.

                                                                                                                            Docco (and its clones) take as input ordinary source files containing markdown-formatted comments, and render them to two-column HTML files with comments to the left, and the code blocks that follow them to the right. That encourages writing the comments as a narrative, but limits the temptation to twiddle with typography – it basically lets you keep a nicely read- & skimmable version of the source code at hand at all times.

                                                                                                                            All of this is far more modest than what you are working on, of course. I’ll be keeping an eye on it!

                                                                                                                            1. 2

                                                                                                                              I’m a little familiar with Docco. As you said, it’s a little better because you don’t tweak the presentation endlessly. However, I think it also throws the baby out with the bathwater by dropping the flexibility to reorder parts of a program to present them in the most comprehensible way. There’s a few “quasi-literate” systems like that out there. They’re certainly better than nothing! But I think they have trouble scaling beyond short single-file examples.

                                                                                                                            2. 2

                                                                                                                              The best place to see a good example is: https://imatix-legacy.github.io/mop/index.html They are the ones that wrote ZMQ.

                                                                                                                              The basic idea is simple. Put same data in a serialized hierarchical format, that is also very easy for humans to read (for gsl, this is XML). It’s best if it has a CDATA like feature, for unquoted/raw data.

                                                                                                                              Next, there is what is called the “template”. This is another format of data, that you apply to your xml files to generate output.

                                                                                                                              Finally, there is what is called the “script”. This lets you do things like include xml files as subtrees of others, run them through multiple templates, and execute pretty basic code.

                                                                                                                              Through these pieces, you can generate what gsl calls the model. With a given set of scripts and templates, you can hypothetically reduce the “uniqueness” of your implementation to a series of XML files. zproto and other tools use this to actually generate entire server & client implementations with XML descriptions of your format. You can see a description of how zproto kind of works here : http://hintjens.com/blog:75

                                                                                                                              So it’s the happy combination of a templating language, some xml / hierarchy manipulation tooling, and a simple scripting language. I then took it into whacko land by saying all of that just sounds like lisp macros with some custom reader settings, so why not implement each of these three pieces as scheme, and add on top my desire for literate programming as part of the “generation” work it does. It’s not very far off the ground yet, but I do think there is something in here. We’ll see.

                                                                                                                              1. 3

                                                                                                                                Most interesting. I see the connection you’re drawing with Lisp macros. It sounds like GSL is basically lisp macros that get a tree of data rather than just individual variables. So like lots of nested pattern matching on the s-expression you pass into the macro, so that you can name different parts of it to access them more conveniently. Does this seem right? Now I wish I could read more about it, and that the original series had more posts :)

                                                                                                                          2. 3

                                                                                                                            That looks like it ends up equivalent to AOP, which is a nightmare to debug when done badly (and any widely-used programming tool ends up being used badly). I read of someone doing something like this in Ruby that was allegedly highly maintainable - writing the simple version of a function, thoroughly testing it, then locking it down and using monkeypatching to handle cross-cutting concerns rather than complicating the original function - but it’s so contrary to my experience that I struggle to believe it.

                                                                                                                            What I prefer is something like the “final tagless” style - just enough inversion of control, write your function in terms of commands and you can invoke the same function in either a simple or a complex context, provided it offers those commands.

                                                                                                                            1. 3

                                                                                                                              AOP is very different from MOP in terms of actual day-to-day usage and thought pattern, but I think you’re entirely right that “how will it be used badly?” is a great pattern to try and compare approaches.

                                                                                                                              One part of AOP that makes things more difficult is there is no way to see the steps/expansions visually anywhere other than with tooling. With MOP (largely, I’m only talking about gsl), it’s something explicit in your templates, and you can run iterations of gsl to see what is generated very easily. Hypothetically you could use gsl to implement AOP-style programming, but most of the culture around MOP is to focus on the “model” that is repeated in your business logic (think state machines) and making them easier to see. AOP is much more about taking large cross cutting things like logging or error handling and rolling them up into these aspects.

                                                                                                                              As I’m thinking more about this though, I’m seeing more of your point. It’s a slippery slope to just munging crap everywhere.

                                                                                                                            2. 3

                                                                                                                              This is extremely interesting! I am a big fan of literate programming because it allows me to create a narrative for understanding my program, but LP is not without its problems: you often don’t get access to your regular development tools (syntax highlighting and auto-indenting for your language, integration with IDE-like utils such as Racer for Rust) and this makes development more difficult and painful.

                                                                                                                              Your idea allows the creation of a similar narrative—order the fragments in an order that tells the story to the reader—but also gives the programmer access to his usual coding environment, save perhaps for a quick fix to allows for the directive at the top.

                                                                                                                            3. 4

                                                                                                                              A function name is better in another way: it doesn’t become outdated as easily as a comment.

                                                                                                                              Has anyone else found that making many small helper methods tends to make reading the code much more difficult though?

                                                                                                                              I have found it makes reading the code somewhat more difficult, on the first read through, iff I need to thoroughly understand or review all the code. Once I’ve done that, it makes reading the code easier, because I don’t need to parse every line to understand what it does: it tells me what it does and I know I can trust the name to be accurate.

                                                                                                                              If I only need to fix a bug and know I can trust the author, I know I don’t need to check all the implementations.

                                                                                                                              If you constantly need to check all the implementations, then it indeed only hurts to make many small well named methods. But I assert that means you are doing something wrong.

                                                                                                                              1. 2

                                                                                                                                look up the implementation? I’m confused why wouldn’t the implementation be in the function which contains the helper functions?

                                                                                                                                1. 2

                                                                                                                                  Look at the link that mikejsavage posted above for the “helper” functions I’m thinking of.

                                                                                                                                  Putting the functions inline inside a function is only useful in languages that support that style (which c++ does wonderfully now), but I’m thinking about a slightly higher point around documentation.

                                                                                                                                  1. 1

                                                                                                                                    Oh so our definitions of helper functions are entirely different things o__o. cool. I guess author did say helper methods, not helper functions.

                                                                                                                                2. 1

                                                                                                                                  Nope. Small helpers make it easier to read, provided they’re well-named and well-typed. As @danielrheath says, if they mutate state they’re very hard to understand - but don’t do that, thread state through explicitly instead.

                                                                                                                                1. 5

                                                                                                                                  I’m not very good at understanding the wording of licenses, so can anyone tell me if I understand the patent issue correctly?

                                                                                                                                  A piece of software made by Alice and using the Apache License 2.0 makes use of an algorithm A that is patented. Bob downloads and installs Alice’s software; he is given the “authorization” to use the patented algorithm, free of charge. However, should Bob decide to sue Alice for using a patented algorithm B in her software, his authorization to use the algorithm A is voided.

                                                                                                                                  In the BSD+PATENTS case, the patent authorization is voided should Bob sue Alice for any patent whatsoever, even if it has nothing to do with Alice’s software.

                                                                                                                                  Am I getting this right or completely missed the point?

                                                                                                                                  1. 2

                                                                                                                                    IANAL, but this matches my understanding.

                                                                                                                                    1. 2

                                                                                                                                      Such questions shouldn’t even come up if a licence is well written. Facebook’s BSD+PATENTS looks like a very sloppily written licence to me. All that matters in the end is Facebook’s intent, and the text is supposed to communicate that intent clearly.

                                                                                                                                      Facebook stated that this licence is deliberately incompatible with the APLv2. In the linked Jira issue, Roy T. Fielding said:

                                                                                                                                      I have discussed that license with Facebook’s legal counsel. It is not BSD (which relies on implied patent grants) and is intentionally incompatible with the Apache License.

                                                                                                                                  1. 7

                                                                                                                                    Regarding IDEs, I think that a very exciting “modern” development are the language servers. Rather than packaging all the typical IDE features into a single program, a small server responds to IDE queries (e.g., what are the valid completions for “wri”, where is logger.warn defined, etc.). This makes it possible to keep using your favorite editor without having to sacrifice useful features. OCaml’s Merlin and Rust’s Racer and RLS are examples of this.

                                                                                                                                    1. 2

                                                                                                                                      I’m reading the Angband Let’s Plays by TooMuchAbstraction. Very fun!