1. 8

    Evil idea: Fingerprint the internal tech stack of a web app by taking advantage of the fact that all JSON parsers have different behavior. Sending carefully-crafted JSON payloads that will throw an error for one “valid, but odd” syntax but not another, like {"a": 0.}.

    1. 4

      This is why Postel’s law is a bad idea something you have to be very careful about.

      1. 2

        My version of Postel’s Law, with the benefit of hindsight: “If you accept crap, then crap is what you will get.”

    1. 4

      Direct link to code (with commentary): https://play.golang.org/p/83fLiHDTSdY (and on the Go2go Playground: https://go2goplay.golang.org/p/83fLiHDTSdY)

      I think I see what he’s doing, building up an interpreter DSL, but man, I have no idea what most of his commentary means. For example:

      I exploit the duality between universal and existential quantification and encode sum types using the existential dual of the Boehm-Berarducci isomorphism.

      1. 11

        Well it’s pretty simple, really. Types are all about the composition of things, so the question is how do we compose what we have (language primitives) to get what we want. We can guess, try things out, but if that doesn’t work out we can use math to derive the result we seek, because these are all simple settled questions in math, and due to the Curry–Howard corespondence we should be able to implement them in our language.

        Alonzo Church showed us how to encode anything into (untyped) lambda calculus, and indeed, we can encode variants using Church encoding like so:

        a+b ≡ λa b f . f a b
        fst ≡ λt . t (λa b . a)
        snd ≡ λt . t (λa b . b)
        

        a+b takes three arguments, but we only supply two when we create it. Then a+b is a pending computation, and fst and snd extract the first or second component by supplying the continuation that tells the sum type what to do!

        Unfortunately, this is not quite what we want, because untyped lambda calculus is untyped, and the whole point of the exercise is to be typed so this encoding won’t cut it. But it’s ok, this captures the procedural aspect of the idea, it tells us what we have to do with the data, but not with the types. Now we have to find a way to determine a typeful encoding.

        Well, it’s pretty easy if we use math. A type like a can always be written like ∀z . (a→z) → z. But we are interested in a+b, so we just plug-in a+b, and we do some trivial algebra and we get:

        ∀z . (a+b →z) →z = ∀z . ((a→z) * (b→z)) →z
        

        This looks suspiciously similar to Church encoding, we can see the two continuations. However, this is a type equation. It doesn’t tells us anything what to do with the values, it’s just a relation between types. But we know what to do with the values from the CPS interpretation of the Church encoding, and now we can implement this in any language that has parametric polymorphism. This is the road to the Boehm-Berarducci encoding, which expands on this idea (except that it goes a step further and encodes it as ∀z . (a→z) → (b→z) →z).

        We can implement this in Haskell.

        {-# LANGUAGE ExistentialQuantification #-}
        {-# LANGUAGE RankNTypes #-}
        
        import Text.Printf
        
        data Variants a b z = Variants (a->z) (b->z)
        type Sum a b = (forall z. Variants a b z -> z)
        
        plus :: Int -> Sum Int Int
        plus x (Variants f _) = f x
        minus :: Int -> Sum Int Int
        minus x (Variants _ f) = f x
        
        neg :: Sum Int Int -> Sum Int Int
        neg v = v (Variants (\x -> minus x) (\x -> plus x))
        
        foo = plus 42
        bar = minus 16
        baz = neg foo
        
        example :: Sum Int Int -> String
        example v = v (Variants
            (\x -> printf "got a plus %d" $ x)
            (\x -> printf "got a minus %d" $ x))
        
        main = do
            putStrLn (example foo)
            putStrLn (example bar)
            putStrLn (example baz)
        

        Prints:

        : z800:aram; a
        got a plus 42
        got a minus 16
        got a minus 42
        

        However, we can’t implement this in Go. Until very recently, Go didn’t have parametric polymorphism at all, and even so, notice that the forall in Haskell is inside the type. We need rank-2 polymorphism which Go doesn’t have. But the fact that we had to enable ExistentialQuantification in Haskell should give us a hint.

        Go interfaces are existential types. In most (all?) languages based on System F/HM existential types are encoded as rank-n universals, but Go is unusual in that Go has always had existential types even before it had parametric polymorphism.

        But universal quantification in logic is dual to existential quantification! We can always switch back and forth between the two (at least in classical logic, but because of intentional limitations of the Go type system, the excluded middle axiom should always hold, so we should be ok here).

        So we can just translate any statement using universal quantifiers into its dual that uses only existential quantifiers, basically through double negation.

        ∀x P(x) ≡ ¬∃x ¬P(x)
        

        So we just do that, and we get what’s implemented in my program!

        Notice that I make the above transformation in logic, not in Go. If I’d made it in Go I would have to assume that the Curry–Howard isomorphism holds in the Go type system, which it may or it might not depending on the semantics of its generics and interfaces. I do not make this assumption. I do not assume we can use that axiom inside Go, but of course, we can use it in the meta-theory.

        I explained all of this starting from Church encoding, going though Boehm-Berarducci, etc, but that’s not how I discovered this at all. I was trying to formalize Go generics, and equations such as the ones above popped out as intermediary results. I then immediately saw the potential of this, and trying to make sure that it works, and that I understood it, I started from the existential representation, and transformed it into the Boehm-Berarducci encoding (which I wasn’t even aware it existed when I started this exercise).

        I only considered ADTs (not GADTs) in this post, but I hope this at least gives you some intuition.

        Hope this helps.

        1. 2

          I appreciate you taking time with this answer, and I did read it all, but unfortunately I just don’t have the math and computer science background to comprehend this. Back to school for me!

          1. 2

            This explanation would be more effective for me and other laypeople if you include a brief tutorial on the math notation you use.

            It’s easier to pick up the Haskell than it is to pick up the math notation.

            1. 2

              Alonzo Church showed us how to encode anything into (untyped) lambda calculus, and indeed, we can encode variants using Church encoding like so:

              a+b ≡ λa b f . f a b
              fst ≡ λt . t (λa b . a)
              snd ≡ λt . t (λa b . b)
              

              a+b takes three arguments, but we only supply two when we create it. Then a+b is a pending computation, and fst and snd extract the first or second component by supplying the continuation that tells the sum type what to do!

              Something is off here. That’s the Church encoding for products, not variants/sums.

              1. 1

                maybe because of this equivalence?

                ∀z . (a+b →z) →z = ∀z . ((a→z) * (b→z)) →z

                1. 2

                  It’s true that destructing a sum amounts to providing a product of handlers, one for each case. But that’s not how OP explained it. OP said the provided Church encoding was for variants/sums, which it isn’t, and then didn’t seem to use it at all in the Haskell code, since the Variants constructor is already a product of exponentials and doesn’t need to be Church-encoded. I think it’s just a mistake. It doesn’t detract from the Go demonstration, which is really cool. But Church encoding explanation is wrong, and I can see why people in these comments are confused.

                2. 1

                  Thanks! That is what happens when you didn’t sleep for 24 hours. It should have been:

                  c₁ ≡ λt f₁ f₂ . f₁ t
                  c₂ ≡ λt f₁ f₂ . f₂ t
                   m ≡ λt f₁ f₂ . t f₁ f₂
                  

                  e.g for usage: swap ≡ λt . m t c₂ c₁

                  (I can’t edit the mistake in the original post.)

              2. 5

                It’s like casting a spell. ‘I invoke the elemental magics of Yogg-Zonnroth, and cast the energies of Zum-ra into the ether!’

                1. 1

                  Boehm-Berarducci encoding

                  Oleg has a good paper about this. Basically you can translate all A(lgebraic)DT operations (construction and destruction (aka pattern-matching)) into lambdas and applications of lambdas, meaning you can embed ADTs in languages that do not have them.

                  1. 1

                    The code fails to run in both playgrounds … I’m guessing they are not using the right golang version ?

                    1. 1

                      Yes, it only works in the latest version of the compiler.

                  1. 7

                    I do this all the time with rebase -i which I know the article discounts, but the workflow is just:

                    git rebase -i master
                    <edit the line with tmp commit to below the one to amend, change lead to f>
                    <save quit>
                    
                    1. 3

                      The nice thing about doing git commit --fixup=<SHA> first is that it will automatically do both the moving it to the right place and changing it to fixup in the interactive rebase.

                      1. 5

                        Only if you use rebase --autosquash (or set rebase.autoSquash true). There’s a lot that isn’t on by default that should be…

                        1. 3

                          This is true, and I too am surprised that --autosquash isn’t a normal default. It must be a default in my company’s config. Either way, I couldn’t imagine operating without it being set to true. Why bother doing --fixup otherwise?

                          1. 2

                            You imagine git designed!

                            1. 3

                              git: 'designed' is not a git command. See 'git --help'.

                              ;)

                    1. 3

                      Maybe these “attitude” icons, since there are apparently still people playing Civ IV with the BUG mod in 2021.

                      1. 12

                        There’s a real interesting historical connection here. The author of this, Wouter van Oortmerssen, also created FALSE in 1993. Not only was FALSE one of the first “true” esolangs, it inspired Chris Penner’s Befunge and Urban Müller’s… brainfuck! I don’t think esolangs would be nearly as vibrant today without Wouter’s work.

                        1. 3

                          Wouter was huge in the Amiga scene. He created the E programming language, which was definitely my favorite language to use on the Amiga.

                          EasyGUI (also written by Wouter, I believe), was the sweet spot for GUI development on the Amiga. It got you 80% of the way to MUI without needing MUI (and trust me, GUI development on the Amiga without something BOOPSI-based like MUI was terrible.)

                          Wouter then went on to develop Flat Buffers at Google. He’s had an amazing career.

                          1. 5

                            He also did TreeSheets!

                            1. 4

                              which is written in Lobster, incidentally!

                              1. 8

                                That’s what I would do if I’d write it today, but it is actually in C++ (using wxWidgets), though nowadays has a way to script it in Lobster. TreeSheets actually predates Lobster by a few years (2008 vs 2010 or so).

                            2. 1

                              He created the E programming language

                              I’m assuming this is a different E than the one of capabilities fame created by Mark Miller ?

                              EDIT: nm, the wikipedia page links to AmigaE, which is the one you were referring to.

                              1. 1

                                Yeah, sorry. This one: https://en.wikipedia.org/wiki/Amiga_E

                            3. 2

                              Can you enlighten me on what an esolang is?

                              1. 3
                                1. 1

                                  Hey it’s not a “previously on” because I never submitted that one to lobsters :P

                                  (The lecture is definitely going up, if it doesn’t kill me first. One more week to go…)

                                  1. 2

                                    Wouter’s homepage is worth a browse.

                                  2. 1

                                    inspired Chris Penner’s Befunge

                                    just to clarify (incase anyone else was confused like me), it’s Chris Pressey that invented Befunge; Chris Penner is someone else; at least I think so :)

                                    1. 1

                                      Mea culpa! No idea why I wrote “Penner”

                                    2. 1

                                      The visual programming language he did for his PhD thesis (late ’90s) is interesting as well: http://strlen.com/aardappel-language/

                                    1. 37

                                      My favorite aspect (mentioned in the article) is that files that use this (logs, whatever) automatically sort sensibly!

                                      1. 9

                                        Only until we get to year 10000. :-(

                                        1. 8

                                          I vote that when that happens, we change the first digit of the year to hexadecimal so we can put off figuring out a real fix until the year 16000.

                                          1. 6

                                            At that point we switch to ISO 08601

                                            1. 2

                                              The Long Now Foundation hasn’t tried to update the ISO standard (that I’m aware of), but they’ve been using 5 digit years for a while now.

                                              1. 2

                                                There’s already a solution for that: RFC2550

                                            1. 2

                                              As to why Babbage decided on 50-digit numbers, he was very inspired by Prony who had a large team calculating sine values up to 25 decimal places and logarithms up to 19 places, and it’s possible that he wanted to outdo him. Or, perhaps once Babbage had figured out how to do carries in O(1) instead of O(Ndigits) time, he wanted to take advantage of this to the fullest!

                                              1. 8

                                                To say nothing of Ada Lovelace - a true genius.

                                                1. 5

                                                  This post is pretty interesting because it removes a lot of the doubt and mystery about her contributions to CS - she even wrote the first bug!

                                                  1. 1

                                                    Did Ada Lovelace have any books or essays? I Would like to have a comprehensive grasp of her work.

                                                    1. 6

                                                      Her sole published work was her report on Babbage’s engine. At the time it was still difficult for women to be accepted as scientists and interpreting men’s work was one of the few paths open; Ada’s tutor Mary Somerville was (jointly) the first woman ever accepted into the Royal Astronomical Society, and she had made her name by translating Laplace’s work from French into English. (In the previous century Émilie du Châtelet had followed a similar path translating Newton from English into French.) George Boole who was Ada’s contemporary — they were born in the same year — had a daughter who was the first ever female professor of chemistry: it was still early times for women in science… Ada herself was unable to access even the Royal Society’s library, and that with her husband being a Fellow. (The RS would not accept women until 1945!)

                                                      As for what happened after publishing her report, she had a falling-out with Babbage and, while they eventually reconciled, they never properly worked together again. She lived for less than a decade after her article was published and spent many of those years sick. A quote from “The Lovelace–De Morgan mathematical correspondence: A critical re-appraisal”:

                                                      It is the contrast between the mathematics that she actually wrote and her mathematical potential that has fuelled much of the debates regarding Lovelace’s mathematical ability; for despite De Morgan’s prediction that she would “get beyond the present bounds of knowledge”, we have no evidence that she created original mathematics in either published or unpublished form. The reasons for this await further research, but were probably a combination of a lack of training, lack of good health, lack of a collaborator and ultimately a lack of time before her death in 1852 at the early age of 36.

                                                      It is possible that with more time and better health, she would have published more work. But she didn’t.

                                                  1. 2

                                                    It could have been more APL like, rather than what it is, an apple script clone. Ancient Chinese is supposed to be concise.

                                                    1. 1

                                                      I think that leveraging Hangul would be more APL and compact if each sound “letter” could be combined in one character.

                                                      1. 1

                                                        Hangul denotes sound by a 2D arrangement of letters. That would introduce a vocabulary of about 10000 code points. There are about 80000 unicode code points for CJK ideographs. We could put them together with emoticons and Hieroglyphs, and make a language with about 100000 unicode code points.

                                                        1. 1

                                                          I’ve wondered about this for shortening links (visually, not byte-wise). You can more easily type in hangul and have them turn into what you want but figuring out how to do it with Chinese besides memorization still seems to escape me. APL does seem to lend itself to memorization but I think being able to compress logic into variable width characters while still being able to piece together what something does seems kinda cool.

                                                          I mean, take for instance 말 vs something like 零. With the first you could compose functions together in an interesting way like map ㅁ, some random function or variableㅏ, reduce ㄹ and you could remember them because of ligatures and the fact that it’s an alphabet.

                                                          Whereas with 零 you can remember the name of each stroke, the stroke order, the meaning of the entire character plus it’s sound but at the same time I’m not sure how you’d remember how to type it in without help from your editor. Maybe there is a way to do this that makes sense? I would be the first to try it.

                                                          1. 2

                                                            I mean, take for instance 말 vs something like 零. With the first you could compose functions together in an interesting way like map ㅁ, some random function or variableㅏ, reduce ㄹ and you could remember them because of ligatures and the fact that it’s an alphabet.

                                                            That was the spirit of my previous comment (sorry I was on the phone). By mixing that and notion from tacit programming and Raku/Perl about topical variables etc. You can compress (visually) a lot of information in one representation with a limit.

                                                            Using Hangul can be seen as Scheme sort with a minimal set of operand to combine together where using CJK ideographs can lead to a battery included approach where an ideograph can pack a lot of information in it. I don’t think that using strokes as a minimal unit of operand is a good idea because the rules for the construction of the character would be so complex. Even going from radicals to construct a restrained vocabulary.

                                                      2. 1

                                                        I had similar ideas: https://twitter.com/porges/status/1159161836203155456 (but this is a hodgepodge, I have no knowledge of classical or modern Chinese)

                                                      1. 14

                                                        I’ve been really tempted to buy a remarkable2. But the reviews I see say it’s great for note taking but not so great for just reading PDFs. Mostly I want to read PDFs. I’m still on the fence.

                                                        1. 14

                                                          As long as your PDFs don’t require color, it is 100% worth it. Definitely one of my favorite devices at the moment.

                                                          1. 5

                                                            Same. In the month or so I’ve had one, it hasn’t caused me a single frustration (and I’m the kind of person who gets annoyed at the user interfaces of my own Apple products). It works exactly as advertised. Anyone who thinks it might be worth the price tag should watch a third party review video and check out the official and awesome list projects. It has been awhile since I’ve stayed this excited about a new device so long after buying it.

                                                          2. 12

                                                            I picked one up recently hoping that I could migrate a lot of my ebooks and pdfs to it. I don’t plan on returning it, but I wouldn’t recommend it.

                                                            I was a huge fan of the kindle dx, but I’ve managed to break the buttons on a couple which renders them practically useless. I was on the fence with the first remarkable device but figured I’d given the latest iteration a shot. I figured it’d be a good DX substitute. It’s not. I want to like it, the physical design is really good, but the software sucks.

                                                            I have a large collection of documents (epub/pdfs) that I was looking forward to getting on the device. Largely a mix of books published in electronic formats from regular publishers (O’Reilly, Manning, PragProg, etc.) as well as a few papers and docs I’ve picked up here and there.

                                                            First, the reMarkable desktop/mobile app that you have to rely on for syncing is a little wonky. Syncing between the device and mobile/desktop versions of the app works, but leaves a little to be desired. Second, I have yet to load a pdf or epub that isn’t brutally slow to navigate (just page by page). If the document has images or graphics (even simple charts and illustrations) it will affect navigation performance. Occasionally a document will load relatively quickly, and navigate reasonable well, only to slow down after a few page turns. Epubs tend to be a little more difficult to work with - particularly if you decide to change the font. All I have to compare this device to is my broken DX, which, everything considered, positively smokes the reMarkable.

                                                            It’s usable. It works alright for PDFs, less so for epubs. On the positive side, the battery life is quite good.

                                                            1. 3

                                                              I agree with your analysis in most regards. Syncing a lot of ebooks and pdfs to it is not something at which it would excel by default. I have a large Calibre library, and I haven’t synced it over for that reason. However, it’s something I’m looking forward to investigating with KOReader, which supports the reMarkable.

                                                              I haven’t experienced the lag that you talk about, but can understand that that would be bothersome – though I definitely have experienced the “wonkiness” of the companion apps.

                                                              1. 1

                                                                My understanding is that epubs are converted to PDF before being synced? Is that actually the case?

                                                                1. 4

                                                                  It renders the epub to pdf for display but that’s all in-memory. It’s still an epub on disk.

                                                                  1. 1

                                                                    I don’t know. I’ve got a couple books that are both pdf and ePub, and the pdf version behaves a little better. You can also resize and change fonts for ePub doc, but not for PDFs.

                                                                    1. 1

                                                                      Along these lines, another interesting observation I’ve made has to do with the way some kinds of text get rendered. In particular, I’ve encountered epubs with code listings that render fine in other apps and on other devices, but render horribly on the remarkable2 device. Interestingly, in some of those cases I will also have a publisher provided PDF that renders just fine.

                                                                      Further, epubs and PDFs are categorized differently in both the app and the device. With epubs you can change the justification, page margins, line spacing, fonts, and font size. With PDFs you have fewer options, but you do have the ability to adjust the view (which is great for papers since you can get rid of the margins).

                                                                    2. 2

                                                                      I don’t think so – from my playing around with ssh, there are definitely some epubs stored on device. I actually think the browser extension generates epubs, rather than pdfs which was surprising.

                                                                      1. 2

                                                                        Huh. Cool. Hmmm. The real reason I shouldn’t get one is that I always fall asleep with my e-reader and it often bounces off my face.

                                                                        1. 3

                                                                          That’s a pro, for the device, it weighs next to nothing. I’ve damn near knocked myself out dropping an iPad Pro on my head when reading in bed.

                                                                          1. 1

                                                                            For me, it’s more the fact that the Kobo then ends up falling onto the floor. I’m not crazy with that with a $120 device, so …

                                                                  2. 7

                                                                    I own Gen 1 and Gen 2. I love the simplicity and focus of the device. It’s an amazing… whiteboard.

                                                                    Note taking is not suuuper great. Turns out marking up a PDF to take notes actually isn’t that great because the notes quickly get lost in the PDF. It’s not like in real life, where you can put a sticky note to jump to that page. The writing experience is fantastic though. I have notebooks where I draw diagrams/ideas out. I like it for whiteboarding type stuff.

                                                                    Reading is terrible. I mean, it works. Searching is painfully slow. The table of contents doesn’t always show up (even though my laptop PDF reader can read the TOC just fine). When you do get a TOC, the subsections are flattened to the top level, so it’s hard to skim the TOC. PDF links don’t work. Text is often tiny, though you can zoom in. EPUBs appear to get converted to PDFs on the fly and their EPUB to PDF conversion sucks. Though, I’ve found doing the conversion myself in Calibre is way better.

                                                                    Overall, I like the device for whiteboarding. But it’s kinda hard to recommend.

                                                                    1. 2

                                                                      Marking up PDFs works better in color, since you can pick a contrasting ink color. I do it in Notability on my iPad Pro (which is also great for whiteboarding / sketching.)

                                                                      I was tempted by reMarkable when the first version came out, but I couldn’t see spending that kind of money on something that only does note taking and reading. I’m glad it’s found an audience though, it’s a cool device.

                                                                      1. 1

                                                                        Turns out marking up a PDF to take notes actually isn’t that great because the notes quickly get lost in the PDF. It’s not like in real life, where you can put a sticky note to jump to that page.

                                                                        So far the best experience I’ve seen for this is LiquidText on an iPad Pro. While you can write on the PDF as any other annotator, there’s also a lot of more hypertext type of features, like collecting groups of notes in an index, or writing separate pages of notes that are bidirectionally hyperlinked to parts of the document they refer to. Or do things like pull out a figure from a paper into a sidebar where you attach notes to it.

                                                                        The main downside for me is that you do more or less have to go all-on on LiquidText. It supports exporting a workspace to flat PDFs, but if you used the hypertext features in any significant way, the exported PDFs can be very confusing with the lack of expected context.

                                                                        1. 1

                                                                          Agreed that it is hard to find notes. There should be a way to jump to pages that have notes on them (this is how Drawboard PDF works, for example).

                                                                          1. 1

                                                                            What is the advantage over drawing on a piece of paper or on a whiteboard, then taking a photo of what you’ve drawn, if needed?

                                                                            1. 1

                                                                              I tried paper note books, but I’m too messy and make too many mistakes. Erasing, moving, and reordering is hard on paper.

                                                                              A whiteboard is pretty good for temporary stuff and erases better than paper. But, it can be a bit messy.

                                                                              I also tried Rocketbook for a while. I got the non-microwaveable (yes you read that right) one. That was okay. A little meh for me.

                                                                              And of course, you can’t read PDFs on any of these.

                                                                        1. 1

                                                                          Out of what I read in 2020, I would most recommend:

                                                                          • Soft X-Ray/Mindhunters, an entirely wordless graphic novel but one of the most mindbending things I’ve read
                                                                          • 20020 by Jon Bois, the only person who can make me read about sports (read 17776 first if you haven’t!)
                                                                          • Until You Continue To Behave, a free dystopian ‘maximalist’ novel about working in tech, worth reading online for the music. Read it if you like this video!
                                                                          • Lisa See’s books The Island of Sea Women about freedivers on Jeju Island (Korea), and The Tea Girl of Hummingbird Lane about pu-erh tea. She pours an incredible amount of research into these.
                                                                          • Sisters of the Vast Black by Lina Rather, a nuns-in-space novella that manages to fit in a better-developed world than many novels.
                                                                          • Maria Dahvana Headley’s Beowulf, I want to see a live performance of this!
                                                                          1. 23

                                                                            Part I starts with a faulty premise. This means that our explanations might not fulfill the explainable-AI requirements. Why? Because the discovery of the Higgs boson was made by theories about why particles have mass having various implications, and those implications being followed through with formal logic. In this arena of reasoning, we are not doing statistical analysis on lots of pseudo-continuous data, but instead our inputs are proof statements.

                                                                            If I had to explain to a lay reporter why we expected the Higgs boson, I would start my attempt with something like, “Electricity and magnetism are two ways to look at the same single happenstance in physics, where negatively- and positively-charged objects interact with each other. When we look at particle physics, we see a way to combine the behaviors we see, with atoms splitting and joining, with electromagnetism. This helps us build our models. But imagine that we switched ‘positive’ and ‘negative’. Look up the history with Ben Franklin, it’s a great story. The point is that those labels could have been switched, and that gives us a sort of symmetry for particles. The Higgs boson is a particle that should exist according to what we know about particle symmetries, and we call it ‘Higgs’ after one of the people who first noticed that it should exist.”

                                                                            Crucially, this explanation uses a real example as an analogy to bootstrap intuition about the unknown. Rather than handwaving, poisoning the well, or appealing to authority; the explanation lays out a context, including specific symbols (keywords) which can link to further details. The explanation does not rely on a vague statistical argument made using many weak indicators, but uses one main concept, symmetry, as its focus.

                                                                            Now, having said all this, I strongly suspect that the author might reasonably reply that the question they wanted to ask was more like, “what pattern in experimental data prompted the theoretical work which led to the proposal of the Higgs mechanism?” This does sound like something that could be answered with a data-driven correlation. And, indeed, that is what happened; the models of that time were faulty and predicted that certain massive particles should be massless. But the actual statistical techniques that were used were the standard ones; the explanation could begin and end with a t-test.

                                                                            All of this context is necessary to understand what will happen to poor Steve. Historically, Steve’s last name might be the most important input to the algorithm, or possibly their ethnic background if the algorithm can get hold of it more directly. And Steve’s inability to participate in society is explained away by the reassurance that there are millions of parameters which might have influenced the decision. This is exactly the sort of situation that explainable-AI proponents are trying to avoid.

                                                                            But in both cases, the reasoning is not simple, there’s no single data point that is crucial, if even a few inputs were to change slightly the outcome might be completely different, but the input space is so fast it’s impossible to reason about all significant changes to it.

                                                                            I don’t agree with this. Specifically, I don’t think that those millions of parameters are actually used much. Instead, I think that NNAEPR and there are only a handful of parameters which account for almost all of the variance in loan amounts, and that the error of the remaining parameters is subject to roundoff. Similarly, only one measurement, mass, needed to be wrong to provoke the development of the Higgs mechanism in theory.

                                                                            The explanation in part III is not a valid proof, because correlation is not transitive. I do appreciate the exploration here into epistemology and the nature of justification. But I can’t ignore the fact that the maths are incorrect; if an AI can generate English which successfully bullshits people, then is it really explaining or just lying? In a hypothetical world where AIs have civil rights, we would expect AI explanations to be just as cross-examinable as human explanations, and thus to stand up under scrutiny. What difference is there between an opaque AI loan officer and an opaque human loan officer?

                                                                            As we have explored here before, we must be extremely skeptical of the argument that it is simply too hard to explain ourselves to others, in the context of the immense social harm which results from being judged by opaque corporations. Specifically, when they claim that they cannot be legible to outsiders, they are trying to find ways to be less responsible for their own actions; be assured that the corporation is legible to itself.

                                                                            1. 10

                                                                              we must be extremely skeptical of the argument that it is simply too hard to explain ourselves to others, in the context of the immense social harm which results from being judged by opaque corporations

                                                                              Just want to say that I think this is a really thoughtful and true thing, beyond the rest of your commentary. Ultimately the worth of these tools, surely, must be measured in how beneficial they are to society.

                                                                              If a neural net loan officer saves society a few tens of of thousands human-labor-hours a year, subsequently making loans slightly cheaper and more profitable, that’s good. But if they do that while also making it impossible to answer the question “why was this loan denied”, then well, the net effect is that you made the world worse and more open to exploitation and your approach should be curtailed.

                                                                              1. 5

                                                                                Back in 1972 John Kemeny (co-developer of BASIC) was warning about uninterrogable decision-making (in Man and the Computer):

                                                                                I have heard a story about the design of a new freeway in the City of Los Angeles. At an open hearing a number of voters complained bitterly that the freeway would go right through the midst of a part of the city heavily populated by blacks and would destroy the spirit of community that they had slowly and painfully built up. The voters’ arguments were defeated by the simple statement that, according to an excellent computer, the proposed route was the best possible one. Apparently none of them knew enough to ask how the computer had been instructed to evaluate the variety of possible routes. Was it asked only to consider costs of building and acquisition of property (in which case it would have found routing through a ghetto area highly advantageous), or was it asked to take into account the amount of human suffering that a given route would cause? Perhaps the voters would even have agreed that it is not possible to measure human suffering in terms of dollars. But if we omit considering of human suffering, then we are equating its cost to zero, which is certainly the worst of all procedures!

                                                                                (This message brought to you by the Campaign for the Rehabilitation of BASIC.)

                                                                                1. 3

                                                                                  You raise an important point about model interpretability. All models that predict the future by training on historical data propagate historical bias. This is an effect, not a side-effect.

                                                                                  A simple example can be found in natural language processing, where words become numbers to be usable as model features. With a model trained on a corpus of human-written documents, you’ll be able to “subtract” the word “woman” from the word “king” to get the result of “queen” and think yourself quite clever. Then, you’ll subtract the word “woman” from the word “doctor” and find yourself uncomfortable to discover the result is “nurse”.

                                                                                  An additional example drawing from the above comment: if it is illegal and unethical to deny a loan on the basis of race, but you build an opaque model to predict loan outcome that (under the hood) incorporates e.g. census block as a feature, you will still have built a redlining AI that reinforces historical racial segregation.

                                                                                2. 1

                                                                                  I don’t agree with this. Specifically, I don’t think that those millions of parameters are actually used much. Instead, I think that NNAEPR and there are only a handful of parameters which account for almost all of the variance in loan amounts, and that the error of the remaining parameters is subject to roundoff. Similarly, only one measurement, mass, needed to be wrong to provoke the development of the Higgs mechanism in theory.

                                                                                  Okay, How do you explain why you believe the parameters necessary are smaller? If you want to counter his argument based on the maths being wrong you have to explain why you think the maths are wrong. And that in some sense is playing straight into his argument.

                                                                                  1. 4

                                                                                    I strongly suggest that you spend some time with the linked paper. From a feature-based POV, polynomial regression directly highlights the relatively few robust features which exist in a dataset. Neural nets don’t do anything desirable on top of it; indeed, they are predicted and shown to have a sort of collinearity which indicates redundancy in their reasoning and can highlight spurious features rather than the robust features which we presumably desire.

                                                                                    Even leaving that alone, we can use the idea of entropy and surprise to double-check the argument. It would be extremely surprising if variance in Steve’s last name caused variance in Steve’s loan qualifications, given the expectation that loan officers do not discriminate based on name. Similarly, it would be extremely surprising if variance in Steve’s salary did not cause variance in Steve’s loan qualifications, given the expectation that salaries are correlated with ability to pay back loans. This gives us the ability to compare an AI loan officer with a human loan officer.

                                                                                  2. 1

                                                                                    This means that our explanations might not fulfill the explainable-AI requirements. Why? Because the discovery of the Higgs boson was made by theories about why particles have mass having various implications, and those implications being followed through with formal logic. In this arena of reasoning, we are not doing statistical analysis on lots of pseudo-continuous data, but instead our inputs are proof statements.

                                                                                    If I had to explain to a lay reporter why we expected the Higgs boson, I would start my attempt with something like, “Electricity and magnetism are two ways to look at the same single happenstance in physics, where negatively- and positively-charged objects interact with each other. When we look at particle physics, we see a way to combine the behaviors we see, with atoms splitting and joining, with electromagnetism. This helps us build our models. But imagine that we switched ‘positive’ and ‘negative’. Look up the history with Ben Franklin, it’s a great story. The point is that those labels could have been switched, and that gives us a sort of symmetry for particles. The Higgs boson is a particle that should exist according to what we know about particle symmetries, and we call it ‘Higgs’ after one of the people who first noticed that it should exist.”

                                                                                    I think we might have different intuitions for what “explanation” means here.

                                                                                    The above is the kind of news-conference explanation that is not at all satisfying. It’s a just-so story, not something you’d want from, e.g., an emergency system controlling the launch of nuclear missiles… or even from an organ transplant model that decides who the most likely to benefit patients are.

                                                                                    Maybe, if you actually know physics yourself, try to answer a question like:

                                                                                    “Why is the mass of the higgs boson not 125.18 ± 0.15 GeV/c^2 instead of 125.18 ± 0.16 (as per Wikipedia) ?”

                                                                                    or

                                                                                    “What would it take for the mass to be 125.18 ± 0.15 ? How would the theory or experiments have to differ ?”

                                                                                    Those are the kind of explanations that, I presume, a physicist working on the Higgs boson could give (not 100% sure how accessible they are, maybe anyone with a physics PhD could given a bit of digging). But the likelihood of me understand them is small, you can’t make a “just-so story” to explain those kinds of details.

                                                                                    Yet ultimately it’s the details that matter, the metaphysics are important but explaining those does not give a full picture…. they are more like intuition pumps to begin learning from.

                                                                                    I don’t agree with this. Specifically, I don’t think that those millions of parameters are actually used much. Instead, I think that NNAEPR and there are only a handful of parameters that account for almost all of the variance in loan amounts, and that the error of the remaining parameters is subject to roundoff. Similarly, only one measurement, mass, needed to be wrong to provoke the development of the Higgs mechanism in theory.

                                                                                    This I actually agree with, and my last 4 months of leisure time research have basically been spent on reading up on parameter pruning and models that use equations that are quick to forward prop but take long to fit (quick to forward prop is often ~= easy to understand, but I prefer to focus on it since it has industry application).

                                                                                    That being said it’s not all clear to me that this is the case, it could be the case in some scenarios but (see III.2 and IV in the article) it’s probably an NP-hard problem to distinguish those scenarios. And maybe my intuition that most models are over-parameterized and using backprop-optimized operations is just wishful thinking.

                                                                                    Again, not to say our intuitions differ here, but I tried to go against my intuitions when writing this article, as state in it.

                                                                                  1. 3

                                                                                    Some corrections:

                                                                                    • AbDTs are from CLU, not simula 67.
                                                                                    • While simula 67 introduced objects, certain qualities of OOP first appeared with Smalltalk-72.
                                                                                    • PL/I had overloading before Algol 68.

                                                                                    And a few I’m less sure of:

                                                                                    • References first appeared in CPL, not Algol 68.
                                                                                    • PL/I had user defined datatypes before Algol 68.
                                                                                    1. 2

                                                                                      COMTRAN (1960, see e.g. page 103) pretty much had user-defined datatypes which were adopted into COBOL and then PL/I picked up things from there. There’s probably still a predecessor to that, though!

                                                                                      FLOW-MATIC maybe sorta did but the DDL is pretty obtuse (but the FLOW-MATIC manual is really nice).

                                                                                      1. 1

                                                                                        Deleted (replied to the incorrect comment)

                                                                                        1. 1

                                                                                          While simula 67 introduced objects, certain qualities of OOP first appeared with Smalltalk-72

                                                                                          Which qualities?

                                                                                          1. 1

                                                                                            Treating primitives as objects is the big one. In Simula, the number 26 is a primitive, not an object.

                                                                                            1. 3

                                                                                              I contend that “everything is an object” is not a “quality” of OOP. Are Java, C# or C++ limited in terms of object-oriented programming by not doing so?

                                                                                              1. 2

                                                                                                They certainly are, since the distinction between an “object” and a “primitive” is an artificial one, made for the compiler implementer’s sake rather than the user’s.

                                                                                                1. 2

                                                                                                  You’re right and I’m wrong.

                                                                                                  1. 0

                                                                                                    Yes.

                                                                                            1. 5

                                                                                              I’m quite familiar with the C++ version of this mistake, which looks like

                                                                                              const char *cstr = something_returning_string().c_str();

                                                                                              It’s an easy mistake to make. Fortunately, I always turn on the Clang Address Sanitizer when I build in debug mode, and it immediately catches this at runtime. Does Rust have some equivalent of this? (I guess there’s Valgrind, but IIRC that’s more complicated and slower.)

                                                                                              At the heart of the problem … is a function to gift files from the server to the recipient.

                                                                                              I can’t help pointing out the irony that “gift” is German for “poison”…

                                                                                              1. 3

                                                                                                Address Sanitizer works with Rust — it’s an LLVM feature, not a C feature. Valgrind works too.

                                                                                                Rust also has Miri, which is a Rust interpreter that catches Undefined Behavior quite precisely (more so than tools based on memory pages or only information in the compiled program).

                                                                                                fn main() {
                                                                                                    let c = std::ffi::CString::new("oops").unwrap().as_ptr();
                                                                                                    unsafe {
                                                                                                        *c;
                                                                                                    }
                                                                                                }
                                                                                                

                                                                                                error: Undefined Behavior: pointer to alloc2052 was dereferenced after this allocation got freed

                                                                                                However, in OP’s case the pointer was dereferened on the C side, so the Rust interpreter wouldn’t see it.

                                                                                                1. 1

                                                                                                  Clang will also catch this at compile time with -Wlifetime: https://godbolt.org/z/M3oEWh

                                                                                                  It would also be nice if c_str was marked with the “&” ref-qualification so that you couldn’t invoke it on temporaries, but with lifetime analysis this is less of an issue.

                                                                                                  Really this indicates the need for Rust to have lifetime parameters on raw pointers…

                                                                                                  1. 3

                                                                                                    It’s actually nice to call .c_str on temporaries if you are immediately passing the resulting pointer to a function in the same full-expression (and the function uses the pointer only within that call) (since the temporary string would survive till the end of the full l-expression anyways).

                                                                                                    eg. someFoo((someString() + "bar").c_str());

                                                                                                1. 14

                                                                                                  Ted made a classic mistake in unsafe code (one I’ve made myself). This is why the Rust community normatively discourages writing unsafe code if it can be avoided. If I were a code reviewer, I’d be paying specific attention to any module containing unsafe code.

                                                                                                  It’s worth saying that the as_ptr documentation explicitly notes this issue, and another option instead of the one shown would be to use CString::into_raw which you can safely cast from *mut c_char to *const c_char.

                                                                                                  1. 7

                                                                                                    I’m not a Rust developer, but it seems to me Ted’s point here is that it seems this scenario would be harder to catch by the code reviewer paying special attention, while any reviewer of Go code would see the opening/accessing of a resource (even if they were themselves on shaky ground with unsafe code) and think “that next line better be a defer statement to clean up”.

                                                                                                    1. 13

                                                                                                      First, I want to say I appreciate Ted for voicing his experience with Rust in this area, and I do think he hit this speed bump honestly, did his best to resolve it, and is understandably displeased with the experience. Nothing I say here is intended as a judgment of Ted’s efforts or skill. This is a hard thing to do right, and he did his best.

                                                                                                      To your comment: I agree that is part of his point. I don’t agree in his assessment that the Go code is “clearer” [1]. In Rust, any unsafety is likely to receive substantial scrutiny, and passing clippy is table-stakes (in this case, Clippy flags the bad code, which Ted notes). I view the unsafe presence in the module as a red-flag that the entire module must be assessed, as the module is the safety boundary due to Rust’s privacy rules (see, “Working with Unsafe” from the Rustonomicon).

                                                                                                      That said, Rust can improve in this area, and work is ongoing to better specify unsafe code, and develop norms, documentation, and tooling for how to do unsafe code correctly. In this case, I’d love to see the Clippy lint and other safety lints like it incorporated into the compiler in the future.

                                                                                                      [1] Assessing “clarity” between two languages is really tough because clarity is evaluated in the context of socialization around a language’s norms, so a person can only effectively make the comparison if they are at least sufficiently knowledgeable in both languages to understand the norms around code smells, which I don’t think Ted is.

                                                                                                      1. 5

                                                                                                        Is clippy used that pervasively? I’d rather have that particular warning in the compiler itself, in this case. Clippy seems to be super verbose and I don’t care too much for a tool that has strong opinions on code style.

                                                                                                        1. 8

                                                                                                          Feel free to file an issue to uplift the warning into rustc: Clippy warnings, especially deny-by-default clippy warnings – are totally fair game for uplifting into rustc if they’re seen as being useful enough.

                                                                                                          Clippy has a lower bar for lints, but for high value correctness lints like this one the only reason a lint is in clippy is typically “rustc is slower to accept new lints”, and if a lint has great success from baking in clippy, it can be upstreamed.

                                                                                                          People use clippy quite pervasively. Note that in this case the clippy lint that is triggered is deny-by-default, and all deny-by-default clippy lints are “this is almost definitely a bug” as opposed to “your code looks ugly” or whatever. You can easily turn off the different classes of clippy lints wholesale, and many people who want correctness or perf lints but not complexity or style lints do just that.

                                                                                                          1. 2

                                                                                                            In my experience, yes Clippy is used pervasively (but maybe I’m wrong).

                                                                                                            1. 1

                                                                                                              We have a few Rust projects at $work and we use cargo clippy -- -D warnings as part of our builds. It’s a very popular tool.

                                                                                                        2. 3

                                                                                                          This is not about unsafe code as this code works in safe rust but it is still not correct. Proof

                                                                                                          I think using as_ptr (and similar functions) in this way should 100% be an error.

                                                                                                          1. 1

                                                                                                            However, there’s no bug in the example you’ve written as the pointer is never dereferenced.

                                                                                                          2. 1

                                                                                                            Unable to edit the post anymore, but others are right to point out that I missed part of the point here, and it’s important to note that CString::into_raw will leak the memory unless you reclaim it with CString::from_raw, and that CString::as_ptr is a challenging API because it relies on the lifetime of the original CString in a way that isn’t unsafe per Rust’s formal definition, but is still sharp-edged and may easily and silently go wrong. It’s caught by a Clippy lint, but there are some for whom Clippy is too aggressive.

                                                                                                            Thanks @hjr3 and @porges.

                                                                                                            1. 1

                                                                                                              Be careful (to other readers) that into_raw used in the same way here would leak the memory. Really as_ptr is very suspicious in any code, in exactly the same way that c_str is in C++.

                                                                                                            1. 9

                                                                                                              Great news! I am eager to try this!

                                                                                                              Turn on -XLinearTypes, and the first thing you will notice, probably, is that the error messages are typically unhelpful: you will get typing errors saying that you promised to use a variable linearly, but didn’t. How hasn’t it been used linearly? Well, it’s for you to puzzle out. And while what went wrong is sometimes egregiously obvious, it can often be tricky to figure the mistake out.

                                                                                                              So, basically, GHC just got its own “Syntax error” a la OCaml… just a bit more specialized :p.

                                                                                                              1. 11

                                                                                                                Maybe it’s just me, but to me OCaml’s errors are terse and unhelpful and GHC’s errors are… verbose and unhelpful. ;)

                                                                                                                There are interesting papers that show working ways to improve both, but I wonder why none of those improvements are in the mainline compilers.

                                                                                                                1. 2

                                                                                                                  Good error reporting is easiest if it’s built into the compiler front end from the start. If a new algorithm comes along to improve the error information it’s almost never going to be a simple job to drop it into an existing compiler.

                                                                                                                  You need type information & parse information from code that’s potentially incorrect in both spaces, so any error algorithm usually has to be tightly integrated into both parts of the compiler front end. That tight integration usually means that improving compiler errors is a significant amount of work.

                                                                                                                  1. 3

                                                                                                                    It varies. What puzzles me is that a lot of time ready to use, mergeable patches take much longer to merge than they should.

                                                                                                                    Like this talk: https://ocaml.org/meetings/ocaml/2014/chargueraud-slides.pdf

                                                                                                                    1. 1

                                                                                                                      Do you also have a link for a patch for the improved error messages?

                                                                                                                      A lot of work has been going on to move OCaml to a new parser and improve error messages. Even though there is a lot still needed to be done, latest releases have started improving a lot. Maybe we can still extract some useful bits from that effort and try again

                                                                                                                      1. 2

                                                                                                                        Turns out it was even made into a pull request that isn’t merged yet: https://github.com/ocaml/ocaml/pull/102

                                                                                                                        1. 1

                                                                                                                          Thanks. It is quite an informative PR actually, and explains why the change is not there yet and once can infer why it is easier to add informative messages in new languages and complier but it may be quite hard to retrofit them to seasoned ones

                                                                                                                2. 7

                                                                                                                  Would you be kind enough to give me an ELI5 about what linear types are and what you can do with them?

                                                                                                                  1. 29

                                                                                                                    In logic, normal implication like A implies B means whenever you have A, you can derive B. You have tautologies like “A implies (A and A)” meaning you can always infinitely duplicate your premises.

                                                                                                                    Linear implication is a different notion where deriving B from A “consumes” A. So “A linearly implies B” is a rule that exchanges A for B. It’s not a tautology that “A linearly implies (A and A).”

                                                                                                                    The classic example is you can’t say that “$3 implies a cup of coffee” but “$3 linearly implies a cup of coffee” makes sense. So it’s a logical form that reasons about resources that can be consumed and exchanged.

                                                                                                                    Same in functional programming. A linear function from type A to type B is one that consumes an A value and produces a B value. If you use it once with an A value then you can’t use it again with the same A value.

                                                                                                                    This is nice for some performance guarantees, but also for encoding safety properties like “a stream can only be read once” etc.

                                                                                                                    1. 6

                                                                                                                      Keynote: Linear Haskell: Practical Linearity in a Higher-Order Polymorphic Language https://skillsmatter.com/skillscasts/11067-keynote-linear-haskell-practical-linearity-in-a-higher-order-polymorphic-language

                                                                                                                      1. 5

                                                                                                                        It can be used to model protocols with type signatures. The following is in theory what you should be able to do.

                                                                                                                        data ConsoleInput
                                                                                                                            = Input String ConsoleOutput
                                                                                                                            | ExitInput
                                                                                                                        
                                                                                                                        data ConsoleOutput
                                                                                                                            = PrintLines ([String] ⊸ Console)
                                                                                                                            & PrintLastLines ([String] ⊸ ())
                                                                                                                        
                                                                                                                        greet :: ConsoleOutput ⊸ ()
                                                                                                                        greet console
                                                                                                                            = let PrintLines f = console
                                                                                                                              in step2 (f ["name?"])
                                                                                                                        
                                                                                                                        step2 :: ConsoleInput ⊸ ()
                                                                                                                        step2 ExitInput = ()
                                                                                                                        step2 (Input input console)
                                                                                                                            = let PrintLastLines f = console
                                                                                                                              in f ["hello " ++ input]
                                                                                                                        

                                                                                                                        If you combine it with continuation passing style, you get classical linear logic and it’s a bit more convenient to use.

                                                                                                                        If you model user interfaces with types, they should be quite useful.

                                                                                                                        I’m also examining and studying them: http://boxbase.org/entries/2020/jun/15/linear-continuations/

                                                                                                                        1. 1

                                                                                                                          Wikipedia gives a reasonable overview. The closest analogy would be something like move semantics – for example ownership in Rust can be considered as manifestation of linear types.

                                                                                                                          1. 6

                                                                                                                            Rust ownership is linear affine types. Linear types are similar but differ in the details. A shitty way of understanding it is affine types mimic ref counting and prevent you from having a ref count < 0. Linear types are more a way of acting like RAII in that you might create a resource but just “know” that someone later on in the chain does the cleanup.

                                                                                                                            Which I’m sure sounds similar but affine types allow for things like resource leaks but linear types should guarantee overall behavior to prevent it.

                                                                                                                            This all assumes my understanding and explanation is apt. I’m avoiding a ton of math and i’m sure the shitty analogy doesn’t hold up but behaviorally this is how I have it in my brain.

                                                                                                                            1. 2

                                                                                                                              Linearity Design Space: https://i.imgur.com/s0Mxhcr.png

                                                                                                                              1. 2

                                                                                                                                I’m personally of the stance that the 2020 linear ghc stuff is more <= 1 usage, and kinda misses out on a lot of really fun expressivity that can fall out of making full classical linear logic first class. But that’s a long discussion in its own right , and I’ve yet to make the time to figure out the right educational exposition on that front

                                                                                                                                1. 1

                                                                                                                                  it definitely seems more limited in scope/ambition compared to the effort ongoing for dependent types, for better or worse. Can’t say I know much about what first class linear logic would look like, but perhaps now there will be more discussion about such things.

                                                                                                                                  1. 2

                                                                                                                                    The really amazing thing about full linear logic is it’s really sortah a rich way to just do mathematical modelling where everything has a really nice duality. The whole thing about linearity isn’t the crown jewel (though wonderfully useful for many applications ), it’s that you get a fully symmetric bag of dualities for every type / thing you can model.

                                                                                                                                    The paper that really made it click for me was mike shulmans linear logic for constructive mathematics paper. It’s just a fun meaty read even at a conceptual level. There’s a lot of other work by him and other folks that taken together just point to it being a nice setting for formal modelling and perhaps foundations of category theory style tools too!

                                                                                                                                2. 1

                                                                                                                                  Not sure I can agree that Uniqueness types are the same as Linear types. Care to explain they’re similar sure but not the same thing and your… screenshot of a powerpoint? isn’t very illustrative of whatever point you’re trying to make here.

                                                                                                                                  And from my experience with Idris, I’m not sure I’d call what Rust has Uniqueness types.

                                                                                                                                  1. 1

                                                                                                                                    They are different rows in the matrix because they are different, of course.

                                                                                                                                    it’s from this presentation about progress on linear ghc a little over a year ago https://lobste.rs/s/lc20e3/linear_types_are_merged_ghc#c_2xp2dx skip to 56:00

                                                                                                                                    What is meant by Uniqueness types here is “i can guarantee that this function gets the unique ptr to a piece of memory” https://i.imgur.com/oJpN4eN.png

                                                                                                                          2. 2

                                                                                                                            Am I the only one thinking this is not how you ship language features?

                                                                                                                            If the compiler can’t even report errors correctly, the feature shouldn’t ship.

                                                                                                                            1. 15

                                                                                                                              If the compiler can’t even report errors correctly, the feature shouldn’t ship.

                                                                                                                              Its more this is an opt-in feature with crappy error reporting for now using computer programming design features not in use in most programming languages. Its going to have rough edges. If we required everything to be perfect we’d never have anything improved. Linear types like this also might not have a great way to demonstrate errors, or the domain is new so why not ship the feature for use and figure out what kind of error reporting you want based on feedback.

                                                                                                                              1. 13

                                                                                                                                Many people do not realize that haskell is a research language and GHC is one of the main compilers for it. This is an experimental feature in a research language. If it works out well, then it will be standardized.

                                                                                                                              2. 5

                                                                                                                                Other people have sort-of said it, but not clearly enough I think. This is not a language feature being added. It is a feature-flagged experimental feature of a particular compiler. Most such compiler extensions never make it into real Haskell, and the ones that do take years after they are added to a compiler to make it to a language spec.

                                                                                                                                1. 4

                                                                                                                                  for all practical purposes isn’t “real Haskell” defined by what ghc implements these days?

                                                                                                                                  1. 2

                                                                                                                                    Yes, all the other implementations are dead. They still work, but they won’t run most modern Haskell code, which usually uses a bunch of GHC extensions.

                                                                                                                                    1. 1

                                                                                                                                      You might say “isn’t it not popular to write standards-compliant Haskell these days?” and you’d be right. Of course it’s often trendy to write nonstandard C (using, say, GNU extensions) or nonstandard HTML/JavaScript. However, ignoring the standard being trendy doesn’t mean the standard doesn’t exist, or even that it isn’t useful. I always make sure my Haskell is Haskell2010, and I try to avoid dependencies that use egregious extensions.

                                                                                                                                    2. 2

                                                                                                                                      Honestly curious: are there any other Haskell compilers out there? Are they used in production?

                                                                                                                                      Also, what is a definition of a true Haskell? I always thought it’s what’s in GHC.

                                                                                                                                      1. 5

                                                                                                                                        There’s a Haskell which runs on the JVM - Frege. But it makes no attempt to be compatible with the version of Haskell that GHC impements, for good reasons. Hugs is a Haskell interpreter (very out of date now, but still works fine for learning about Haskell.) There a bunch of other Haskell compilers, mostly research works that are now no longer in development - jhc, nhc98 etc etc.

                                                                                                                                        But GHC is the dominant Haskell compiler by far. I don’t think there are any others in active development, apart from Frege, which isn’t interested in being compatible with GHC.

                                                                                                                                        (“True Haskell” is the Haskell defined in the Haskell Report, but real world Haskell is the Haskell defined by what GHC + your choice of extensions accepts.)

                                                                                                                                        1. 2

                                                                                                                                          There are other compilers and interpreters. None of them is anywhere near as popular as GHC, and usually when one does something interesting GHC consumes the interesting parts.

                                                                                                                                          There is definitely a standard, though: https://www.haskell.org/onlinereport/haskell2010/

                                                                                                                                          The whole reason language extensions are called “extensions” and require a magic pragma to turn on is that they are not features of the core language (Haskell) but experimental features of the compiler in question.

                                                                                                                                        2. 1

                                                                                                                                          In short, GHC Haskell is a language designed by survival-of-the-fittest.

                                                                                                                                        3. 3

                                                                                                                                          Overly terse error messages are bad, but they are better than wrong error messages. Some things are much harder to give helpful error messages for than others.

                                                                                                                                          I wish people spend more time improving error reporting, at least in cases when the way to do it is well understood. There is no reason for say TOML or JSON parsers to just say “Syntax error”. But, YAML parsers are pretty much doomed to give unhelpful errors just because the language syntax is ambiguous by design.

                                                                                                                                          And then some errors are only helpful because we know what their mean. Consider a simple example:

                                                                                                                                          Prelude> 42 + "hello world"
                                                                                                                                          
                                                                                                                                          <interactive>:1:1: error:
                                                                                                                                              • No instance for (Num [Char]) arising from a use of ‘+’
                                                                                                                                              • In the expression: 42 + "hello world"
                                                                                                                                                In an equation for ‘it’: it = 42 + "hello world"
                                                                                                                                          

                                                                                                                                          How helpful is it to a person not yet familiar with type classes? Well, it just isn’t. It’s not helping the reader to learn anything about type classes either.

                                                                                                                                          1. 1

                                                                                                                                            I’ve seen some good suggestions on r/haskell for improving the wording of these errors.

                                                                                                                                          2. 2

                                                                                                                                            The error they’re talking about is a kind of type error they’ve not worked with. It’s produced if you forget to construct or use a structure. I I’m guessing it’s technically “proper” but the produced error message may be difficult to interpret.

                                                                                                                                            They’ve ensured it’s a feature you can entirely ignore if you want to. Everybody’s not convinced they need this.

                                                                                                                                            I otherwise dunno what they’re doing and I’m scratching my head at the message. Something like “Oh cool you’ve done this… … … So where are the types?”

                                                                                                                                            1. 2

                                                                                                                                              So you never got a C++ template error in the good olden days? Seriously though, it just got merged. It’s not released or “shipped” in any means.

                                                                                                                                              1. 0

                                                                                                                                                So you never got a C++ template error in the good olden days?

                                                                                                                                                No, because I looked at the language, figured out that the people involved completely lost their fucking mind, and moved on.

                                                                                                                                                Seriously though, it just got merged. It’s not released or “shipped” in any means.

                                                                                                                                                They took 4 years to arrive at the current state, which I’ll approximate at roughly 10% done (impl unfinished, spec has unresolved questions, documentation doesn’t really seem to exist, IDE support not even on the radar).

                                                                                                                                                So if you assume that there will be a Haskell version in the next 36 years, then this thing is going to end up in some Haskell release sooner or later.

                                                                                                                                                1. 2

                                                                                                                                                  So if you assume that there will be a Haskell version in the next 36 years, then this thing is going to end up in some Haskell release sooner or later.

                                                                                                                                                  Could you elaborate on this? If practical users of linear types will only use them if they have good error messages, and early testers want to work out the kinks now, what’s wrong with having a half-baked linear types feature with no error messages permanently enshrined in GHC 8.12?

                                                                                                                                          1. 5

                                                                                                                                            mostly watching the USA devolve into a military dictatorship

                                                                                                                                            1. 23

                                                                                                                                              I don’t; I gaze upon a wasteland of attempts.

                                                                                                                                              I wish I had figured something out by now.

                                                                                                                                              1. 2

                                                                                                                                                I will admit, the unicode one got me. I was expecting to be able to reverse the glyphs, but I suppose that probably isn’t that simple.

                                                                                                                                                1. 6

                                                                                                                                                  Translate it to Arabic or Hebrew ;-)

                                                                                                                                                  1. 3

                                                                                                                                                    I don’t know any Arabic, but to give a Hebrew example, reversing the letters of חיים should give you מייח . The mem glyph changes.

                                                                                                                                                    1. 4

                                                                                                                                                      It was a joke answer to a trick question. I meant that if the string was in English (a left-to-right language) you could translate it into a right-to-left language to “reverse” it.

                                                                                                                                                      1. 5

                                                                                                                                                        Non-joke answer:

                                                                                                                                                        There’s a crate for extended grapheme cluster-aware string reversal: https://crates.io/crates/unicode-reverse

                                                                                                                                                        I have a use case for string reversal using this crate (though the use case doesn’t really need to be extended grapheme cluester-aware, since ISO-8859-8 doesn’t support vocalized Hebrew): Synthesizing visual Hebrew from a corpus of logical Hebrew text for testing visual Hebrew detection.

                                                                                                                                                        1. 2

                                                                                                                                                          Yeah that completely wooshed me

                                                                                                                                                    2. 2

                                                                                                                                                      Consider: how does one reverse “ijk”? Next: how does one reverse “Dijkstra”?

                                                                                                                                                      1. 1

                                                                                                                                                        Fun example, but falls flat if the person answering the question doesn’t know a damn about the Dutch “ij”. I think it’s a regular diphthong, as far as diphthongs can be regular :P

                                                                                                                                                        Now you can take this as proving or disproving your point, but if you give me a word/name in a language I don’t speak I don’t have any qualms about reversing it and pretending to not know (or really don’t know) if there’s a diphthong in it.

                                                                                                                                                    1. 5

                                                                                                                                                      The libra project depends on several fairly new “wild west” libraries for building experimental cryptosystems that have only emerged in the last few years

                                                                                                                                                      Uh, Curve25519 based stuff is everywhere, including TLS, SSH and Tor. It’s very much production quality. And dalek is a very good implementation that has been audited. This part of the article is terrible.

                                                                                                                                                      As for everything else… yeah, typical cryptocurrency stuff.

                                                                                                                                                      1. 4

                                                                                                                                                        Implementations, not algorithms.

                                                                                                                                                        1. 2

                                                                                                                                                          I mentioned the implementations, as you can see.

                                                                                                                                                          The

                                                                                                                                                          experimental cryptosystems

                                                                                                                                                          reference sort of looks like it refers to the algorithms. Might be referring to the stuff they’re using the algorithms for, of course (“depends […] for building …” not “libraries [that are] for building …”) but whatever