1. 6

    Couldn’t Apple avoid this mess by making a local cache of approved certs on each Mac and updating it periodically? I.e. the same thing APT does for package metadata (for all packages available in enabled repositories). You can search the cache (with apt search or apt-cache search) without contacting any servers. Only, in the case of Apple, it wouldn’t be a cache of package metadata, but a cache of approved developer certs. Then they could check all the apps without contacting any servers. This way both privacy and performance concerns are addressed. It would also allow users to launch programs during server downtime.

    1. 8

      That is indeed what they do, for security reasons they don’t require an explicit action from the user to refresh the cache, but they do have a cache, and only check it if a certain amount of time has passed. From the article:

      I should also add that after closing Firefox and opening it again, no requests were made. This is reasonable, and indicates that certificate checking isn’t performed at each launch but only after it hasn’t been performed for a certain period of time.

      1. 6

        That’s a different thing though. I’m talking about a cache of certificates, while the article seems to be talking about a cache of cert check responses. It means that server still gets some metadata about what programs the user launches. It’s also unclear why in this case people started having problems launching any non-Apple programs during server downtime.

        1. 1

          It needs to check in with the OCSP server occasionally to see if the developer certificate hasn’t been revoked, and the only way to chek using OCSP is to send the developer certificate to the OCSP server.

          People started having problems because the OCSP was available reachable, but never responded with an OK, presumably because it was being overloaded by new Bug Sur requests.

          1. 4

            yakubin meant to have a full copy of all certificates on the computer and updating that in full every so often. Therefore no data about individual apps is sent over the network, when launching an app.

            1. 2

              One could also use the google safe links algorithm where a partial hash of the app is sent and a bunch of responses is sent such that the server doesn’t know which apps the client is actually using.

              1. 2

                That only solves the privacy issue. It doesn’t solve the problem of the time needed to launch an app and reliability of the system in the face of server downtime or packets dropped by a firewall. Generally, I don’t think that execve on its own should ever prompt any network communication.

                EDIT: Your solution, by introducing frequent collisions, makes the whole mechanism essentially useless. Now you can’t differentiate between detected malware and legitimate software. To make it work at this point, you need to send the full hash to the server, defeating the point. I misunderstood, see @pgeorgi’s reply.

                1. 4

                  Your solution, by introducing frequent collisions, makes the whole mechanism essentially useless. Now you can’t differentiate between detected malware and legitimate software. To make it work at this point, you need to send the full hash to the server, defeating the point.

                  The solution works like this: client to server: I have a hash here, starting with 12. server to client: I have a set of good hashes 1234, 1235, 1237 and bad hashes 1236, 1239. This set is good for 24 hours.

                  That way the server or a listener can’t infer which hash is specifically requested while the client gets precise information to work with (plus some more that they probably won’t need)

                  1. 1

                    I stand corrected. Still, it solves only one of the issues.

                    1. 2

                      Sure, it doesn’t solve all the issues, but some minor additions can. I think there are two reasonable approaches:

                      1. Periodically receive the total list of revoked certs from Apple (which is presumably not huge).
                      • benefit: privacy
                      • cost: latency (it takes a while for clients to discover that an application is bad), potentially bandwidth, tho it’s probably fine.
                      1. Use OCSP but with some privacy preserving measures, like the safe search thing
                      • benefit: latency, bandwidth
                      • cost: you have to decide what to do if you can’t get at the server, but that’s not super hard: timeout fast; and cache results.

                      If Apple had just set a really small timeout on their request this OCSP system would have worked fine and no one would have noticed this outage.

                      1. 2

                        From the privacy point of view, looks like they will introduce encryption: https://www.macrumors.com/2020/11/15/apple-privacy-macos-app-authenticaion

                        1. 2

                          Thanks for the link. Obviously this approach only offers increased privacy Vs actors that aren’t Apple. But if you’re running a Mac obviously you have to trust Apple a bit, so I don’t think it’s unreasonable.

            2. 2

              The neat thing is that this means Apple can effectively remove software from your computer in addition to watching what you run.

              1. 5

                Everyone who ever built an antivirus or any sort of malware-removal tool is an active foot-soldier in the war against general-purpose computing, apparently. Seeing as all of those tools are designed to identify and remove software from your computer…

                1. 1

                  Those all have an off switch

                  1. 4

                    As does Apple’s equivalent.

                2. 2

                  So a professional is going to be checking if Transmission got their dev server owned this week instead of every single user having to check, cool.

          1. 15

            I have been on both sides of GitHub DMCA, so I can speak from experience.

            GitHub strongly favors keeping the content up.

            So yeah, its down now. But the repo owners can send a Counter Notice. Then RIAA has 14 days to file a copyright infringement suit in Federal Court, and then present a copy of the filing to GitHub.

            If GitHub does not receive the filing in time, or anything is wrong with the paperwork, the repo will go back up.

            https://docs.github.com/en/free-pro-team@latest/github/site-policy/dmca-takedown-policy

            1. 4

              Do you know if this is true also in this case? The letter is not really the usual takedown notice: more here https://twitter.com/xor/status/1319861757301710848

              1. 9

                Sorry but I cant follow that. Twitter is absolute garbage. Maybe that guy can repost it to a blog or something.

                1. 6

                  https://nitter.net/xor/status/1319861757301710848

                  Nitter is a free and open source alternative Twitter front-end focused on privacy.

                  1. 0

                    One reason Twitter easily spirals down to utter garbage, is the lack of downvotes. We can only retweet, and if we disagree, we can only shout back. There seems to be very little moderation. And of course, the length limit on the damn tweets effectively bans nuanced thought, which require too many characters. (Incidentally, this limitation makes me wonder how Twitter managed to get so popular.)

                    A free front end is unlikely to solve those problems.

            1. 4

              I wrote this article to give other people an introduction to NaN-boxing. Please let me know what you think!

              1. 2

                The link seems broken at tHe moment, can you update it?

                1. 1

                  Are you getting an actual 404 or the „Cannot find article“ message? If its the latter please try and reload the page. JS isn‘t my strong point :P

                  1. 2

                    “Cannot find article”. Indeed, after reloading a couple of times it did load. Thanks!

              1. 1

                Unfortunately most Medium links require an account with them for access.

                1. 2

                  Sorry I was not aware of that, I do not have an account myself but I can see the article. That’s bad (and it is a pity since the article is really good), I’ll avoid sharing medium articles from now on.

                1. 1

                  The examples are mindblowing

                  1. 2

                    There is a very interesting new article on it! Hackers Tell the Story of the Twitter Attack From the Inside https://www.nytimes.com/2020/07/17/technology/twitter-hackers-interview.html

                    1. 7

                      Twitter has announced preliminary investigation results that this was a social engineering attack of internal tooling.

                      1. 3

                        I hope they will make a detailed post explaining what happened once they finish their forensic analysis and restore the system.

                        1. 2

                          This might be a case where their desire for transparency conflicts with their exposure to liability. It’ll be interesting to see how it plays out.

                      1. 24

                        That headline is pretty confusing. It seems more likely twitter itself was compromised, than tons of individual users (billionaires, ex-leaders, etc)?

                        1. 18

                          You’re right. This is a case of Verge reporting what they’re seeing, but the scope has grown greatly since the initial posts. There have since been similar posts to several dozen prominent accounts, and Gemini replied that it has 2FA.

                          Given the scope, this likely isn’t accounts being hacked. I suspect that either the platform or an elevated-rights Twitter content admin has been compromised.

                          1. 12

                            Twitter released a new API today (or was about to release it? Not entirely clear to me what the exact timeline is here), my money is on that being related.

                            A ~$110k scam is a comparatively mild result considering the potential for such an attack, assuming there isn’t some 4D chess game going on as some are suggesting on HN (personally, I doubt there is). I don’t think it would be an exaggeration to say that in the hands of the wrong people, this could have the potential to tip election results or even get people killed (e.g. by encouraging the “Boogaloo” people and/or exploiting the unrest relating to racial tensions in the US from some strategic accounts or whatnot).

                            As an aside, I wonder if this will contribute to the “mainstreaming” digital signing to verify the authenticity of what someone said.

                            1. 13

                              or even get people killed

                              If the Donald Trump account had tweeted that an attack on China was imminent there could’ve been nuclear war.

                              Sounds far-fetched, but this very nearly happened with Russia during the cold war when Reagan joked “My fellow Americans, I’m pleased to tell you today that I’ve signed legislation that will outlaw Russia forever. We begin bombing in five minutes.” into a microphone he didn’t realize was live.

                              1. 10

                                Wikipedia article about the incident: https://en.wikipedia.org/wiki/We_begin_bombing_in_five_minutes

                                I don’t think things would have escalated to a nuclear war that quickly; there are some tensions between the US and China right now, but they don’t run that high, and a nuclear war is very much not in China’s (or anyone’s) interest. I wouldn’t care to run an experiment on this though 😬

                                Even in the Reagan incident things didn’t seem to have escalated quite that badly (at least, in my reading of that Wikipedia article).

                                1. 3

                                  Haha. Great tidbit of history here. Reminded me of this 80’s gem.

                                  1. 2

                                    You’re right - it would probably have gone nowhere.

                                2. 6

                                  I wonder if this will contribute to the “mainstreaming” digital signing to verify the authenticity of what someone said

                                  It’d be nice to think so.

                                  It would be somewhat humorous if an attack on the internet’s drive-by insult site led to such a thing, rather than the last two decades of phishing attacks targeting financial institutions and the like.

                                  1. 3

                                    I wonder if this will contribute to the “mainstreaming” digital signing to verify the authenticity of what someone said.

                                    A built-in system in the browser could create a 2FA system while being transparent to the users.

                                    1. 5

                                      2fa wouldn’t help here - the tweets were posted via user impersonation functionality, not direct account attacks.

                                      1. 0

                                        If you get access to twitter, or the twitter account, you still won’t have access to the person’s private key, so your tweet is not signed.

                                        1. 9

                                          Right, which is the basic concept of signed messages… and unrelated to 2 Factor Authentication.

                                          1. 2

                                            2FA, as I used it, means authenticating the message, via two factors, the first being access to twitter account, and the second, via cryptographically signing the message.

                                            1. 3

                                              Twitter won’t even implement the editing of published tweets. Assuming they’d add something that implicitely calls their competence in stewarding people’s tweets is a big ask.

                                              1. 2

                                                I’m not asking.

                                    2. 2

                                      A ~$110k scam

                                      The attacker could just be sending coins to himself. I really doubt that anyone really falls for a scam where someone you don’t know says “give me some cash and I’ll give you double back”.

                                      1. 15

                                        I admire the confidence you have in your fellow human beings but I am somewhat surprised the scam only made so little money.

                                        I mean, there’s talk about Twitter insiders being paid for this so I would not be surprised if the scammers actually lost money on this.

                                        1. 10

                                          Unfortunately people do. I’m pretty sure I must have mentioned this before a few months ago, but a few years ago a scammer managed to convince a notary to transfer almost €900k from his escrow account by impersonating the Dutch prime minister with a @gmail.com address and some outlandish story about secret agents, code-breaking savants, and national security (there’s no good write-up of the entire story in English AFAIK, I’ve been meaning to do one for ages).

                                          Why do you think people still try to send “I am a prince in Nigeria” scam emails? If you check you spam folder you’ll see that’s literally what they’re still sending (also many other backstories, but I got 2 literal Nigerian ones: one from yesterday and one from the day before that). People fall for it, even though the “Nigerian Prince” is almost synonymous with “scam”.

                                          Also, the 30 minute/1 hour time pressure is a good trick to make sure people don’t think too carefully and have to make a snap judgement.

                                          As a side-note, Elon Musk doing this is almost believable. My friend sent me just an image overnight and when I woke up to it this morning I was genuinely thinking if it was true or not. Jeff Bezos? Well….

                                          1. 12

                                            People fall for it, even though the “Nigerian Prince” is almost synonymous with “scam”.

                                            I’ve posted this research before but it’s too good to not post again.

                                            Advance-fee scams are high touch operations. You typically talk with your victims over phone and email to build up trust as your monetary demands escalate. So anyone who realizes it’s a scam before they send money is a financial loss for the scammer. But the initial email is free.

                                            So instead of more logical claims, like “I’m an inside trader who has a small sum of money to launder” you go with a stupidly bold claim that anyone with a tiny bit of common sense, experience, or even the ability to google would reject: foreign prince, huge sums of money, laughable claims. Because you are selecting for the most gullible people with the least amount of work.

                                      2. 5

                                        My understand is that Twitter has a tool to tweet as any user, and that tool was compromised.

                                        Why this tool exists, I have no idea. I can’t think of any circumstance where an employee should have access to such a tool.

                                        Twitter has been very tight-lipped about this incident and that’s not a good look for them. (I could go on for paragraphs about all of the fscked up things they’ve done)

                                        1. 5

                                          or an elevated-rights Twitter content admin

                                          I don’t think content admins should be able to make posts on other people’s account. They should only be able to delete or hide stuff. There’s no reason they should be able to post for others, and the potential for abuse is far too high for no gain.

                                          1. 6

                                            Apparently some privileges allow internal Twitter employees to remove MDA and reset passwords. Not sure how it played out but I assume MFA had to be disabled in some way.

                                          1. 5

                                            That’s a good article! Vice has updated that headline since you posted to report that the listed accounts got hijacked, which is more accurate. Hacking an individual implies that the breach was in their control: phone, email, etc. This is a twitter operations failure which resulted in existing accounts being given to another party.

                                        1. 2

                                          Just to add more to the picture, development of multicore is steadily ongoing, with more and more patches getting integrated into the main compiler: https://discuss.ocaml.org/t/multicore-ocaml-june-2020/6047

                                          1. 9

                                            Great news! I am eager to try this!

                                            Turn on -XLinearTypes, and the first thing you will notice, probably, is that the error messages are typically unhelpful: you will get typing errors saying that you promised to use a variable linearly, but didn’t. How hasn’t it been used linearly? Well, it’s for you to puzzle out. And while what went wrong is sometimes egregiously obvious, it can often be tricky to figure the mistake out.

                                            So, basically, GHC just got its own “Syntax error” a la OCaml… just a bit more specialized :p.

                                            1. 11

                                              Maybe it’s just me, but to me OCaml’s errors are terse and unhelpful and GHC’s errors are… verbose and unhelpful. ;)

                                              There are interesting papers that show working ways to improve both, but I wonder why none of those improvements are in the mainline compilers.

                                              1. 2

                                                Good error reporting is easiest if it’s built into the compiler front end from the start. If a new algorithm comes along to improve the error information it’s almost never going to be a simple job to drop it into an existing compiler.

                                                You need type information & parse information from code that’s potentially incorrect in both spaces, so any error algorithm usually has to be tightly integrated into both parts of the compiler front end. That tight integration usually means that improving compiler errors is a significant amount of work.

                                                1. 3

                                                  It varies. What puzzles me is that a lot of time ready to use, mergeable patches take much longer to merge than they should.

                                                  Like this talk: https://ocaml.org/meetings/ocaml/2014/chargueraud-slides.pdf

                                                  1. 1

                                                    Do you also have a link for a patch for the improved error messages?

                                                    A lot of work has been going on to move OCaml to a new parser and improve error messages. Even though there is a lot still needed to be done, latest releases have started improving a lot. Maybe we can still extract some useful bits from that effort and try again

                                                    1. 2

                                                      Turns out it was even made into a pull request that isn’t merged yet: https://github.com/ocaml/ocaml/pull/102

                                                      1. 1

                                                        Thanks. It is quite an informative PR actually, and explains why the change is not there yet and once can infer why it is easier to add informative messages in new languages and complier but it may be quite hard to retrofit them to seasoned ones

                                              2. 7

                                                Would you be kind enough to give me an ELI5 about what linear types are and what you can do with them?

                                                1. 29

                                                  In logic, normal implication like A implies B means whenever you have A, you can derive B. You have tautologies like “A implies (A and A)” meaning you can always infinitely duplicate your premises.

                                                  Linear implication is a different notion where deriving B from A “consumes” A. So “A linearly implies B” is a rule that exchanges A for B. It’s not a tautology that “A linearly implies (A and A).”

                                                  The classic example is you can’t say that “$3 implies a cup of coffee” but “$3 linearly implies a cup of coffee” makes sense. So it’s a logical form that reasons about resources that can be consumed and exchanged.

                                                  Same in functional programming. A linear function from type A to type B is one that consumes an A value and produces a B value. If you use it once with an A value then you can’t use it again with the same A value.

                                                  This is nice for some performance guarantees, but also for encoding safety properties like “a stream can only be read once” etc.

                                                  1. 6

                                                    Keynote: Linear Haskell: Practical Linearity in a Higher-Order Polymorphic Language https://skillsmatter.com/skillscasts/11067-keynote-linear-haskell-practical-linearity-in-a-higher-order-polymorphic-language

                                                    1. 5

                                                      It can be used to model protocols with type signatures. The following is in theory what you should be able to do.

                                                      data ConsoleInput
                                                          = Input String ConsoleOutput
                                                          | ExitInput
                                                      
                                                      data ConsoleOutput
                                                          = PrintLines ([String] ⊸ Console)
                                                          & PrintLastLines ([String] ⊸ ())
                                                      
                                                      greet :: ConsoleOutput ⊸ ()
                                                      greet console
                                                          = let PrintLines f = console
                                                            in step2 (f ["name?"])
                                                      
                                                      step2 :: ConsoleInput ⊸ ()
                                                      step2 ExitInput = ()
                                                      step2 (Input input console)
                                                          = let PrintLastLines f = console
                                                            in f ["hello " ++ input]
                                                      

                                                      If you combine it with continuation passing style, you get classical linear logic and it’s a bit more convenient to use.

                                                      If you model user interfaces with types, they should be quite useful.

                                                      I’m also examining and studying them: http://boxbase.org/entries/2020/jun/15/linear-continuations/

                                                      1. 1

                                                        Wikipedia gives a reasonable overview. The closest analogy would be something like move semantics – for example ownership in Rust can be considered as manifestation of linear types.

                                                        1. 6

                                                          Rust ownership is linear affine types. Linear types are similar but differ in the details. A shitty way of understanding it is affine types mimic ref counting and prevent you from having a ref count < 0. Linear types are more a way of acting like RAII in that you might create a resource but just “know” that someone later on in the chain does the cleanup.

                                                          Which I’m sure sounds similar but affine types allow for things like resource leaks but linear types should guarantee overall behavior to prevent it.

                                                          This all assumes my understanding and explanation is apt. I’m avoiding a ton of math and i’m sure the shitty analogy doesn’t hold up but behaviorally this is how I have it in my brain.

                                                          1. 2

                                                            Linearity Design Space: https://i.imgur.com/s0Mxhcr.png

                                                            1. 2

                                                              I’m personally of the stance that the 2020 linear ghc stuff is more <= 1 usage, and kinda misses out on a lot of really fun expressivity that can fall out of making full classical linear logic first class. But that’s a long discussion in its own right , and I’ve yet to make the time to figure out the right educational exposition on that front

                                                              1. 1

                                                                it definitely seems more limited in scope/ambition compared to the effort ongoing for dependent types, for better or worse. Can’t say I know much about what first class linear logic would look like, but perhaps now there will be more discussion about such things.

                                                                1. 2

                                                                  The really amazing thing about full linear logic is it’s really sortah a rich way to just do mathematical modelling where everything has a really nice duality. The whole thing about linearity isn’t the crown jewel (though wonderfully useful for many applications ), it’s that you get a fully symmetric bag of dualities for every type / thing you can model.

                                                                  The paper that really made it click for me was mike shulmans linear logic for constructive mathematics paper. It’s just a fun meaty read even at a conceptual level. There’s a lot of other work by him and other folks that taken together just point to it being a nice setting for formal modelling and perhaps foundations of category theory style tools too!

                                                              2. 1

                                                                Not sure I can agree that Uniqueness types are the same as Linear types. Care to explain they’re similar sure but not the same thing and your… screenshot of a powerpoint? isn’t very illustrative of whatever point you’re trying to make here.

                                                                And from my experience with Idris, I’m not sure I’d call what Rust has Uniqueness types.

                                                                1. 1

                                                                  They are different rows in the matrix because they are different, of course.

                                                                  it’s from this presentation about progress on linear ghc a little over a year ago https://lobste.rs/s/lc20e3/linear_types_are_merged_ghc#c_2xp2dx skip to 56:00

                                                                  What is meant by Uniqueness types here is “i can guarantee that this function gets the unique ptr to a piece of memory” https://i.imgur.com/oJpN4eN.png

                                                        2. 2

                                                          Am I the only one thinking this is not how you ship language features?

                                                          If the compiler can’t even report errors correctly, the feature shouldn’t ship.

                                                          1. 15

                                                            If the compiler can’t even report errors correctly, the feature shouldn’t ship.

                                                            Its more this is an opt-in feature with crappy error reporting for now using computer programming design features not in use in most programming languages. Its going to have rough edges. If we required everything to be perfect we’d never have anything improved. Linear types like this also might not have a great way to demonstrate errors, or the domain is new so why not ship the feature for use and figure out what kind of error reporting you want based on feedback.

                                                            1. 13

                                                              Many people do not realize that haskell is a research language and GHC is one of the main compilers for it. This is an experimental feature in a research language. If it works out well, then it will be standardized.

                                                            2. 5

                                                              Other people have sort-of said it, but not clearly enough I think. This is not a language feature being added. It is a feature-flagged experimental feature of a particular compiler. Most such compiler extensions never make it into real Haskell, and the ones that do take years after they are added to a compiler to make it to a language spec.

                                                              1. 4

                                                                for all practical purposes isn’t “real Haskell” defined by what ghc implements these days?

                                                                1. 2

                                                                  Yes, all the other implementations are dead. They still work, but they won’t run most modern Haskell code, which usually uses a bunch of GHC extensions.

                                                                  1. 1

                                                                    You might say “isn’t it not popular to write standards-compliant Haskell these days?” and you’d be right. Of course it’s often trendy to write nonstandard C (using, say, GNU extensions) or nonstandard HTML/JavaScript. However, ignoring the standard being trendy doesn’t mean the standard doesn’t exist, or even that it isn’t useful. I always make sure my Haskell is Haskell2010, and I try to avoid dependencies that use egregious extensions.

                                                                  2. 2

                                                                    Honestly curious: are there any other Haskell compilers out there? Are they used in production?

                                                                    Also, what is a definition of a true Haskell? I always thought it’s what’s in GHC.

                                                                    1. 5

                                                                      There’s a Haskell which runs on the JVM - Frege. But it makes no attempt to be compatible with the version of Haskell that GHC impements, for good reasons. Hugs is a Haskell interpreter (very out of date now, but still works fine for learning about Haskell.) There a bunch of other Haskell compilers, mostly research works that are now no longer in development - jhc, nhc98 etc etc.

                                                                      But GHC is the dominant Haskell compiler by far. I don’t think there are any others in active development, apart from Frege, which isn’t interested in being compatible with GHC.

                                                                      (“True Haskell” is the Haskell defined in the Haskell Report, but real world Haskell is the Haskell defined by what GHC + your choice of extensions accepts.)

                                                                      1. 2

                                                                        There are other compilers and interpreters. None of them is anywhere near as popular as GHC, and usually when one does something interesting GHC consumes the interesting parts.

                                                                        There is definitely a standard, though: https://www.haskell.org/onlinereport/haskell2010/

                                                                        The whole reason language extensions are called “extensions” and require a magic pragma to turn on is that they are not features of the core language (Haskell) but experimental features of the compiler in question.

                                                                      2. 1

                                                                        In short, GHC Haskell is a language designed by survival-of-the-fittest.

                                                                        1. -2

                                                                          If you haven’t noticed, the language spec is dead.

                                                                        2. 3

                                                                          Overly terse error messages are bad, but they are better than wrong error messages. Some things are much harder to give helpful error messages for than others.

                                                                          I wish people spend more time improving error reporting, at least in cases when the way to do it is well understood. There is no reason for say TOML or JSON parsers to just say “Syntax error”. But, YAML parsers are pretty much doomed to give unhelpful errors just because the language syntax is ambiguous by design.

                                                                          And then some errors are only helpful because we know what their mean. Consider a simple example:

                                                                          Prelude> 42 + "hello world"
                                                                          
                                                                          <interactive>:1:1: error:
                                                                              • No instance for (Num [Char]) arising from a use of ‘+’
                                                                              • In the expression: 42 + "hello world"
                                                                                In an equation for ‘it’: it = 42 + "hello world"
                                                                          

                                                                          How helpful is it to a person not yet familiar with type classes? Well, it just isn’t. It’s not helping the reader to learn anything about type classes either.

                                                                          1. 1

                                                                            I’ve seen some good suggestions on r/haskell for improving the wording of these errors.

                                                                          2. 2

                                                                            The error they’re talking about is a kind of type error they’ve not worked with. It’s produced if you forget to construct or use a structure. I I’m guessing it’s technically “proper” but the produced error message may be difficult to interpret.

                                                                            They’ve ensured it’s a feature you can entirely ignore if you want to. Everybody’s not convinced they need this.

                                                                            I otherwise dunno what they’re doing and I’m scratching my head at the message. Something like “Oh cool you’ve done this… … … So where are the types?”

                                                                            1. 2

                                                                              So you never got a C++ template error in the good olden days? Seriously though, it just got merged. It’s not released or “shipped” in any means.

                                                                              1. 0

                                                                                So you never got a C++ template error in the good olden days?

                                                                                No, because I looked at the language, figured out that the people involved completely lost their fucking mind, and moved on.

                                                                                Seriously though, it just got merged. It’s not released or “shipped” in any means.

                                                                                They took 4 years to arrive at the current state, which I’ll approximate at roughly 10% done (impl unfinished, spec has unresolved questions, documentation doesn’t really seem to exist, IDE support not even on the radar).

                                                                                So if you assume that there will be a Haskell version in the next 36 years, then this thing is going to end up in some Haskell release sooner or later.

                                                                                1. 2

                                                                                  So if you assume that there will be a Haskell version in the next 36 years, then this thing is going to end up in some Haskell release sooner or later.

                                                                                  Could you elaborate on this? If practical users of linear types will only use them if they have good error messages, and early testers want to work out the kinks now, what’s wrong with having a half-baked linear types feature with no error messages permanently enshrined in GHC 8.12?

                                                                          1. 2

                                                                            It’s not clear from the webpage why/how this is a breakthrough. Please elucidate.

                                                                            1. 1

                                                                              Inverting the Laplace Transform has been central in many applications (and an issue) for a long time. They develop a method that is not just fast, but superior to all known ones in all respects. The 2019 paper linked in the article is open access, and does a very good job to explain the problem, their solution and the metrics for comparison (the demo at the end of the page already gives a good hint)

                                                                              1. 2

                                                                                I understand the problem and its importance. But the paper seems to be proposing another solution that is better in some cases than the alternatives.

                                                                                I would not call it a breakthrough.

                                                                            1. 5

                                                                              What’s the actual status of this? I’ve always had a hard time gluing together these things that all seem to be true:

                                                                              1. Functional programming is where parallelism is ( easy | even really possible )
                                                                              2. OCaml is the place where most usable functional programming is happening
                                                                              3. OCaml’s parallel processing support is coming RSN (for what appears to be ~10 years)

                                                                              Am I missing something? It almost seems like cold fusion at this point.

                                                                              1. 8

                                                                                You can find the most recent update, which adds context to that paper here: https://discuss.ocaml.org/t/multicore-update-april-2020-with-a-preprint-paper/5630 The changes coming from the multicore work is being merged in the main compiler already since a while.

                                                                                It will arrive in batches, with typed effects arriving last, and it will take more time before it fully happens. The work is steadily moving forward and the benchmarks are already quite good. It is taking long because it wants to be backward compatible at all levels and it wants to preserve the single core speed of the current ocaml implementation.

                                                                                1. 1

                                                                                  Thanks for posting this!

                                                                                2. 4

                                                                                  Functional programming techniques can enable parallelism (see Erlang) but other factors in the language can play a big role (see Rust). The OCaml folks a long time ago made a decision to optimize running really fast but on one thread at a time. This had its benefits and drawbacks; a lot of people chose to take advantage of the speed and correctness guarantees that OCaml can offer, while working around the single thread limitation–much like JavaScript, Python, and Ruby people routinely do.

                                                                                1. 3

                                                                                  OCaml has a few emergent tools, already used for some research in numerics and machine learning: owl (ocaml.xyz), lacaml (mmottl.github.io/lacaml) and slap (akabe.github.io/slap). They can also be used in jupyter nktebooks, by means of the ocaml-jupyter package, although for plotting they are not at the level of python yet.

                                                                                  For machine learning there are also ocaml-torch and tensorflow-ocaml which provide bindings to libtorch and tensorflow.

                                                                                  1. 7

                                                                                    For those of us who don’t follow OCaml development, what’s the big story line here?

                                                                                    1. 14

                                                                                      Externally visible changes: better error messages and some new optimizations. Internally there have been a lot of bugfixes and code cleanups, making it easier in the future to merge the new big features, like multicore (this does not mean that they are there yet, but things are moving at a good pace)

                                                                                    1. 6

                                                                                      I went through a (long) period of OCaml development. It’s surprisingly common in the enterprise, or at least was when I was using it. It’s a truly beautiful language.

                                                                                      That being said, I’ve always fallen on the Caml side of the ML-vs-OCaml debate. I’ve always been loathe to mix paradigms and the “O” part of OCaml is, while absolutely done in the right way, a lot to add on top of Caml (or ML).

                                                                                      (Again, it’s easy to criticize. I’ve never written a beautiful language used by millions. It’s solely a personal qualm of mine.)

                                                                                      1. 2

                                                                                        Are there advantages to Caml over Standard ML?

                                                                                        1. 3

                                                                                          Countless advantages. It’s purely a minimalism thing on my part.

                                                                                        2. 1

                                                                                          Hi,

                                                                                          could you sketch out the type of projects you were building in Ocaml? If starting over, would you consider Haskell as a suitable replacement?

                                                                                          1. 1

                                                                                            Does the “O” part leak into everything else or can you easily work more or less only with the Caml side of OCaml?

                                                                                            1. 5

                                                                                              In my experience (YMMV), you can pretty much ignore the O part…but (a) that was years ago and (b) I’m really bad at ignoring things even when I’m not using them.

                                                                                              1. 2

                                                                                                It is still the same. In fact you almost never need to use The O part, and also when you think you do, in most cases, it is more idiomatic to use modules and module functors (parametrised modules)

                                                                                                1. 2

                                                                                                  Though there are cases where the O part might be useful. Personally I don’t use it, but just as @lorddimwit noted it is quite an interesting take on “objects” that might even benefit predominantly functional programmers.

                                                                                                2. 1

                                                                                                  Thank you.

                                                                                            1. 8

                                                                                              Whenever I read about yet another GUI toolkit that doesn’t use native widgets, I want to scream, “No! Don’t!” Here’s a comment that succinctly lists several problems with non-native widgets. Many of them are macOS-specific, but some are applicable to any platform.

                                                                                              1. 8

                                                                                                while this appeals to the purist, we are no longer just dealing with the QTs and wxWidgets of the world. The evil facing us is a multi-100mbs heavy electron app that eats similar amounts of RAM for showing a few widgets on screen (eg: privtunl’s toolbar).

                                                                                                1. 3

                                                                                                  “Evil” is an awfully strong word in this context. For all of Electron’s problems, at least it inherits Chromium’s support for platform accessibility APIs (on Windows and Mac). This makes Electron infinitely better for, say, a blind user who relies on a screen reader, compared to a typical non-native GUI toolkit with no accessibility support. Of course, for many users, like the author of the comment I linked previously, a truly native GUI would be better still. But let’s not block some users from using our applications because Electron offends our aesthetic sense.

                                                                                                2. 4

                                                                                                  I have not tried it, but for native widgets there is also https://github.com/briskml/brisk that seems promising

                                                                                                  1. 2

                                                                                                    What can someone even use to make cross platform native apps with native widgets? I’ve been looking for a desktop UI framework to use but it looks like the only “good” way is to rewrite the UI once with Cocoa, once with whatever Windows uses (is there even a standard?), and once with GTK+.

                                                                                                    1. 3

                                                                                                      Things like wxWidgets and Eto use native widgets. Of course the result is going to look a bit weird because macOS, Windows and GTK have very different UI idioms and showing the same UI everywhere would look out of place everywhere (except Windows, where inconsistency is kinda the norm, and MS has multiple independent UI toolkits as part of the system). But with great effort, you could put platform specific adaptations with these frameworks and make it look okay.

                                                                                                    2. 2

                                                                                                      I think this only applies to mac. And even then only minimally.

                                                                                                      Linux doesn’t even really have a native, there’s qt, gtk, and then <random other apps> that all make up the core parts of any given system and all style differently.

                                                                                                      Windows technically has a native, but not even microsoft seems to care about it anymore.

                                                                                                      The vast majority of time spent on most computers is probably in the web browser, which on all platforms does not look “native”.

                                                                                                      Arguably “native” these days should probably mean “imitate the web” not “use OS APIs” since that’s what most people are “native” to anyways.

                                                                                                      1. 1

                                                                                                        I think this only applies to mac.

                                                                                                        Almost certainly. There are many people who use a Mac because it’s Unix with a usable interface; many (although fewer these days) who use them because it’s what they had in school; but there are some, and this is a shrinking minority, I fear, who use Macs because they prefer the Macintosh, and they make an affirmative choice to dwell in the Mac, and select their tools based on how well they conform to platform convention. This is not a thing that people who prefer Windows or non-Mac Unixes has ever used as a decision procedure.

                                                                                                    1. 5

                                                                                                      Somewhat contrary to the OCaml community at large we tend to use Jane Street Core and where relevant Async. Core is an alternative, extensive and opinionated standard library to replace the default (“compiler”) standard library.

                                                                                                      The other alternatives in the standard library space are less exciting:

                                                                                                      • The compiler standard library is missing many useful things, the functions tend not to be tail-recursive and it defaults to throwing exceptions. In general this library is not ready for production use and leads to every project having a random module with more or less well implemented missing bits.
                                                                                                      • Extlib went a long time unmaintained and keeping compatibility with the standard library makes it default to exceptions
                                                                                                      • Batteries started as maintained superset of Extlib, but is hardly maintained nowadays. […]
                                                                                                      • Containers is a good library, carefully designed. If not for Base, this would be the most interesting contender for a good standard library replacement

                                                                                                      Would it be fair to say that there is no actual standard library for OCaml? In the ReasonML world (which I’m vaguely familiar with) there appears to be another one emerging, called Belt.

                                                                                                      From a Python perspective it’s kind of surprising that there are so many re-implementations of what is presumably core functionality. Are there things in one “standard” library that you wish you could have in another? Or is that that they all have the same features, but implemented differently? How much does the choice of a standard library affect you in terms of the other libraries you can use? Like, if some library is built on a standard library you don’t use, are you SOL or is it just a lot of fiddling to get it working?

                                                                                                      In JS land, where I’m also very comfortable, the complete lack of a standard library is also a problem, but because JS types are so basic, interop between libraries built on different foundations usually isn’t that much of a problem in practice. But I’d imagine the type system in OCaml makes things more complicated.

                                                                                                      Or am I just misunderstanding how OCaml works?

                                                                                                      1. 6

                                                                                                        OCaml ships with a standard library: https://caml.inria.fr/pub/docs/manual-ocaml/libref/index.html . This has a lot of stuff: lists, maps, arrays, I/O, file manipulation, networking, etc. But for industrial-strength usage you might need more. This is where Jane Street’s Core (or now the lighter-weight Base) comes in. It has issues like complexity, thin documentation, and somewhat closed development style. But for production use it can get the job done. In ReasonML you have Belt as you pointed out which is designed to compile down to really tight JavaScript.

                                                                                                        The upshot is that you will probably want some kind of a ‘standard’ library package for production use, but this is not a huge problem because of the strong package ecosystem. As to interop the two main competing libraries are Lwt and Async which are concurrency libraries, but serious packages which need concurrency support ship adapter packages for both of them, e.g. library foo will ship foo-lwt and foo-async.

                                                                                                        1. 1

                                                                                                          It is also true that there has been a few prerequisite needed to start refreshing the standard library. In addition to the strong backward compatibility story of the language itself (look how many years and releases took to move to safe-strings, or for the deprecation of a few strings functions), the flat namespacing was a potential blocker for many changes and reorganisations, to name a couple. The compatibility story in particular requires each change to be weighted and discussed very thoroughly: once it lands it got to stay for a very long time.

                                                                                                          Some of this issues have been now solved, and indeed the standard library in ocaml 4.08.0 will already be noticeably improved, and the improvement work is ramping up so I will not be surprised if ocaml 4.09.0 will be getting even better.

                                                                                                          I think the biggest pain point is currently the lack of safe resource acquisition and cleanup in the standard library, but discussions on this front have already started.

                                                                                                        2. 5

                                                                                                          Author of the post here.

                                                                                                          Would it be fair to say that there is no actual standard library for OCaml? In the ReasonML world (which I’m vaguely familiar with) there appears to be another one emerging, called Belt.

                                                                                                          The compiler standard library is the “standard library” in a way. I call it that in the blog post because it often seems like it is mostly there to implement OCaml, but missing many useful bits that people actually need for other real-world projects, which is why many projects use alternative “standard libraries” or reimplement their own (most of the time, badly).

                                                                                                          It is sort of similar to Python, where you have all kinds of things included but for actual production software it is not great to use it (asyncore/asynchat, all the command line parsers, config file parsers, HTTP clients, HTTP servers).

                                                                                                          Reason is not helping much there, by introducing yet another library that you might potentially want to use in OCaml. I am not a big fan of the “reimplement everything that already exists in OCaml”, but I guess it is an entirely different community with different values. I think it might be a bit of a lost opportunity but what can you do.

                                                                                                          Are there things in one “standard” library that you wish you could have in another? Or is that that they all have the same features, but implemented differently?

                                                                                                          There are two approaches

                                                                                                          1. extend the existing library with things people reimplement on their own (like exceptionless variants of functions, some useful features like operators and combinators) which is what Extlib, Batteries and Containers are doing. This can be useful if you’re using the default standard library and just want some additional functionality.
                                                                                                          2. replace the existing library by vastly changing the API, to provide an API that is more aligned to how you would want the stdlib to look like if there were no backwards-compatibility concerns. This is what Base and Core are doing.

                                                                                                          Like, if some library is built on a standard library you don’t use, are you SOL or is it just a lot of fiddling to get it working?

                                                                                                          It is not a problem at all. You can use all code just fine, no matter which library it uses without having to care what it was implemented in. There is some resistance to Core, because it pulls in a lot of dependencies, which is why there are “smaller” variants like Core_kernel and Base with reduced amounts of dependencies, while preserving as much of the Core API as possible.

                                                                                                          On the other hand, if the library uses a different concurrency library than what you use, then you’re sort-of SOL, since you can’t really combine an Lwt.t (from Lwt) with an Deferred.t (from Async). This is e.g. similar to Twisted and Tornado in Python, where it is not really feasible to combine them. I heard some considerations to base them on a common foundation but not sure where this is going. The effect system will likely obsolete concurrency libraries in the future in any case.

                                                                                                          1. 3

                                                                                                            That was helpful - thanks for the detailed reply!

                                                                                                            1. 2

                                                                                                              wanted to note a thanks as well, really appreciate it when the author dives in to extend the conversation from an interesting piece.

                                                                                                          1. 3

                                                                                                            This was quite and interesting reading!

                                                                                                            It would be interesting to see also an example that deals with more complex ocaml structures, like records containing strings, bytes, and (why not) arrays or lists. Or maybe a GADT. Although I think it would just make the plumbing More complex but be not much more than what you do writing the usual C bindings

                                                                                                            1. 5

                                                                                                              I would love to see a language like OCaml (or another ML like dialect) to be able to make use of Rust’s emerging ecosystem, but I’m afraid we’re still a long way from seeing that happen. Also check out Gluon http://gluon-lang.org/ for an ML like language embedded in Rust (personally I’d prefer a standalone language that works with Rust though).

                                                                                                              1. 2

                                                                                                                I’m a little confused. Outside of being developed in Rust what makes gluon “embedded in Rust” vs a “standalone language”? It appears that while gluon is heavily influenced by Rust it can, and does, function as a standalone language as well if you wanted it to. Which, as they point out, is similar to lua.

                                                                                                                I feel like embedded in the context of gluon only applies to their goal of being easy to embed as opposed to being embedded itself.

                                                                                                                1. 2

                                                                                                                  I agree that it ‘can’ function as a standalone language as well. But I am not sure if I agree it currently does, AFAIK you can’t create a gluon ‘script’ that runs with #!/usr/bin/env gluon and neither can you compile a gluon source to a binary without embedding it first in a Rust application. I am also unsure whether you can write a gluon module using rust and then import it from your ‘standalone’ gluon program.

                                                                                                                  So I disagree that embedded in the context of gluon only applies to their goal of being easy to embed as opposed to being embedded itself.

                                                                                                                  But I do realize my criticism might be easily solved (although I have some doubts about the ‘write a gluon module using rust and then import it from your standalone gluon program’ part).

                                                                                                                  Please correct me where I am wrong, I’m fairly new to both Rust and Gluon.

                                                                                                                  1. 1

                                                                                                                    I’m fairly new to both Rust and Gluon.

                                                                                                                    Ditto. Thanks for clarification.