1.  

    That is amazing. This was not especially expected, and the author themselves note that none of the big questions have been answered, but they’re all now in much more tension. Can RP be derandomized? Whether P and RP can be separated is now an even bigger question than before.

    I’m not yet sure whether the proof convinces me. The high-level sketch seems reasonable, and the pieces seem alright, but I don’t yet grok the deep algorithmic insight that makes the entire scheme work.

    The author proposes a bold new research direction: Let’s assume that P ≠ RP and that RP cannot be derandomized! Instead, let’s focus on separating P from RP.

    The paper collapses PH to BPP, which ends a lot of interesting possibilities around BPP. BPP might still equal BQP, at least; that’s still open. We still have two interesting avenues of separation; we could follow zero-knowledge proofs, or we could explore constraint satisfaction problems. Intuitively, surely CSP = NP; but we haven’t proven it, and maybe it’s time to examine that intuition more closely. Acronyms are confusing; a big diagram is helpful for me.

    1. 2

      It’s an intersection of my interests! I have a pull request for getting the relpipe tools into nixpkgs, I like Wikidata, and I’m working on relational semantics for Lojban. The idea of using Lojban to query Wikidata is nearing seriousness.

      1. 1

        Thanks! Maintaining packages in distributions in very helpful.

      1. 2

        I am interested in not just what this toy theory reproduces, but what it lacks. Without the Kochen-Specker theorem, there is no Free Will Theorem. Additionally, the theory works with permutations of data, which lend it a reversibility. I think that these are connected: The fact that contextuality requires some sort of question-and-answer conversation with particles being measured is related to the fact that we cannot reverse real-world processes in the laboratory.

        There are lots of fun small observations, too. For example, the author notes that pairs of classical bits are used to implement the data for qubits, which is analogous to the Chu construction for linear logic, where one reversible datum is implemented as two data, one traveling “forward” or “truthward” and one traveling “backward” or “contradictionward”.

        The central mystery of the paper is a deep and good one: What is it about reality that, due to this essential epistemic incompleteness of qubits, we will never be able to directly observe? Does it have term-rewriting co-hygiene over some sort of rewriting grammar? Does it implement gravity?

        To add something not present in the paper, let’s quickly analyze Newcomb’s Paradox in this setting. I am fond of a recent realization, which is that the Free Will Theorem demolishes Newcomb’s Paradox by, in a nutshell, forcing the Predictor to predict the outcome of a coin flip, and ensuring that the coin flip cannot be predicted by letting it exercise some Free Will. Here, our toy theory doesn’t have the Free Will Theorem. Instead, note that we are manipulating epistemic states, but the Predictor’s claimed data is about ontic states, which is forbidden by the theory’s rules. So, just like with the flipped coin, the Predictor can never have better than a 50/50 chance of correctly predicting which ontic state corresponds with a given epistemic state, which is a total defeat.

        1. 1

          Thanks for linking this article - I’ve heard of Goedel’s proof before of course, but never previously tried to understand it. It’s interesting that one of the key insights involves creating a mapping between the typographical symbols of written math proofs, and integers. This insight seems a little more obvious now that ASCII has existed for over half a century and any programmer or hardware engineer is used to treating textual symbols as numbers inside a computer. Goedel had this insight in 1931, which was before the age of the electronic computer - but not that many years before it.

          1. 1

            Turing had many of the same insights around the same time, and everybody was, if not talking to each other, at least vaguely aware of each other. What’s really interesting to me is that around three decades later, Lawvere simplified and unified these results of Turing, Tarski, Gödel, and also Cantor and Russell, into Lawvere’s fixed-point theorem, and this unified theorem is still quite unknown, even among folks who grok Gödel’s work.

          1. 4

            The video covers an introduction to the landscape of mathematics, category theory, set theory and discrete math, type systems, abstract data types and data structures, and semiotics, but does not actually introduce type theory. I think that type systems are here being substituted for type theories.

            I think that it would have been nice to flesh out the correspondence between some portion of type systems and category theory or logic. For example, the mantra that “well-typed programs never go wrong” corresponds to the totality of arrows in FinSet, the category whose arrows are functions between finite sets.

            I appreciate the mention of object-free category theory, although I wish that it would have been made more precise. Once we reduce objects down to mere labels which ensure that arrows are well-typed when they are composed, then identity arrows can fully represent their source/destination labels, and we can simplify the axioms accordingly.

            I wish that more attention would have been given to ideas like row polymorphism (“duck typing” in the video) for statically-typed languages, although a lot of the same ground is covered in the discussion on typeclasses. The presentation makes it seem as though languages like OCaml, where objects can have rows of methods and be somewhat incompletely typed, are not statically typed.

            The ontological argument at the end of the video is interesting, and while it has some edges of definite truth to it, I’m hesitant to embrace the central claim that computers simply do what we tell them to do. I dislike the anthropic bias and the idea that humans aren’t meatbag computers, and as a result, I’m not keen on modelling humans as having some special agentive nature which computers lack. Indeed, in object-capability theory, we say that every object is its own agent, capable of planning and pursuing its own goals, and we do this so that we are not surprised when a normally-useless object ends up being powerful due to its position in a plan. Humans are merely special objects which reside outside the heap, just like objects on other computers far away.

            1. 2

              Thank you for this. Your review is both in-depth and useful.

              I literally threw this together in a couple of hours the other morning. I was interested to see if I could scan and summarize the field in a quick and accessible manner.

              When we summarize, we by necessity abbreviate, skim. I was focusing on getting to the common terms we use in CS and how we get to them. I would have loved to have spent more time at the beginning. If I continue to develop this, I’ll give that some serious thought.

              I was especially fascinated by your reaction to my claims at the end regarding semiotics. I spent over 50 pages in my book reaching this conclusion from an entirely-different angle. I find with my nerd buddies that if I make this point too quickly, I get the same response as you gave me. I am not making a human-centric argument in any way, although I can see where it might seem that way. This conclusion would be true of intelligent machines, aliens, perhaps even dolphins and monkeys. It’s a problem/feature of intelligence. It’s not even related to biology.

              If I take the time to explain and prove it, people zone out. (At least most do). It does not seem to be related to technology and programming. However if I go at it too quickly, they misunderstand — and it seems they misunderstand in the same way. Let me know if you’d like to see the more in-depth version. I’d be fascinated to see if there are holes there as well.

              I always love a good critique.

            1. 17

              Nothing was explained. The author seems to think that they’re demystifying the topic, but they’re just showing how to implement monads in Rust. If I were to try to explain monads by posting Monte code implementing monads, then nothing would be revealed or communicated, other than that monads are possible in Monte. But monads are possible in most environments, and are usually only impossible because of language expressivity problems; for example, neither the author’s Rust presentation nor my Monte presentation enforce the monad laws.

              I think the author understands this criticism. When they say:

              monads exist in functional programming to encapsulate state in a way that doesn’t explode functional programming

              I think that they know exactly how wrong this is. Monads are natural structures in both senses of the word; they arise out of endofunctors, which themselves are extremely natural and common structures in programming languages because we often want to repeat actions (which leads to the list monad and non-determinism), pass arguments to subfunctions (which leads to the state monad), or add to a mutable accumulator (which leads to the writer monad). Even more advanced patterns, like inversion of control, lead to continuation-passing monads!

              On the other side of the argument, languages like OCaml have both a modicum of functional programming and also true mutability. Monads are not only not needed to manage state, but actively frowned upon as often overcomplicating otherwise-simple code; the community seems to only use monadic styles when doing asynchronous or networking work, as in Lwt.

              Honestly, this last point could be expanded upon, because I could give an entirely different justification for monads which ignores the entire “functional programming” context: Monads are the simplest abstraction which precisely describes asynchronous promise pipelining behaviors while preventing plan interference. OCaml’s Lwt, Python’s Twisted’s Deferred, Ruby’s EventMachine, and the entire design of Node.js and Deno all revolve around behaviors which turn out to look an awful lot like some sort of asynchronous error-handling monad.

              I found it useful to contrast this article with this one and that one which are also both focused on implementing monads in Rust. However, neither of them are trying to explain what monads are. I think that this artcile would have been sharper if it had focused either on how monads could be implemented in Rust, or instead on the mathematical foundations of monads.

              1. 3

                I do like how you’ve separated out two things that are another regular cause of Monad Tutorialitis: what they are and when they’re the right choice.

                Monads are a way of modeling call-and-response effects into an environment without needing RTS support. So they let you port different kinds of RTS behavior in wherever you want. This leads to lots of fun attributes like failure, nondet, streaming, and, of course, asynchrony.

                If your RTS supports all the effects you need then monads will seem like outrageous overkill. They’re also not the only way to achieve this modeling (you can twist exceptions very far, e.g.) but it turns out that few people are familiar with data flow languages or want JS to be one, so the monad pattern shows up again and again as an implementation of first-order asynchrony semantics.

              1. 3

                For anyone wondering what a complex jq script looks like, check this out:

                https://github.com/b0o/neuron-extras/blob/master/neuron-autoindex.jq

                1. 3

                  This is… unfortunate. People should reach for a different, more readable tool when jq gets beyond a single line. (I say this as someone who likes jq for what it’s good at.)

                  1. 4

                    jq is a good language for working with streaming data. It’s not perfect, but I’m not sure in which language or with which other tool I would have rather written my own complex jq script. The ability to write modules is nice because it allows for factoring out code.

                    1. 2

                      Thanks for this, it’s a way more readable complex script.

                      1. 1

                        Personally, for something that’s intended to be short lived or that has varied structure, I’d probably reach for Python with JMESPath. For something long-lived and performance critical, I’m not sure. I’ve used Clojure and Haskell with success, but today might reach for Rust (and serde) since it’s closer to the top of the heap. I think all of those choices offer more readable and conventional semantics for long scripts.

                        1. 3

                          Critically, though, the jq script (fugly though it may be) will just work without the ecosystem issues the other answers have. “Make sure you have jq installed and run this script” is a lot less likely for a junior to have problems with than “Make sure you have Rust installed and can compile this”.

                          1. 1

                            For now. Is that true of older tools with similar selling points, though? I think about the differences in versions of grep, awk, and sed, and figure similar dialect issues will accrue as time passes.

                  1. 13

                    To recall an analogy from object-capability theory, both a water balloon and a sponge hold water. When either a water balloon or a sponge is stabbed, the integrity of the system is compromised and water leaks out. However, for a water balloon, the stabbing fatally breaches the surface of the balloon and ruptures it, resulting in catastrophic total failure; in contrast, because the sponge has internal structure, the stabbing has almost no effect other than locally around the edges of the knife.

                    When we consider identity, the first thing to remember is that we have already made an epistemic mistake. Quoting from Chip Morningstar’s article, “What Are Capabilities?”:

                    If you’re like most people, the first thing you’re likely to think of is to ask the requestor “who are you?” The fundamental insight of the capabilities paradigm is to recognize that this question is the first step on the road to perdition.

                    Understandably, you instead recommend some sort of prismatic identity, where a single person is correlated to many different accounts. But why have accounts at all? One of the major lessons of capability theory is that it does not matter who invokes a capability; the possession of the capability is the authorization required to invoke it. With emphasis added, I’m going to rephrase a couple of your central conclusions:

                    If this proof verification is done for several accounts on different platforms, it is beyond reasonable doubt that the same person has access to the keys of said accounts.

                    Only you, the holder of the keypair, can add new proofs.

                    You are the author of your proofs and your online identity.

                    The emphasis that you place on ownership and personhood is understandable but incongruent with the actual maths. Cryptographic key material is not unforgeable, merely unguessable.

                    Edit: I am poor at spelling.

                    1. 2

                      I couldn’t agree more. After working on key-based identity for a few years with Scuttlebutt I’m very optimistic that ‘accounts’ aren’t the future.

                    1. 8

                      I think that a massive part of the problem is that most folks working with computers believe that they understand computation far better than they actually do. This leads to mismatched expectations; we expect that the computer will empathize with us and recover the gist of our encoded meanings, mostly because after spending so much time thinking like a computer, we start to disbelieve that the computer can’t think just a little bit like us.

                      I often say that our industry doesn’t really know how to program computers, and I imagine that the author takes umbrage with that sort of comment as encouraging impostorship. Hardly! Instead, I think that we should be much less tolerant of brashly carrying oneself around as if one understands computation.

                      But don’t blindly listen to me, either. I have been rude and contemptuous in the workplace when I have interacted with peers who were assigned power and responsibility but don’t understand the details of the role into which they’ve been placed, and I’m relatively certain that this rudeness comes from a fundamental unwillingness to tolerate my co-workers’ overcomfident self-images.

                      1. 4

                        I like this line of thinking.

                        Perhaps what we think of as “healthy” levels of self-confidence isn’t exactly helpful when dealing with computers. But, due to the context many of us operate within, it is expected that we perform in this way, and we become the role we were told to be, regardless of whether it’s the best fit.

                        The world certainly biases for over-confidence, because it produces action, and action begets Results(tm). I’d even say it’s not wrong for optimizing for this, based on my personal experience. If I’m feeling low, sometimes the best thing to do is to just do something. But just because Something > Nothing doesn’t mean Something is even close to the better thing to do in a situation. And that better thing you can only reach by conscious, painful deliberation that doesn’t involve shipping.

                        1. 2

                          This is really good and important point, which I think deserves some broader framing. It’s not just some abstract concept of “computation” that we computer people don’t understand nearly as well as we’d like to think. We mostly don’t even understand the concrete technologies that we work with every day. How could we? There are huge stacks of them, with every layer full of shifting complex details. What’s essential and what’s merely incidental about any arbitrary sample of this mess? Even the experts don’t often agree! Overall, computing is a very ambitious, heterogeneous, and immature field. But it’s not just computers that we need to understand to do our work: it’s human cognition, economics, hyper-localized “business logic”, a handful of application-specific (or sometimes more general) mathematical and scientific theories… and endless tangles of historical detail.

                          But somehow, we have to get things done, so we blunder bravely forward – or at least, those of us who do stand a better chance of survival. This biases the field towards those who can cope with the uncertainty, either through worry or through denial.

                        1. 4

                          The bit about the doors brings back some memories. I recall two different jobs where I noticed that a security door had been replaced with a door that, like its predecessor, still had an RFID reader and was technically a security barrier, but was trivial to open with a variety of techniques. Perhaps you can imagine this sort of door in your mind: A thick pane of glass, tastefully frosted, with a centimeter-wide gap of air between the door and jamb, and a motion sensor on the secure side for easy unlocking.

                          I have come to think of these doors as emblematic of the transition from engineering to marketing, from startup to entrenched player, from results-driven to people-driven, and from attention to detail to spectacle and splendor. And yes, they’re deeply hypocritical. The purpose of a door isn’t to be transparent or easy to bypass or revealing.

                          1. 2

                            Similar experience here, at a previous startup job, WiFi was shutdown, we had a local IT room because everything was so confidential, no internet access in some rooms, etc… but then you could enter the parking by forcing the door, and open almost every door of the building with a regular kitchen knife…

                            This was so bad that at some point it became a running gag internally.

                          1. 8

                            As a former editor, I’m really happy to see this sort of permanent extension of Wikipedia. Automatic generation of articles or knowledge boxes based on Wikidata is just the sort of ability that is needed to continue scaling the encyclopedic effort. Mere data is so much more inclusive than individual hand-written articles, in the inclusionist/deletionist paradigm.

                            The technical paper is pretty good and addresses every concern one might have about the problem. The actual implementation plan looks much more daunting, though.

                            1. 1

                              Mere data is so much more inclusive than individual hand-written articles

                              That statement is at odds with several statements in the whitepaper, such as

                              contributing to Abstract Wikipedia will possibly have a higher bar than contributing to most Wikipedia projects

                              and

                              compared to natural language the expressivity of Wikidata is extremely limited

                              But, more to the point, “mere data” has no meaning in itself. Gathering it is the easiest part of the problem.

                              I find the whitepaper almost humorous in how thoroughly it hand-waves its way through a big swath of really thorny problems, while studiously ignoring many others. Not to mention the long history of failed attempts at this particular pipe dream. The first bullet point in the “risks” section says

                              Leibniz was neither the first nor the last, but probably the most well-known proponent of a universal language,which he called Characteristica universalis[74]. Umberto Eco wrote an entire book describing the many failures towards this goal [24]. Abstract Wikipedia is indeed firmly within this tradition, and in preparation fort his project we studied numerous predecessors

                              … but, you just have to take his word for it, because he doesn’t actually cite any real prior work, let alone do any analysis.

                              I’d be willing to take 10:1 odds on this project going nowhere. I sure hope the Wikimedia Foundation doesn’t waste any money on it. They should instead invest in cleaning up their (absolutely terrible) code base.

                              1. 2

                                Most Wikipedia projects are the single-language encyclopedias, and most of those have relatively low bars to contribution. The proposal author works on Croatian Wikipedia, and part of the motivation is to gain lots of articles in Croatian with the same structure and details as in English. This leads to the imagined bar for quality; we might imagine that Abstract Wikipedia presentations are capable of generating English Wikipedia Featured Articles, which have one of the highest bars across all of the Foundation’s projects.

                                When I say that this move is inclusive, I mean explicitly that the number of articles that can be maintained this way, generalized and template-generated, is strictly greater than the number that can be handwritten. Additionally, some articles that might have been too expensive to write by hand, due to the long-tail effect, might be easy to generate automatically; this could be the future of stubbed articles.

                                Wikidata is highly structured. While any single datum is not meaningful, the structure between data can be recovered. If we look at a particularly famous datum representing a famous author, we can see that each piece of associated data is structured as a span which connects the author, by each resepctive relation, to each respective associated datum. Also, citation is built directly into the structure.

                                This structure of spans gives a well-studied category which generalizes Rel, the category of sets and relations. (Details are at the nCat page on spans.) The upshot is that we can do all of our ordinary relational algebra on Wikidata. While I’m not a fan of SPARQL, there is a well-documented SPARQL interface to Wikidata.

                                Hm, ten-to-one is about 91%, which is just over one nine. I would take one nine’s belief that this effort, or something extremely similar with the same design and goals, replaces the knowledge-box construction that’s currently used across Wikipedia. Those boxes are built from MediaWiki templates, which is nice for letting ordinary editors work on them, but perhaps they could start to be built from Wikidata queries instead. Now, as to replacing actual article generation on any single-language Wikipedia, I’d say less than one nine. This is a grand design that needs to achieve many intermediate steps in order to even possibly succeed.

                                I think we all know that MediaWiki will never be replaced, and also that nothing short of a full rewrite can fix its architectural issues. I don’t like it either.

                            1. 2

                              author here, if anyone has questions

                              1. 1

                                Thank you for bringing evidence and rigor to randomizers. It is refreshing, after listening to so many commentators saying, “well, there’s a fifty-fifty chance it’s in one of those two places,” to have actual numbers.

                                I am curious about conditional placement. Are there circumstances in which the Wrecked Ship is likely to pay off, based on the items found during the first trip through Green Brinstar? Similarly, what are the odds that the “meme route,” where early Power Bombs are used to go directly from Morphing Ball’s room into Green Hill Zone, is any good when it’s available?

                                In terms of advanced strategic routing, particularly “hellruns” where heated rooms in Norfair are traversed without the Varia Suit, is there any route that is likelier to pick up Energy Tanks?

                                1. 2

                                  if you want some actual answers to those questions, i’d have to figure out a good way to turn them into queries (my attempts weren’t productive). and of course it would depend on which logic set you want to analyse. in the SMRAT logic a lot of items would put Wrecked Ship in play: power bombs + any of Gravity Suit, Grapple Beam, or Space Jump. i’ve seen crazy seeds in where both suits were in WS. another runner was interested in breaking down some of those stats based on the items in Green Brin but i don’t know if they had much luck.

                                  during my runs in the tournament i gambled on meme route a lot because i figured i could check Pink/Green Brinstar a little earlier, hit Gauntlet, and then go down Red Tower and never touch Wrecked Ship. based on the data i did crunch that seemed optimal, but in practice you often run out of bombs in Pink and then have to skip locations or leave to refill. that scenario definitely needed more data.

                                  as for hellruns, i never considered looking at them because i thought it was always better to make one trip to Norfair and go all the way down to Ridley, instead of having to leave and come back with access into Lower Norfair (at least, given my own shoddy skills). players comfortable with Reverse Boss Order could absolutely benefit from knowledge of likely energy tanks though. i just ran my distribution query for tanks by area since it was easy to modify:

                                           area          | tank
                                  -----------------------+------
                                   Kraid                 | 1356
                                   Sand Pits             | 1488
                                   Business Center       | 1232
                                   Brinstar Reserve      | 1463
                                   Gauntlet              | 1520
                                   Pink Brinstar         | 1729
                                   Green Brinstar        | 1814
                                   Blue Brinstar Cleanup | 1702
                                   Ice                   | 1645
                                   Golden Torizo         | 1782
                                   Lower Norfair         | 1811
                                   Inner Maridia         | 1665
                                   Lower Wrecked Ship    | 1421
                                   Red Tower             | 1341
                                   Sphere 0              | 1162
                                   Bubble Mountain       | 1509
                                   Crocomire             | 1580
                                   Outer Maridia         | 1460
                                   Upper Wrecked Ship    | 1522
                                  

                                  overall tanks seem to be distributed fairly evenly, LN actually being on the high end making a suitless play more likely to be survivable by getting a full heal while you’re down there. however i think players going for that strategy would probably just rely on a lot of Crystal Flashes, which depends on the distribution of ammo, which the randomizer debug output doesn’t capture by default. most non-Varia suit items can be found in Upper Norfair but given how many are required to get into Lower i think the gamble wouldn’t be wothwhile unless you just wanted to make the run more interesting.

                                  if anyone is interested in the data and knows how to construct queries that are like “if this then where’s that” i’d be happy to run them on the dataset, or if you need help reconstructing it from my lazy instructions. you could even make a new dataset for different settings like those used by the Super Metroid Randomizer League, which would produce very different statistics.

                                2. 1

                                  How long until someone makes a randomizer that leaves all the items in their usual places but shuffles doors/blocks/bosses to make the key items for each area different? :D

                                  1. 2

                                    I made a manual hack to test that out of curiosity, in some parts of the game it was a lot of work to make a room different but still recognizeable. the locations of some items makes it hard to do much different with the surrounding rooms. there are however already randomizer options to shuffle the bosses and doors at the area transitions. if players really wanted to push the boundaries for fresh content, we might one day see completely brand new generated maps to explore akin to roguelikes.

                                1. 2

                                  This empirical data matches my intuitions as well as the very basic numbers I’ve crunched from a more analytical approach to the logic. It also helps explain why popular randomizer configurations forbid either suit from appearing in the Gauntlet.

                                  The numbers suggest that Hi Jump Boots and Charge Beam really can be just about anywhere, including irritatingly deep into Lower Norfair or Maridia. This helps validate the strategy of not searching for Charge Beam and instead deliberately picking up spare Missile packs during the early game.

                                  1. 10

                                    You guys are fast :) I published an announcement an hour ago which gives more context and design insights:

                                    https://drewdevault.com/2020/06/21/BARE-message-encoding.html

                                    1. 9

                                      Before you commit to reinventing Capn Proto, including copying the schema language, I want to let you know that the Capn Proto build tools do not mandate using a C++ API, nor using the libkj event loop. The process for writing new plugins is documented, but since you’re using existing languages, you could use capnpy for Python, capnproto2 for Go, or capnp for Rust, none of which use the C++ API. Instead, Capn Proto’s compiler is designed to come apart into subprocesses naturally, with each subprocess using Capn Proto itself to request schema compilation. The bootstrapping phase takes effort, but it is possible and people have already done a lot of the hard work already for many popular languages.

                                      I also would wonder whether you’re prepared to take on the security burden. Capn Proto’s upstream maintainers set an extremely high bar, and their vulnerability writeups have garnered serious praise for their quality and depth.

                                      Finally, although it is not mandatory, Capn Proto defines a custom compression algorithm, “packing”, intended specifically to address your concern that fixed-width and aligned data types are wasteful. The authors also recommend LZ4 or zlib for when packing isn’t sufficient.

                                      In summary, I think that your:

                                      $ go run git.sr.ht/~sircmpwn/go-bare/cmd/gen -p models schema.bare models/gen.go
                                      

                                      Could be:

                                      $ capnp compile -I$GOPATH/src/zombiezen.com/go/capnproto2/std -ogo schema.capnp > models/gen.go
                                      

                                      With the package information included in the Capn Proto schema file.

                                      1. 6

                                        Like I said in the article, I did evaluate Cap’n Proto, and concluded it was not the right fit.

                                        Before you commit to reinventing Capn Proto, including copying the schema language

                                        The schema language is not copied from Cap’n Proto, and frankly I don’t understand how you would make this assumption. The two schema languages are very different. Unless you mean the idea of having a schema language at all, in which case you are clearly ignorant of the alternatives and their history.

                                        I want to let you know that the Capn Proto build tools do not mandate using a C++ API

                                        C++ is, in fact, required:

                                        $ capnp compile -I$GOPATH/src/zombiezen.com/go/capnproto2/std -ogo schema.capnp > models/gen.go here -^

                                        I don’t evaluate technology in a vacuum. The total complexity of the implementation, including code I didn’t have to write myself, is part of my evaluation. There is no attempt at a specification or any motions to support third-party implementations from Cap’n Proto.

                                        The bootstrapping phase takes effort

                                        It seems that the typical amount of effort required to make a complete BARE implementation is 3-5 days. I also looked at the C++ frontend for Cap’n Proto: 3,153 lines of support code for C++. The entire Go implemenation of BARE is 3,629 lines of code, including marshaling & unmarshaling, parsing, code generation, exhaustive tests, example code, and generated code, and it has no external dependencies (well, getopt for the generator CLI, and a assertion framework for the tests).

                                        I also would wonder whether you’re prepared to take on the security burden. Capn Proto’s upstream maintainers set an extremely high bar, and their vulnerability writeups have garnered serious praise for their quality and depth.

                                        I find it ironic that the security vulnerability you use as an example is the result of C++ programmers doing C++ programmer things, namely over-use of templates (any use of templates being an over-use), and works in favor of my argument that Cap’n Proto’s programming culture is symptomatic of the broken culture that values tools like C++. An ounce of prevention is worth a pound of cure, and in BARE’s case both the spec and implementation are 10x simpler - and simpler always means more secure.

                                        Finally, although it is not mandatory, Capn Proto defines a custom compression algorithm

                                        This also argues against you. The message encoding should have nothing to do with compression, this just increases the complexity of both the specification and implementation (by the way, I keep refering to “the specification”, but in reality such a thing doesn’t exist - this fact also works against Cap’n Proto’s favor). The specification cannot be “complete”, either - the compression techniques will become obsolete over time, and the specification will have to be revised to accomodate.

                                        In general, BARE messages tend to have the same entropy as the underlying information they represent, because the BARE encoding is barely more than the literal data directly copied to the message. Unlike Cap’n Proto, which has alignment and fixed message size wasting space (this is done in exchange for performance, unless you’re decompressing the messages, in which case the performance improvement is totally lost), BARE messages stand to save very little from encoding-aware compression. For this and other reasons besides, compression is out of scope for BARE. Instead, you should feed BARE messages to your favorite compression algorithm yourself, such as zstd, and you will get results comparible to compressing the underlying information.

                                        Not having compression in-scope simplifies the implementation and specification. Reckless lack of consideration for scope is a major problem which ruled out many of the alternatives I explored before working on BARE.

                                        1. 2

                                          Your evaluation can be quoted in full, as it is barely a paragraph, with only half of its fragments qualifying as sentences:

                                          Cap’n Proto: fixed width, alignment, and so on — good for performance, bad for message size. Too complex. RPC support is also undesirable for this use-case. I also passionately hate C++ and I cannot in good faith consider something which makes it their primary target.

                                          BARE is not simpler; feel free to define a metric before revisiting that point. RPC support is optional. We’ve covered everything else.

                                          Some of your statements are either wrong or questionable:

                                          The two schema languages are very different.

                                          Really? All of these declarative schema languages look very similar to me. Which one were you trying to reinvent?

                                          There is no attempt at a specification or any motions to support third-party implementations from Cap’n Proto.

                                          Baldly contradicted, and I will continue to link to evidence as needed.

                                          in BARE’s case both the spec and implementation are 10x simpler

                                          This needs a metric.

                                          I, too, passionately hate C++. And yet, somehow, nobody really cares about my feelings. Capn Proto does not make C++ the primary target, nor does it mandate the C++ API. You are pointing at a subprocess invoked via shell, which is a common linguafranca across many different languages. There is no C++ linkage, no C++ name mangling, and no C++ templates. The reference implementation makes only one request, and it is a polite one: For interoperability, use the same struct offsets every time, and preferably use exactly the same algorithm as the original.

                                          Capn Proto has a complete encoding specification. Only the struct offsets are computed, but they are not necessary for deserializing a message, only for interpreting it according to certain names and schemata. Indeed, when I bootstrapped Capn Proto support for Monte, I started by writing a deserializer for unnamed buffers and messages, and only later added in support for schemata.

                                          On a more personal note, attacks and rudeness like:

                                          frankly I don’t understand how you would make this assumption

                                          in which case you are clearly ignorant of the alternatives and their history

                                          Are how you maintain your reputation for toxicity. I don’t believe in moderating people unkind, but I do think that you could stand to look at your words and how you escalated this conversation.

                                          1. 8

                                            Your evaluation can be quoted in full, as it is barely a paragraph, with only half of its fragments qualifying as sentences

                                            Yes, because it’s sharing a list of bullet points with 4 other alternatives in an article whose purpose isn’t to provide a detailed critique of Cap’n Proto.

                                            BARE is not simpler; feel free to define a metric before revisiting that point. RPC support is optional.

                                            Everything is optional, I could just choose not to use a subset of either system’s primitives and call it “less complicated”. Your argument doesn’t make any sense. A complete, conformant implementation of BARE is simpler than one of Cap’n Proto. Here’s one metric: the encoding spec you linked to is 3,908 words. The BARE spec is 1,430, and the BARE spec also includes a grammar for the schema DSL.

                                            Really? All of these declarative schema languages look very similar to me. Which one were you trying to reinvent?

                                            Give me a break. What they have in common appears to be the enum keyword, convention of PascalCase for user-defined types, the use of “:” to separate field names from types, and the use of braces “{ }” for structs. There are dozens of other languages that we share all of these traits with. In every other respect they are quite different:

                                            • BARE allows you to make any kind of user-defined type, not just structs
                                            • Cap’n Proto has explicit field offsets, BARE lacks them
                                            • Cap’n Proto has semicolons, BARE lacks them
                                            • Cap’n Proto has semicolons, BARE lacks them
                                            • BARE has a different syntax for arrays (lists in cap’n terms)
                                            • BARE has fixed arrays, maps, and first-class tagged union types, and syntax for each
                                            • Cap’n Proto has default values, and BARE’s DSL can’t describe values at all
                                            • Cap’n Proto has the concept of groups, which BARE lacks
                                            • BARE has no RPC and therefore no syntax for it

                                            I find your acusation of copying the schema completely unfounded and extremely rude.

                                            There is no attempt at a specification or any motions to support third-party implementations from Cap’n Proto.

                                            Baldly contradicted, and I will continue to link to evidence as needed.

                                            I will correct myself: there is a specification of the encoding. There is no specification of the DSL. There is documentation, but documentation is not a specification.

                                            They support third-party integrations, not implementations. This is an important difference.

                                            in BARE’s case both the spec and implementation are 10x simpler

                                            This needs a metric.

                                            See my first comment for the implementation size and this comment for the spec size.

                                            You are pointing at a subprocess invoked via shell

                                            A subprocess written in C++. This puts C++ in your dependency tree. Just because you don’t link to it with the C++ ABI doesn’t mean it’s not a dependency. There are multiple ways to skin^Wdepend on a cat.

                                            On a more personal note, attacks and rudeness like:

                                            frankly I don’t understand how you would make this assumption

                                            in which case you are clearly ignorant of the alternatives and their history

                                            Are how you maintain your reputation for toxicity. I don’t believe in moderating people unkind, but I do think that you could stand to look at your words and how you escalated this conversation.

                                            The use of the word “frankly” and using ignorance to explain unfounded arguments is not rude. Telling me that I am maintaining a reputation for toxicity, however, is extremely rude. You are disparaging my work and my character with baseless arguments and false comparisons.

                                            We get it, I don’t like your sacred cow. It is entirely possible for me to have evaluated it and determined it did not meet my needs, and also valid for me to determine that I simply don’t like it, and also valid to conclude that the idea is poorly realized by its implementation. In this case I came to all of these conclusions. The fact of the matter is that your protocol is not perfectly suited to all use-cases and neither is your implementation, and you aren’t entitled to praise for it.

                                            1. 3

                                              You’re kind of being rude here. Also, if you know that the metric hasn’t been defined then how did you decide that BARE is not more simple?

                                              1. 7

                                                They both are, in my opinion. I’d really prefer not see conversations in this tone on lobste.rs (speaking for myself only, of course).

                                                1. 4

                                                  Dunno. I think @ddevault is just defending their post… Doesn’t seem particularly rude to me

                                                2. 2

                                                  I’m trying. I am not a nice person and it’s been nearly a decade of trying to have conversations with the author. But sure, I do not mind receiving the unkind vote. This isn’t the first time that a basic technical discussion has been horribly derailed by their attitude, and not the first time that I’ve been told that I’m part of the problem because I refuse to go along with their framing.

                                                  I did not claim that Capn Proto is simpler than BARE. I instead rejected the idea that there is a natural and obvious metric by which to claim that one is simpler than the other. This is a frustrating ambiguity in English; the word “not” can mean three different things and there is no way to disambiguate other than to write an explanatory paragraph.

                                                  No suitable metric is given. One metric, lines of code, is used; however, lines of code between two different host and target languages are apples-to-oranges. To make it apples-to-apples, the comparison would have to be either between BARE and Capn Proto complete stacks in a single language (presumably C++), or between two Capn Proto implementations in two different languages. Either way, the appropriate comparison hasn’t been drawn.

                                            2. 1

                                              I think the appeal of BARE is that it has a low threshold of entry (like json, msgpack, etc.) for anyone willing to write a code-generator for it. On the one hand, capnproto’s schema language seems cleaner and more expressive; the problem is that the whole encoding format, albeit efficient, is very complex in the name of allowing zero-copy access. There should be room for a language like BARE, more or less in the same niche protobuf occupies currently (fast and binary but not insanely complicated).

                                              Sadly it looks like BARE will not get sum types/general tagged unions, and cannot represent them (except for option). I got fed up in this thread and failed to keep as civil a tone as I wished I had, but anti-intellectualism irritates me.

                                              1. -4

                                                BARE has sum types aka tagged unions. What it does not have is zero-bit types, which often change the way you use tagged unions. After an hour of back and forth it became apparent that c-cube lacks the “intellectual” capacity to understand this difference.

                                                1. 6

                                                  And, this is the main reason I actively avoid working on any of the projects he’s involved in. “Feedback is welcome”, the page says, apparently until it gets annoying or the author has to deal with someone stubborn who has a different way of looking at the world, at which point you get called a moron in public.

                                                  1. 4

                                                    The back and forth made me sad and annoyed too. I think the terminology is confusing and ambiguous, because we have different ideas on what “sum types” precisely entails and could not find the words to carry the difference over short messages.

                                                    The “anti-intellectualism” was also not civil, but so are the straight insults towards functional programming and people who like it.

                                                    1. 0

                                                      Your “different” idea on what “sum types” precisely entails is not just different, but factually incorrect. You approached the discussion with a flawed understanding of the type system you espouse and made incorrect assumptions about the limitations of BARE’s type system. This is the main source of frustration. I appreciate that you were willing to evaluate the specification and share your feedback, but you do not fully understand the problem that you are trying to communicate about.

                                                      Regardless, after more thought to the potential use-cases and the best implementation approach, I have added a void type to the specification, which should enable the types you want to construct with BARE messages.

                                              2. 3

                                                The choice on schema upgrade is interesting (or strange). That means when upgrading to MessageV2, you need to write a manual translation between MessageV1 and MessageV2?

                                                What about incremental updates (schema evolution, i.e. adding and remove fields in a struct)? Protobuf and Flatbuffers both provides an evoution path for these while it seems BARE and Cap’n’Proto elected (not exactly sure about BARE story other than your compatibility section, but definitely Cap’n’Proto) to not support this.

                                                1. 1

                                                  What about incremental updates (schema evolution, i.e. adding and remove fields in a struct)?

                                                  The length is not encoded into a BARE message, so there’s no way to determine when one message stops and the next begins without knowing the schema of the message in advance. If you append fields, you won’t be able to tell that they’re not part of the next message.

                                                  But, nothing is stopping you from adding your own length to messages, either through context (Content-Length) or explicitly adding a length prefix before reading out the struct (the Go implementation provides tools you can use to read primitives yourself for things like this, and I would hope that other languages would do so). So if you prefer that approach, then by all means. Another alternate approach could express a version out of band, like X-Message-Version.

                                                  The tagged union versioning approach works in all contexts in which BARE messages work, so it’s recommended as a universal solution. But if you prefer a different option, you’re welcome to establish the necessary context to support it out-of-band and go with that.

                                                  1. 1

                                                    Thanks. Yeah, read the doc a bit more and realized that the struct just a concatenation of underlying fields. Unlike Protobuf or Flatbuffers, it doesn’t carry any offsets, therefore, adding / removing fields requires either having a brand-new type in the union types, or some other means (depending on the language) to carry the old schema somewhere (out-of-band).

                                                    It is interesting to observe how these are solved. I am more comfortable with the built-in approach by Protobuf / Flatbuffers but I think different solutions fit different usage cases and having additional redirection may be undesirable for many cases.

                                              1. 0

                                                As someone who is in V’s Discord every day being constantly blown away at the progress being made, I am shocked at the level of dishonesty that this strangely anti-V hit piece achieves.

                                                In particular, the degree of cherry-picking (and often then still misrepresenting) a few facts in order to make V appear in the worst possible light is truly breathtaking.

                                                She cites vlang.io saying

                                                V can be bootstrapped in under a second by compiling its code translated to C with a simple

                                                cc v.c

                                                No libraries or dependencies needed.

                                                then argues against it, preposterously, by saying in part,

                                                Git is a dependency, which means perl is a dependency, which means a shell is a dependency, which means glibc is a dependency, which means that a lot of other things (including posix threads) are also dependencies. …

                                                Downloading a .c source file requires git? Does this person know what a “dependency” is? Should JavaScript developers include depending upon the laws of physics in package.json?

                                                Amusingly, the documentation still claims that memory management is both a work in progress and has perfect accuracy for cleaning up things at compile time.

                                                No, the documentation correctly says that memory management is a work in progress, and also that, once completed, will clean up after itself in much the way that Rust does.

                                                An Honest Depiction of Progress

                                                Here are the combined release notes from all of the V releases since December:

                                                Release 0.1.23:

                                                - [Direct x64 machine code generation](https://github.com/vlang/v/issues/2849). Hello world being built in 3 milliseconds.
                                                - Bare metal support via the `-freestanding` flag, allowing to build programs without linking to libc.
                                                - Prebuilt V packages for Linux, macOS, and Windows.
                                                - `string.index()` now returns `?int` instead of `int/-1`.
                                                - Lots of fixes in Generics.
                                                - vweb framework for developing web applications is back.
                                                - Vorum, the forum/blogging software written in V/vweb, can now be compiled and has been added to CI.
                                                - REPL, `v up` have been split up into separate applications to keep the core V compiler small.
                                                - V now enforces short enum syntax (`.green` instead of `Color.green`) when it's enough.
                                                - V UI for macOS.
                                                - Interfaces have been rewritten. `[]interface` support.
                                                - `os.cp()` for copying files and directores.
                                                - Additional compile-time flags: `$if clang, msvc, mingw, x32, x64, big_endian, little_endian {`.
                                                - All C functions now have to be declared, all missing C functions have been defined.
                                                - Global variables (only with the `--enable-globals` flag) for low level applications like kernels and drivers.
                                                - Nothing can be cast to bool (previously code like `if bool(1) {` worked.
                                                - `<<` and `>>` now work with all integer types.
                                                - V detects Cygwin and shows an error. (V supports Windows natively)
                                                - Improved type checking of some operators (`%, |, &` etc).
                                                - Windows 7 support.
                                                - `println(true)` now prints `true` instead of `1`.
                                                - `os.exec()` now uses `CreateProcess` on Windows.
                                                - fast.vlang.io website for monitoring the performance of V after every commit.
                                                - On Windows Visual Studio is now used automatically if GCC is not installed.
                                                - vfmt!
                                                - Lots of cleaning up in the compiler code.
                                                - Multi-level pointers in unsafe code (`****int`).
                                                - MSVC backtrace.
                                                - `$if os {` blocks are now skipped on a different OS.
                                                - C string literals (`c'hello'`).
                                                - AlpineLinux/musl fixes + added to CI.
                                                - Inline assembly.
                                                - Clipboard module (Windows, macOS, X).
                                                - `foo()?` syntax for error propagation.
                                                - Docs have been migrated from HTML to `doc/docs.md`.
                                                - `eventbus` module.
                                                - Haiku OS support.
                                                - `malloc/free` on bare metal.
                                                - `utf8` helper functions (`to_lower()`, `to_upper()`, etc).
                                                - Optimization of `for c in str {`.
                                                - `string/array.left/right/slice/substr` were removed (`[a..b]` slicing syntax should be used instead).
                                                

                                                Release 0.1.24:

                                                - A new parser/generator built on top of an AST that simplifies code greatly and allows to implement new
                                                  backends much faster.
                                                - Sum types (`type Expr = IfExpr | MatchExpr | IntegerLiteral`).
                                                - B-tree map (sped up the V compiler by ~10%).
                                                - `v fmt -w`.
                                                - The entire code base has been formatted with vfmt.
                                                - Generic structs.
                                                - SDL module.
                                                - Arrays of pointers.
                                                - os: `is_link()`, `is_dir()`, `exists()`.
                                                - Ranging through fixed size arrays.
                                                - Lots of fixes in ORM and vweb.
                                                - The first tutorial: [building a simple web application with vweb](https://github.com/vlang/v/blob/master/tutorials/building-a-simple-web-blog-with-vweb.md).
                                                - Match expressions now must be exhaustive.
                                                - freestanding: `malloc()`/`free()`.
                                                - `++` is now required instead of `+= 1` for consistency.
                                                - Interpolated strings now allow function calls: `println('val = $get_val()')`.
                                                - `string.replace_each([])` for an efficient replacement of multiple values.
                                                - More utf8 helper functions.
                                                - `-prealloc` option for block allocations.
                                                - `type` aliases.
                                                - Running `v` with an unknown command will result in an error.
                                                - `atof` implementation in pure V.
                                                - Enums can now have negative values.
                                                - New `filepath` module.
                                                - `math.factorial`.
                                                - `ftp` module.
                                                - New syntax for casting: `val as Type`.
                                                - Fewer libc functions used (soon V will have no dependency on libc).
                                                

                                                Release 0.1.27:

                                                - `vfmt` has been re-written from scratch using the new AST parser. It's much faster, cleaner, and can format
                                                files with compilation errors.
                                                - `strconv`, `sprintf`, and `printf` in native V, without any libc calls.
                                                - Interfaces are now a lot more stable and have all expected features.
                                                - Lots of x64 backend improvements: function calls, if expressions, for loops, local variables.
                                                - `map()` and `filter()` methods can now be chained.
                                                - New `[]int{cap:cap, len:len}` syntax for initializing array length and capacity.
                                                - New `is` keyword for checking the type of sum types and interfaces.
                                                - `as` can now be used to cast interfaces and sum types.
                                                - Profiling with `-profile`. Prints a nice table with detailed information about every single function call:
                                                number of calls, average time per call, total time per function.
                                                - `import(xxx)` syntax has been removed in favor of `import xxx` for simplicity and greppability.
                                                - Lots of fixes and improvements in the type checker.
                                                - `time.StopWatch`
                                                - `dl` module for dynamic loading.
                                                - Automatic `str()` method generation for every single type, including all arrays and fixed size arrays.
                                                - Short struct initialization syntax for imitating named function args: `foo(bar:0, baz:1)`.
                                                - New operator `!in`.
                                                - Performance improvements in critical parts of the builtin data structures (array, map).
                                                - High order functions improvements (functions can now be returned etc).
                                                - Anonymous functions that can be defined inside other functions.
                                                - Built-in JSON module is back.
                                                - Closures.
                                                - Lots and lots of new tests added, including output tests that test error messages.
                                                - Multiple errors are now printed, the compiler no longer stops after the first error.
                                                - The new JS backend using the AST parser (almost complete).
                                                - Variadic functions.
                                                - `net.websocket` module (early stage).
                                                - `vlib` is now memory leak free, lots of `autofree` improvements.
                                                - Simplified and cleaned up `cmd/v`, `v.builder`.
                                                - V UI was updated to work with the new backend.
                                                

                                                After she COMPLETELY ignores the MASSIVE progress mademore than 3000 commits worth from a brilliant and fiercely dedicated team – and judges the current state of V based exclusively on misunderstandings, nitpicks, and on its memory management status after acknowledging that it’s not done yet and that the language is in an alpha state, she snarkily ends with:

                                                Overall, V looks like it is making about as much progress as I had figured it would.

                                                This is almost as bad as the quote she ended with in her previous post on V:

                                                Don’t ever, ever try to lie to the Internet, because they will catch you. …


                                                Honesty, Please!

                                                If you want to know how well V is actually progressing, try it yourself, check out the Discord, look on GitHub, but whatever you do, do not focus on ignorant, dishonest, cherry-picked commentary from haters; that doesn’t serve anyone well, and is incredibly unfair to those who are pouring themselves into this important project.


                                                The Brilliance of V

                                                After my 11 years of programming, including 9.5 of programming in Go (which is the most similar language to V), I consider V to easily be the best-designed programming language that exists.

                                                Yes, it’s learned a lot from Go and C, and maybe Lisp people prefer Lisp, but V successfully combines the simplicity of Go, the programmer ergonomics of Python, the speed C, and almost as many safety guarantees as Rust (once V has finished implementing these latter aspects, of course!).

                                                What I thought would take the V team 5 years to implement has taken less than 1 year. Alex (V’s creator) thought it would take even less time, and now he’s being unfairly raked over the coals for setting extremely ambitious timelines while the same naysayers and bullies ignore everything that has been done.


                                                V Resources

                                                Website (including code examples): https://vlang.io/

                                                GitHub: https://github.com/vlang/v

                                                Wiki page explaining why C is used as in intermediate representation rather than LLVM (another brilliant move that allows V to build on the shoulders of giants and avoid reinventing the wheel in order to bootstrap a new language, but a move that is misunderstood and absurdly used to argue against V for doing things differently/better): https://github.com/vlang/v/wiki/On-the-benefits-of-using-C-as-a-language-backend

                                                1. 25

                                                  I understand that you have strong feelings for your language of choice. Nonetheless, language designers are not entitled to a community, nor are they entitled to shelter from criticism. Indeed, one of the most important parts of programming language design is rejecting new languages based on showstoppingly-unacceptable design choices.

                                                  V does not offer any compelling design choices. Its advertised features can be sorted into libraries, compiler/toolchain offerings, and roughly the level of safety advertised in the 80s when memory-safety was still controversial. Just like Go, V has not learned many lessons, and refuses to offer programmers a more interesting way to express themselves. Even if V were literally Go but better, this would be sufficient to damn it.

                                                  I understand that you might not like it when people point out that the release dates keep slipping; I think it’s curious that you are willing to link to V’s wiki and source code, but not to bumping release dates.

                                                  As a language designer, I think that it is important to not advertise what you don’t yet have written. Monte has had one release, a developer preview, and we are intending to complete another iteration of bootstrapping before even considering another release. We know that almost every feature that typical end users will want is not yet written, and so we are not loudly advertising our offering as usable for everyday general-purpose programming, regardless of how much everyday general-purpose programming I or anybody else actually achieves with it.

                                                  I consider V to easily be the best-designed programming language that exists.

                                                  What’s your favorite ML? I have lately played with OCaml. There are entire universes of language designs which I suspect that you have yet to explore.

                                                  1. -4

                                                    Just like Go, V has not learned many lessons, and refuses to offer programmers a more interesting way to express themselves.

                                                    FP diehards will never understand why Go has been so wildly successful – and V will be even more successful than Go.

                                                    V is like Go but fixes all ~10 things wrong with it, providing a lot more flexibility due to its generic functions, generic structs, generic channels (still in the works), sum types, and TypeScript-style interfaces (also still partially in the works).

                                                    Plus there’s the raw speed factor; V is translated to C before being compiled to machine code, cleverly piggybacking on decades of speed optimizations made by gcc/clang/tcc/etc.

                                                    The simplicity of Go or Python + almost as much safety as Rust + almost exactly as much speed as C + a flexible type system + familiar syntax == a winning combination, I insist!

                                                    1. 16

                                                      The simplicity of Go or Python + almost as much safety as Rust + almost exactly as much speed as C + a flexible type system + familiar syntax == a winning combination, I insist!

                                                      Except all of these are “some time in the future”, and widely incompatible with one another. There’s nothing to support any of these claims. What’s the design for “almost as much safety as rust” (without GC, of course)? The whole thing only just got an AST, and we’re supposed to believe it’s going to be revolutionary? There’s been a lot of grand promises with release dates being pushed back repeatedly, but nothing specific about how the promises will actually be achieved. Making a good language is hard, it takes years (if not decades), and you can’t just magically make something both simple, fast, safe, gc-free, etc. in a field where it’s known that some tradeoffs are inevitable.

                                                      1. -3

                                                        Except all of these are “some time in the future”, and widely incompatible with one another.

                                                        Nope, totally wrong. The simplicity is there, the speed is there, the flexible type system is there, and the familiar syntax is there. A safe subset of C is generated then compiled but not all the safety guarantees are implemented yet.

                                                        There’s been a lot of grand promises with release dates being pushed back repeatedly

                                                        V is the first software project in history to be finished later than originally intended ;-).

                                                        The whole thing only just got an AST

                                                        Completely false; V has always had an AST. The AST-related thing that’s new is representing the generated C code as an AST before outputting it.

                                                        … you can’t just magically make something both simple, fast, safe, gc-free, etc. in a field where it’s known that some tradeoffs are inevitable.

                                                        The big “a-ha” moment for me was this: I now realize that I had falsely assumed that just because prior languages took certain trade-offs that it was impossible to check all these boxes at once. But I was wrong.

                                                        The only inherent tension between any of the things I listed is between simplicity and flexibility. But as I said in the comment you’re replying to,

                                                        V is like Go but fixes all ~10 things wrong with it, providing a lot more flexibility due to its generic functions, generic structs, generic channels (still in the works), sum types, and TypeScript-style interfaces (also still partially in the works).

                                                        The limiting factor is not some innate impossibility of making a language that is simple, fast, and safe. The limiting factor is creativity. But V has learned much from Go, Rust, Python, and other languages, and has unique insights of its own (like its error handling!).

                                                        New things are, in fact, possible… and spelled out in detail on the website and in the docs, in this case. See for yourself: https://github.com/vlang/v/blob/master/doc/docs.md .

                                                        1. 9

                                                          the flexible type system is there

                                                          hydraz below convincingly demonstrated that if function calls have generic types, type is not checked at all(!) in current V. How can you say type system is “there”? I guess it is “there” in terms of code generation, but if you are not checking types, saying type system is there is at best deceptive.

                                                          1. -4

                                                            How can you say type system is “there”?

                                                            …because there are types you can define and instantiate and do all the usual things that programming languages let you do with types…

                                                            hydraz said,

                                                            type errors for parameters in functions with a slapped on them are still silently ignored…

                                                            Silently ignored? If you use a generic type in a way that’s invalid then the program won’t compile (yes, during the C -> machine code step).

                                                            1. 9

                                                              I think you need to read my comments - and indeed, the compiler code that I linked - again. V has roughly no type system at all. The function, foo, that I wrote, isn’t generic!

                                                              • It does have a <T>, but there’s nothing to infer that T from (This should be a type error. It isn’t)
                                                              • It takes a string, but I can give it an int, and this should be an error, but the compiler has code specifically for silently ignoring these errors.
                                                          2. 8

                                                            spelled out in detail

                                                            Let’s see memory management: there’s no explanation, just claims there are (or will be, it’s wip after all) no leaks, no gc, not refcounting, but also no manual memory management (it’s hardly leak free, after all, even in rust). What magic justifies that? Is there just no dynamic allocation? Otherwise I’d like to see the papers and subsequent Turing award for solving memory management once and for all.

                                                            As for the deadlines: the author of V has made ridiculous deadlines so many times, for no good reason (why promise something in a few weeks or months instead of just waiting for it to be polished?!). It’s not like open source projects are tied to pointy haired boss deadlines.

                                                        2. 13

                                                          Interestingly, I’m not an “FP diehard”; I come from an object-based tribe, and I work on object-based designs.

                                                          None of the listed improvements to V over Go are related to what makes Go bad; I have a thread from last year exploring the worst of Go’s flaws. In short, the problem isn’t even an upper limit on abilities, but a lower limit on how much code is required to do even the bare minimum, and a surprising lack of safety in common situations.

                                                          As the original post author and several others have repeatedly indicated throughout current and past discussion about V, the speed claims simply aren’t being substantiated in an open and reproducible configuration which the community can examine and test themselves. Simply changing the host language does not grant speed, unfortunately, because of interpretative overhead, and the only cure is putting effort into the compiler.

                                                          At the same time, it is toolchains and not languages that are fast, and so any novel speed improvements in V should be explicable to the rest of us. For example, in Monte, we use nanopass design and ANF style, originally explored in Scheme. We have a JIT in the Smalltalk/Java tradition, using an off-the-shelf toolkit, RPython.

                                                          As an aside, I would imagine that V would be able to more efficiently take advantage of GCC/LLVM/etc. by picking just one backend, and emitting code just for that backend. This would be due to C’s relatively poor guarantees about how memory will be used.

                                                          1. 5

                                                            V is translated to C before being compiled to machine code

                                                            That, right there, is enough for me to question any safety guarantee V offers (and I like C).

                                                            1. 4

                                                              Nim compiles to C and it’s garbage collected. I believe the reasons they do that are runtime reach and whole program optimization.

                                                              If you can statically guarantee safety it shouldn’t be a problem. (However, its not necessarily a trivial thing to suggest.)

                                                              1. 3

                                                                ATS compiles to C too, if I understand it correctly. And there have been Haskell compilers that compiled to C too and many other programming languages that provide some aspects of safety that the underlying C language, like the machine code, do not provide.

                                                                1. 4

                                                                  Why does using C as an intermediate language in the compilation process necessarily imply that a language’s safety guarantees are bad? Compilers that compile to some kind of bytecode - like rustc compiling to LLVM bitcode, or JVM langauges’ compilers compiling to JVM bytecode - are perfectly capable of being safe, even though they output code in an unsafe language (which may or may not be the final compilation output - it is (I think) in the JVM case, but LLVM bitcode is further transformed into machine-specific machine code). I don’t see why C should be any different in this respect.

                                                                  1. 3

                                                                    I don’t know the guarantees of LLVM or JVM, but at the language level, C has a ton of unspecified and undefined behaviors. Skipping the dangers around pointers and arrays, you still have the following undefined behavior as outlined in the C standard:

                                                                    • shifting an integer by a negative value
                                                                    • shifting an integer more than its size in bits
                                                                    • left shifting a negative value
                                                                    • signed integer representation (sign-magnitude, 1s complement, 2s complement [1])
                                                                    • (quoting from the C99 standard for this one): Whether certain operators can generate negative zeros and whether a negative zero becomes a normal zero when stored in an object
                                                                    • signed integer trap representations
                                                                    • signed integer wrap sematics
                                                                    • padding value
                                                                    • padding in general
                                                                    • reading a union member that wasn’t the last one written to

                                                                    Now, it seems that V is targeting GCC/clang, but even so, you’ll get differences in behavior across different architectures, specifically with shifting (some architectures will mask the shift count, some won’t). When I see “safety” as applied to a computer language, I would expect these issues will be spelled out as to what to expect.

                                                                    [1] In my research, there aren’t many systems in use today that are not 2s complement. They are:

                                                                    • Unisys 1100/2200
                                                                    • Unisys ClearPath A
                                                                    • IBM 700/7000 series

                                                                    I know one of the Unisys systems is still be produced today and has a C compiler (which one, I don’t recall, I think the 1100/2200).

                                                                    1. 2

                                                                      you do realize that source code gets compiled to machine code, which is not safe by definition.

                                                                      The generated C code doesn’t use any of these, and doesn’t have to.

                                                                      1. 3

                                                                        Then what’s your definition of “safe” then? There is way less that’s undefined in assembly than in C. Give me an architecture, and I can look up what it does for the above list. The reason C has so much undefined behavior is precisely because it runs on many architectures and the designers of C didn’t want to favor one over the other. Different languages can make different trade offs .

                                                                2. 5

                                                                  FP diehards will never understand why Go has been so wildly successful – and V will be even more successful than Go.

                                                                  Do you? Go succeeded because it was created and backed by veteran Bell Labs people and Google, to solve existing problems. I’m not talking about marketing only, but also the level of sophistication and simplicity those people were able to bring in.

                                                                  It also succeeded because it didn’t promise anything it didn’t deliver.

                                                                  1. -4

                                                                    Yes. I spotted Go as great technology in November of 2010. Go is simple and fairly powerful considering that simplicity.

                                                                    The original version of V was written in Go, V has learned many lessons from Go, both from its strengths that V builds on and the weaknesses it shores up with generic functions, generic structs, sum types, and more.

                                                              2. 14

                                                                I’d be interested in seeing what kind of these “lots of fixes in Generics are”, because as far as I can tell from reading the compiler source code, type errors for parameters in functions with a <T> slapped on them are still silently ignored…

                                                                  if !c.check_types(typ, arg.typ) {
                                                                    // str method, allow type with str method if fn arg is string
                                                                    if arg_typ_sym.kind == .string && typ_sym.has_method('str') {
                                                                      // note: str method can return anything. will just explode in the C compiler -- hydraz
                                                                      continue
                                                                    }
                                                                    if typ_sym.kind == .void && arg_typ_sym.kind == .string {
                                                                      continue
                                                                    }
                                                                    if f.is_generic {
                                                                      // ignore errors in functions with a <T> -- hydraz
                                                                      continue
                                                                    }
                                                                    if typ_sym.kind == .array_fixed {
                                                                    }
                                                                

                                                                Try this code:

                                                                fn  foo<T>(y string) int {
                                                                  return 0
                                                                }
                                                                
                                                                fn main() {
                                                                  foo(123)
                                                                }
                                                                
                                                                1. -4

                                                                  You had foo take a string then passed in an int :-)

                                                                  EDIT: This works, for example:

                                                                  fn foo<T>(y string) int {
                                                                    return 0
                                                                  }
                                                                  
                                                                  fn main() {
                                                                    println(foo<int>('hi'))
                                                                  }
                                                                  
                                                                  1. 22

                                                                    … Yes, that’s my point. I passed an int to a string parameter, and the V compiler didn’t give a type error: the C compiler did.

                                                                    % make
                                                                    cd ./vc && git clean -xf && git pull --quiet
                                                                    cd /var/tmp/tcc && git clean -xf && git pull --quiet
                                                                    cc  -g -std=gnu11 -w -o v ./vc/v.c  -lm -lpthread
                                                                    ./v self
                                                                    V self compiling ...
                                                                    make modules
                                                                    make[1]: Entering directory '/home/abby/Projects/v'
                                                                    #./v build module vlib/builtin > /dev/null
                                                                    #./v build module vlib/strings > /dev/null
                                                                    #./v build module vlib/strconv > /dev/null
                                                                    make[1]: Leaving directory '/home/abby/Projects/v'
                                                                    V has been successfully built
                                                                    V 0.1.27 b806fff
                                                                    
                                                                    % ./v test.v
                                                                    ==================
                                                                    /home/abby/.cache/v/test.tmp.c: In function ‘main’:
                                                                    /home/abby/.cache/v/test.tmp.c:9476:2: error: implicit declaration of function ‘foo’ [-Werror=implicit-function-declaration]
                                                                     9476 |  foo(123);
                                                                          |  ^~~
                                                                    /home/abby/.cache/v/test.tmp.c: In function ‘vcalloc’:
                                                                    /home/abby/.cache/v/test.tmp.c:4597:1: warning: control reaches end of non-void function [-Wreturn-type]
                                                                     4597 | }
                                                                          | ^
                                                                    /home/abby/.cache/v/test.tmp.c: In function ‘byte_is_white’:
                                                                    /home/abby/.cache/v/test.tmp.c:7227:1: warning: control reaches end of non-void function [-Wreturn-type]
                                                                     7227 | }
                                                                          | ^
                                                                    ...
                                                                    ==================
                                                                    (Use `v -cg` to print the entire error message)
                                                                    
                                                                    builder error: 
                                                                    ==================
                                                                    C error. This should never happen.
                                                                    
                                                                2. 17

                                                                  Author of the post here, let me see if I can try to clear some things up.

                                                                  She cites vlang.io saying

                                                                  V can be bootstrapped in under a second by compiling its code translated to C with a simple

                                                                  cc v.c

                                                                  No libraries or dependencies needed.

                                                                  then argues against it, preposterously, by saying in part,

                                                                  Git is a dependency, which means perl is a dependency, which means a shell is a dependency, which means glibc is a dependency, which means that a lot of other things (including posix threads) are also dependencies. …

                                                                  Downloading a .c source file requires git? Does this person know what a “dependency” is? Should JavaScript developers include depending upon the laws of physics in package.json?

                                                                  Okay I was being a bit unfair, but if we look at the makefile we see that it has the following dependencies:

                                                                  • make (which depends on perl, glibc, autotools and all that nonsense)
                                                                  • git (which depends on perl (even at runtime), glibc, autotools and all that nonsense)
                                                                  • gcc (which depends on perl, glibc, autotools, automake, autoconf and more libraries than I care to list right now)

                                                                  So if you want to be completely honest, even if you cut out the make and git steps (which i care about as someone who builds packages for linux boxes using the unmodified build system as much as possible so I can maintain whatever shred of sanity I have left), it still depends on a C compiler to get bootstrapped. This is a dependency. Then you have to download the bootstrap file from somewhere, which requires dependencies in terms of root certificates and the compiler to bootstrap with (not to mention the server that hosts the bootstrap file both existing and serving the right file back). Given that V in its current form requires you to download files from the internet in order to build it, mathematically it cannot be dependency free (this actually precludes it from being packageable in NixOS, because NixOS doesn’t allow package builds to access the network, all of the tarball/assets need to be explicitly fetched outside the build with fetchgit, fetchurl and similar). Pedantically, requiring someone to have an internet connection is a dependency.

                                                                  Pedantically, the v binary lists the following dynamically linked dependencies when using lld(1):

                                                                  $ ldd ./v
                                                                          linux-vdso.so.1 (0x00007fff2d044000)
                                                                          libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2fb3e4c000)
                                                                          libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2fb3a5b000)
                                                                          /lib64/ld-linux-x86-64.so.2 (0x00007f2fb4345000)
                                                                  

                                                                  If the binary was truly dependency-free, the ldd output would look something like this:

                                                                  $ ldd $HOME/bin/dhall
                                                                          not a dynamic executable
                                                                  

                                                                  This leads me to assume that the v binary has dependencies that the runtime system will need to provide, otherwise the program will not be able to be loaded by the Linux kernel and executed. Binaries produced by v have similar limitations:

                                                                  $ ldd ./hello
                                                                          linux-vdso.so.1 (0x00007ffdfdff2000)
                                                                          libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fed25771000)
                                                                          /lib64/ld-linux-x86-64.so.2 (0x00007fed25d88000)
                                                                  

                                                                  Additionally, I am banned from the V discord and GitHub. The V programming language has censored a neuro-diverse trans woman from being able to contribute to the project in any capacity. I would love to be able to contribute to things at least to make the documentation and website not filled with misleading statements, but I cannot. This is why I ask other people to make issues for me in my posts.

                                                                  I realize things might look snarky when the article is viewed from a certain worldview/lens, but there is a total scalar lack of snark intended in that article when it was written. If you cannot realize that, then I am sorry that my intended tone didn’t have the effect I wanted and I will use this feedback to further refine my writing ability.

                                                                  Direct x64 machine code generation

                                                                  In my testing I was unable to get this working on my Ubuntu server. It still used gcc.

                                                                  1. 8

                                                                    You ended your piece (which utterly trashes V) by saying

                                                                    Overall, V looks like it is making about as much progress as I had figured it would.

                                                                    I criticized your snark, and you replied with

                                                                    I realize things might look snarky when the article is viewed from a certain worldview/lens, but there is a total scalar lack of snark intended in that article when it was written.

                                                                    Do you really expect anyone to believe that?

                                                                    Additionally, I am banned from the V discord and GitHub. The V programming language has censored a neuro-diverse trans woman from being able to contribute to the project in any capacity.

                                                                    Do you really think that’s why you were banned? Does Alex even know you’re trans? You don’t think you were banned for your vicious and misleading attacks on this project?

                                                                    1. 5

                                                                      You’re just ignoring the points she made here, and keep talking besides her and banging on about the article.

                                                                      1. 6

                                                                        @cadey didn’t say they were banned for who they are, they just stated they were X and were banned.

                                                                        You don’t think you were banned for your vicious and misleading attacks on this project?

                                                                        If my memory serves me right, @cadey was banned because of their disagreements with V, such as those voiced in this and the previous (https://christine.website/blog/v-vvork-in-progress-2020-01-03) blog post. I could be wrong. Also “vicious” is being overly dramatic and frankly not productive.

                                                                        1. 4

                                                                          @cadey didn’t say they were banned for who they are, they just stated they were X and were banned.

                                                                          Then why bring it up?

                                                                          If my memory serves me right, @cadey was banned because of their disagreements with V, … . I could be wrong.

                                                                          Is that actually true?

                                                                          Also “vicious” is being overly dramatic …

                                                                          It doesn’t sound like you’re paying very close attention… Two other people criticized her for bullying in this very thread before I even got here. If you read my comments here then I’d hope you would change your mind about how unfairly harsh she has been to Alex, to his project, and to the V team.

                                                                          Consider starting here: https://lobste.rs/s/nfjifq/v_update_june_2020#c_vuofat

                                                                          1. 22

                                                                            You almost never comment except in V threads.

                                                                            Further, many of those comments seem to be today, in last year’s thread.

                                                                            Please just let this be. Flag the submission if you must and move on.

                                                                            1. 6

                                                                              I have removed the post as of this commit. In a few minutes the article will be gone, but a tombstone of the former article will remain.

                                                                              1. 19

                                                                                I don’t think you did anything wrong by posting it, I don’t think you made any mistakes, and I enjoyed reading it. Not saying this to convince you to put the essay back; I just know that I personally feel awful when I get really harsh criticism, even when I don’t respect or care about the person giving it. So wanted to provide a bit of positivity to balance it out ツ

                                                                              2. 8

                                                                                Then why bring it up?

                                                                                I can’t answer that question. While I’m not sure what the benefit is of bringing it up, I don’t see the harm either; it was pretty clear from the comment they were not saying they were banned because of who they are.

                                                                                Is that actually true?

                                                                                Feel free to show otherwise. I can’t, because I’m not the person who banned @cadey; nor am I in contact with them.

                                                                                It doesn’t sound like you’re paying very close attention

                                                                                Please do not make such assumptions. It’s unproductive starting an argument as such, as well as factually incorrect.

                                                                                Two other people criticized her for bullying in this very thread before I even got here.

                                                                                Only one person said it’s starting to look like bullying (https://lobste.rs/s/nfjifq/v_update_june_2020#c_cdxvwk). Other comments mentioning “bullying” either state they are not sure, or don’t see it as bullying. I don’t see anybody else mentioning this is bullying. Am I perhaps overlooking something?

                                                                                If you read my comments here then I’d hope you would change your mind about how unfairly harsh she has been to Alex, to his project, and to the V team.

                                                                                I agree the tone in the blog post is not the most productive. While some parts of the post are a bit pedantic, overall I think it’s not unfairly harsh. V made many big claims both before and after its release. Here are just a few examples:

                                                                                • https://github.com/vlang/v/issues/35
                                                                                • Translating C++ to V, which now has the note “TODO: translating C to V will be available in V 0.3. C++ to V will be available later this year.”
                                                                                • Claiming V has pure functions when they can still have side-effects. “Pure” has a well defined meaning. If you mean “pure but with IO”, then call it something else; otherwise it’s just misleading. This was brought up in this issue, which was closed, but I can’t find any mention of this in the docs here.
                                                                                • Various claims about certain components not having dependencies, only to have dependencies; despite the website advertising “Compiles to native binaries without any dependencies”
                                                                                • Advertising “V is written in V and compiles itself in under a second.”, when according to https://fast.vlang.io/ compiling V with optimisations (if I’m reading the table correctly) takes over one second most of the time.

                                                                                There is a lot more from past discussions (e.g. those on Hacker News), so I suggest taking a look at those.

                                                                                With that all said, I really do hope V succeeds and wish the authors the best of luck. But the authors really need to make sure that what they advertise is actually implemented as advertised, or make it very clear what the current state is. Don’t go around saying “We support X” followed by “oh by the way we’ll release that end of this year”.

                                                                      1. 8

                                                                        Isn’t there a difference between functional code and side-effect-free code? I feel like, by trying to set up all of the definitions just right, this article actually misses the point somewhat. I am not even sure which language the author is thinking of; Scheme doesn’t have any of the three mentioned properties of immutability, referential transparency, or static type systems, and neither do Python nor Haskell qualify. Scouring the author’s websites, I found some fragments of Java; neither Java nor Clojure have all three properties. Ironically, Java comes closest, since Java is statically typed in a useful practical way which has implications for soundness.

                                                                        These sorts of attempts to define “functional programming” or “functional code” always fall flat because they are trying to reverse-engineer a particular reverence for some specific language, usually an ML or a Lisp, onto some sort of universal principles for high-quality code. The idea is that, surely, nobody can write bad code in such a great language. Of course, though, bad code is possible in every language. Indeed, almost all programs are bad, for almost any definition of badness which follows Sturgeon’s Law.

                                                                        There is an important idea lurking here, though. Readability is connected to the ability to audit code and determine what it cannot do. We might desire a sort of honesty in our code, where the code cannot easily hide effects but must declare them explicitly. Since one cannot have a decidable, sound, and complete type system for Turing-complete languages, one cannot actually put every interesting property into the type system. (This is yet another version of Rice’s theorem.) Putting these two ideas together, we might conclude that while types are helpful to readability, they cannot be the entire answer of how to determine which effects a particular segment of code might have.

                                                                        Edit: Inserted the single word “qualify” to the first paragraph. On rereading, it was unacceptably ambiguous before, and led to at least two comments in clarification.

                                                                        1. 7

                                                                          Just confirming what you said: Did you say that Haskell doesn’t have immutability, referential transparency, or a static type system?

                                                                          1. 3

                                                                            I will clarify the point, since it might not be obvious to folks who don’t know Haskell well. The original author claims that two of the three properties of immutability, referential transparency, and “typing” are required to experience the “good stuff” of functional programming. On that third property, the author hints that they are thinking of inferred static type systems equipped with some sort of proof of soundness and correctness.

                                                                            Haskell is referentially transparent, but has mutable values and an unsound type system. That is only one of three, and so Haskell is disqualified.

                                                                            Mutable values are provided in not just IO, but also in ST and STM. On one hand, I will readily admit that the Haskell Report does not mandate Data.IORef.IORef, and that only GHC has ST and STM; but on the other hand, GHC, JHC, and UHC, with UHC reusing some of GHC’s code. Even if one were restricted to the Report, one could use basic filesystem tools to create a mutable reference store using the filesystem’s innate mutability. In either case, we will get true in-place mutation of values.

                                                                            Similarly, Haskell is well-known to be unsound. The Report itself has a section describing how to do this. To demonstrate two of my favorite examples:

                                                                            GHCi, version 8.6.3: http://www.haskell.org/ghc/  :? for help
                                                                            Prelude> let safeCoerce = undefined :: a -> b
                                                                            Prelude> :t safeCoerce
                                                                            safeCoerce :: a -> b
                                                                            Prelude> data Void
                                                                            Prelude> let safeVoid = undefined :: Void
                                                                            Prelude> :t safeVoid
                                                                            safeVoid :: Void
                                                                            

                                                                            Even if undefined were not in the Report, we can still build a witness:

                                                                            Prelude> let saferCoerce x = saferCoerce x
                                                                            Prelude> :t saferCoerce
                                                                            saferCoerce :: t1 -> t2
                                                                            

                                                                            I believe that this interpretation of the author’s point is in line with your cousin comment about type signatures describing the behavior of functions.

                                                                            1. 4

                                                                              I don’t really like Haskell, but it is abusive to compare the ability to write a non-terminating function with the ability to reinterpret an existing object as if it had a completely different type. A general-purpose programming language is not a logic, and the ability to express general recursion is not a downside.

                                                                              1. 3

                                                                                A “mutable value” would mean that a referenced value would change. That’s not the case for a value in IO. While names can be shadowed, if some other part of the code has a reference to the previous name, that value does not change.

                                                                                1. 1

                                                                                  Consider the following snippet:

                                                                                  GHCi, version 8.6.3: http://www.haskell.org/ghc/  :? for help
                                                                                  Prelude> :m + Data.IORef
                                                                                  Prelude Data.IORef> do { r <- newIORef "test"; t1 <- readIORef r; writeIORef r "another string"; t2 <- readIORef r; return (t1, t2) }
                                                                                  ("test","another string")
                                                                                  

                                                                                  The fragment readIORef r evaluates to two different actions within this scope. Either this fragment is not referentially transparent, or r is genuinely mutable. My interpretation is that the fragment is referentially transparent, and that r refers to a single mutable storage location; the same readIORef action applied to the same r results in the same IO action on the same location, but the value can be mutated.

                                                                                  1. 1

                                                                                    The value has been replaced with another. It is not quite the same thing as mutating the value itself.

                                                                                2. 2

                                                                                  From your link:

                                                                                  When evaluated, errors cause immediate program termination and cannot be caught by the user.

                                                                                  That means that soundness is preserved–a program can’t continue running if its runtime types are different from its compile-time types.

                                                                                  1. 1

                                                                                    If we have to run the program in order to discover the property, then we run afoul of Rice’s theorem. There will be cases when GHC does not print out <loop> when it enters an infinite loop.

                                                                                    1. 1

                                                                                      Rice’s theorem is basically a fancier way of saying ‘Halting problem’, right?

                                                                                      In any case, it still doesn’t apply. You don’t need to run a program which contains undefined to have a guarantee that it will forbid unsoundness. It’s a static guarantee.

                                                                              2. 5

                                                                                Thank you for bringing up this point. Unfortunately, “functional programming” is almost-always conflated, today, with lack of side-effects, immutability, and/or strong, static, typing. None of those are intrinsic to FP. Scheme, as you mentioned, is functional, and has none of those. In fact, the ONLY language seeing any actual use today that has all three (enforced) is Haskell. Not even Ocaml does anything to prevent side-effects.

                                                                                And you absolutely can write haskell-ish OOP in e.g., Scala. Where your object methods return ReaderT-style return types. It has nothing at all to do with funcitonal vs. OOP. As long as you do inversion of control and return “monads” or closures from class methods, you can do all three of: immutable data, lack of side-effects, and strong types in an OOP language. It’s kind of ugly, but I can do that in Kotlin, Swift, Rust, probably even C++.

                                                                                1. 2

                                                                                  Scheme, as you mentioned, is functional, and has none of those.

                                                                                  Why is Scheme functional? It’s clearly not made of functions:

                                                                                  Lisp Is Not Functional

                                                                                  A functional language is a programming language made up of functions

                                                                                  What defun and lambda forms actually create are procedures or, more accurately still, funcallable instances

                                                                                  I would say Haskell and Clojure are functional, or at least closer to it, but Scheme isn’t. This isn’t a small distinction…

                                                                                  1. 2

                                                                                    That’s a good point and I actually do agree completely. The issue, I think, is that most programmers today will have a hard time telling you the difference between a procedure and a function when it comes to programming. And it’s totally fair- almost every mainstream programming language calls them both “function”.

                                                                                    So, Scheme is “functional” in that it’s made up of things-that-almost-everyone-calls-functions. But you’re right. Most languages are made of functions and procedures, and some also have objects.

                                                                                    But with that definition, I don’t think Clojure counts as functional either. It’s been a couple of years, but am I not allowed to write a “function” in Clojure that takes data as input and inside the function spawns an HTTP client and orders a pizza, while returning nothing?

                                                                                    It would appear that only Haskell is actually a functional language if we use the more proper definition of “function”

                                                                                    1. 1

                                                                                      But with that definition, I don’t think Clojure counts as functional either. It’s been a couple of years, but am I not allowed to write a “function” in Clojure that takes data as input and inside the function spawns an HTTP client and orders a pizza, while returning nothing?

                                                                                      Hey, the type for main in Haskell is usually IO (), or “a placeholder inside the IO monad”; using the placeholder type there isn’t mandatory, but the IO monad is. Useful programs alter the state of the world, and so do things which can’t be represented in the type system or reasoned about using types. Haskell isn’t Metamath, after all. It’s general-purpose.

                                                                                      The advantage of Haskell isn’t that it’s all functions. It’s that functions are possible, and the language knows when you have written a function, and can take advantage of that knowledge. Functions are possible in Scheme and Python and C, but compilers for those languages fundamentally don’t know the difference between a function and a procedure, or a subroutine, if you’re old enough. (Optimizers for those languages might, but dancing with optimizers is harder to reason about.)

                                                                                    2. 1

                                                                                      That article is about Common Lisp, not Scheme. Scheme was explicitly intended to be a computational representation of lambda calculus since day 1. It’s not purely functional, yes, but still functional.

                                                                                      1. 2

                                                                                        If anything that underscores the point, because lambda calculus doesn’t have side effects, while Scheme does. The argument applies to Scheme just as much as Common Lisp AFAICT.

                                                                                        Scheme doesn’t do anything to control side effects in the way mentioned in the original article. So actually certain styles of code in OO languages are more functional than Scheme code, because they allow you to express the presence of state and I/O in type signatures, like you would in Haskell.

                                                                                        That’s probably the most concise statement of the point I’ve been making in the thread …

                                                                                        1. 2

                                                                                          I take it we’re going by the definition of ‘purely functional programming’ then. In that case, I don’t understand why Clojure, a similarly impure language, gets a pass. Side-effects are plentiful in Clojure.

                                                                                          1. 2

                                                                                            Well I said “at least closer to it”… I would have thought Haskell is very close to pure but it appears there is some argument about that too elsewhere in the thread.

                                                                                            But I think those are details that distract from the main point. The main point isn’t about a specific language. It’s more about how to reason about code, regardless of language. And my response was that you can reap those same benefits of reasoning in code written in “conventional” OO languages as well as in functional languages.

                                                                                            1. 1

                                                                                              That’s fair. It’s not that I disagree with the approach (I’m a big fan of referential transparency!) but I feel like this is further muddying the (already increasingly divergent) terminology surrounding ‘functional programming’. Hence why I was especially confused by the OO remarks. It doesn’t help that the article itself also begs the question of static typing.

                                                                                  2. 2

                                                                                    Isn’t there a difference between functional code and side-effect-free code?

                                                                                    It depends on who you ask to. :)

                                                                                    You may be interested in the famous Van Roy’s organization of programming paradigms: https://www.info.ucl.ac.be/~pvr/VanRoyChapter.pdf. Original graphical summary: https://continuousdevelopment.files.wordpress.com/2010/02/paradigms.jpg, revised summary: https://upload.wikimedia.org/wikipedia/commons/f/f7/Programming_paradigms.svg.

                                                                                    (reposted from https://lobste.rs/s/aw4fem/unreasonable_effectiveness#c_ir0mnq)

                                                                                    1. 1

                                                                                      I like your point about the amount of information in type signatures.

                                                                                      I agree that the type can’t contain everything interesting to know about the function.

                                                                                      I do think you can choose to put important information in the type. In Haskell it’s normal to produce a more limited effect system, maybe one for database effects only, and another for network effects only, and then connect those at the very top level.

                                                                                      So, you can put more in the type signature if you wish, and it can be directly useful to prevent mixing effects.

                                                                                    1. 7

                                                                                      I am not convinced by the premise, mostly because I think that the tradeoff between practice and knowledge is illusory.

                                                                                      Tacit knowledge is knowledge that cannot be captured through words alone.

                                                                                      Most knowledge is not tacit; we can and do use words to transmit the underlying concepts. Indeed, the author is using words alone to communicate the idea of tacit knowledge. In associated articles, the author contradicts their own definition. In The Mental Model FAQ, the author instead defines tacit knowledge in terms of mental representations, and explains that tacit knowledge is implicit in the mental representation of the knower:

                                                                                      Mental representations are difficult — if not impossible! — to communicate explicitly. They are tacit in nature.

                                                                                      So, yes, mental models are incredibly useful. If nothing else, they tell you that the valuable bits of expertise is not what is communicable, but what is tacit and stuck inside the practitioner’s head.

                                                                                      In The Mental Model Fallacy, the author uses “tacit” as in the top-posted article:

                                                                                      But there also exist mental models that are nearly impossible to communicate. We call this tacit knowledge.

                                                                                      Polanyi argued that there is knowledge that cannot be adequately articulated by verbal means, and asserted that most of human knowledge is rooted in tacit knowing.

                                                                                      This is not quite the same as the top-posted article, though. Is tacit knowledge unable to be expressed with words, or difficult to express with words? There is a massive gulf between these two situations, and the gulf is wide and deep enough that, for anything unable to be expressed with words, I expect a proof along the lines of Berry’s paradox. Interesting numbers are not definable numbers.

                                                                                      Really, to be frank, what I think that the author is so close to discovering is that words have no meanings. Since words have no meanings, there isn’t any actual semantic content transferred by an explanation. Therefore, regardless of the number of words used, nobody can really teach anybody how to do anything; words might be a guide, but every learner must discover every fact for themselves, and cannot rely on words to show them how it might be done, because words cannot carry semantics, because words have no meanings. Through this viewpoint, all knowledge is tacit, and all written words are merely reflections of how people feel about knowing things.

                                                                                      1. 3

                                                                                        Really, to be frank, what I think that the author is so close to discovering is that words have no meanings.

                                                                                        I get where you are coming from here and would agree that in the end, everyone has to learn something on their own. However, I think a bigger issue might be confusing experience and rhetorical ability.

                                                                                        I have often had this debate with colleagues, whether we should look for people with innate ability, a good ability to learn, or that can follow instructions well and which of those factors made us successful. I do believe some people learn quicker than others, but the crux of his argument seems to not be the speed at which someone picks something up, but the ability to render it into words.

                                                                                        I am often guilty of speaking like his example “Well, do X. Except when you see Y, then do Z, because A. And if you see B, then do P. But if you see A and C but not B, then do Q, because reason D. And then there are weird situations where you do Z but then see thing C emerge, then you should switch to Q.” But this is not because I have some tacit knowledge the listener does not, it is because I cannot appropriately articulate the underlying heuristics that need to be learned. Often times, if we thought about it more, I’ll bet the example ends up much more refined, closer to codified steps, but the speaker never had to think about the underlying rules.

                                                                                        To illustrate what I mean, I would use the writer’s example of teaching a kid to ride a bike. I would have no idea how to approach that, but the writer was able to distill it down to a step by step guide. He thought about it and found the appropriate words.

                                                                                        So overall, I’d agree he misses the mark, but not as much because words have no meaning as because he is assuming someone who can perform well in their field also has the rhetorical and logical capacity to render their experience into appropriate language.

                                                                                        1. 3

                                                                                          To illustrate what I mean, I would use the writer’s example of teaching a kid to ride a bike. I would have no idea how to approach that, but the writer was able to distill it down to a step by step guide. He thought about it and found the appropriate words.

                                                                                          I disagree. He distilled riding a bike to a step-by-step guide to learning. He was unable to distill it to a step-by-step guide to riding for someone who hadn’t learned.

                                                                                          1. 1

                                                                                            Fair point, I chose a bad example. What I meant was if presented today with the challenge of telling someone how to teach a child to ride a bike, I wouldn’t know how to begin. I have neither the words, nor the experience. The writer had the experience and came up with decent enough words that I could begin to gain the experience, had I a child and a small bike. But, you are right, he has not expressed the right words for riding in here. Does that mean they can’t exist?

                                                                                            1. 4

                                                                                              Riding a bike is a lot about trained reflexes. I’m very confident that no amount of words can instill reflexes in somebody.

                                                                                              Maybe a more direct connection to neurons can do that? Like the Matrix movie depicts it as Neo learns Kung Fu which is also a lot about reflexes.

                                                                                              1. 1

                                                                                                I don’t think it means they can’t. The author cited Boyd as an example of one case where someone did figure out how to put into words a thing that had been widely considered unexplainable.

                                                                                                One point I took away from the piece is that it’s more efficient to set about gaining the “unexplainable” knowledge through guided experience (as in the bike riding example) as opposed to try and brute force explain it or read up on it.

                                                                                                (And if you ever find yourself wanting to teach a small child to ride a bicycle, the “balance bike” is an invention that significantly accelerates the learning the author explained in that example. Source: I’m not much good at bike riding myself, damn sure can’t explain how, and taught two small people how to ride by helping them get the feel on “balance bikes” first.)

                                                                                            2. 1

                                                                                              Are we self-aware enough articulate the underlying heuristics? Humans are biased in lots of different ways and I would assume that all our heuristics are also biased in ways we are not aware of.

                                                                                              One symptom is that seem to forget stuff: “You saw A and C but not B, so you tried Q but it did not help. Oh, did you also check E? In that case Q is of course wrong and you should do R instead.” Did we actually forget it or do we rationalize heuristics below our awareness? If we rationalize it then we can never know if the explicit heuristics are complete or if more special cases are lurking. That seems to match the futility of expert systems where a lot of effort was made to codify such decision making. For some reason there was always something only the humans could do though.

                                                                                            3. 3

                                                                                              You really think there is a “massive gulf” between theoretically impossible and practically impossible? Such pedantry! No wonder the poor guy had to write this (excellent, IMO) essay.

                                                                                              Take any Linguistics 101 and you’ll learn very quickly how much tacit knowledge you needed to have just to express or understand a grammatical sentence in the first place.

                                                                                              1. 2

                                                                                                Isn’t “tacit knowledge is knowledge that cannot be captured through words alone” like Berry’s paradox “the smallest positive integer not definable in under sixty letters”? Of course, we can now discuss what “captured” means just like Wikipedia describes the discussion about “definable”.

                                                                                              1. 4

                                                                                                I’m continuing to tear down my LLC, distilling old NixOS modules into a public repository.

                                                                                                1. 2

                                                                                                  I vastly prefer the approach that lit.sh takes over this. Literate seems to allow for defining arbitrary blocks wherever and then providing primitives for assembling, which is flexible but seems confusing. A more inline, top down approach, like that from org-mode + Babel or lit.sh is the sweet spot for me.

                                                                                                  1. 3

                                                                                                    This is a feature from the original literate programming tools. Wikipedia has a section on it: https://en.wikipedia.org/wiki/Literate_programming#Order_of_human_logic,_not_that_of_the_compiler. In fact, I think it was the whole point of literate programming. The examples Knuth gave were great, he’d say something like:

                                                                                                    The outline of the program is
                                                                                                    
                                                                                                    --- somefile
                                                                                                    @{Includes}
                                                                                                    @{Globals}
                                                                                                    @{Functions}
                                                                                                    @{Main}
                                                                                                    ---
                                                                                                    
                                                                                                    The first part of the problem we will tackle is outputting the information to the screen.
                                                                                                    We would first need to include stdio.h in order to write to the screen
                                                                                                    
                                                                                                    --- Includes+=
                                                                                                    #include <stdio.h>
                                                                                                    ---
                                                                                                    
                                                                                                    We should also name the variables we will store the word count in
                                                                                                    
                                                                                                    --- Globals+=
                                                                                                    int bytes = 0;
                                                                                                    int words = 0;
                                                                                                    int lines = 0;
                                                                                                    ---
                                                                                                    
                                                                                                    Then finally we can write a function to print these out
                                                                                                    
                                                                                                    --- Functions+=
                                                                                                    void print_result() {
                                                                                                      …
                                                                                                    }
                                                                                                    

                                                                                                    Nothing too exciting so far right? But the real kicker comes when we start working on the input to the system, we can go back and append to includes/globals/functions. We’re not using the order the program is written, but the order we want to explain the problem in. This is more relevant in certain runtimes/languages (I’m thinking of (require 'foo) in Cljs or #include in C).

                                                                                                    1. 2

                                                                                                      I hadn’t thought about it that way, but I can see how it makes sense. You can tell the story in a way that is separated from the flow of code. I guess I’d argue that in most languages you can (at least) forward declare functions that aren’t being used yet, making it possible to do this by writing a program from top to bottom in whatever order the functions make sense to introduce them in?

                                                                                                      It seems like a mental gymnastics exercise to piece together the program in the Literate source form, otherwise…? Perhaps I should give it a try and see. I could be completely wrong.

                                                                                                      1. 2

                                                                                                        The given point seems worth taking on its face. I used lit.sh to design literate code support for Monte, and looking at the module where I first developed and used that support, the very beginning of the file starts with a bunch of imports and then an apology for the fact that some things will be declared in an awkward order because we don’t detangle.