Threads for matklad

  1. 4

    If I want to define the API via language independent IDL . . .

    Do you? Why?

    IDLs are schemas that define a formal protocol between communicating parties. They represent a dependency, typically a build-time dependency, coupling producers and consumers. That can bring benefits, definitely. But if I can’t call your API without fetching a specific file from you, somehow, and incorporating it into my client, that’s also a significant cost! Open networks are supposed to work explicitly without this layer of specificity. HTTP is all I need to talk to any website, right?

    HTTP+JSON is pretty language-agnostic. I can even do it at the shell. JSON-RPC is nowhere near as mature, or as supported. What does this extra layer get you? What risks does it actually reduce? And are those risks relevant for your use cases? Why do you think e.g. Stripe doesn’t require JSON-RPC for its API?

    IMO the only place that IDLs make sense are in closed software ecosystems — i.e. corporate environments — that have both experienced specific pains from loosely-defined JSON APIs, and can effectively mandate IDL usage across team boundaries. Anywhere else, I find it’s far more pain than pleasure.

    1. 3

      Heh, I actually was thinking about your’s similar comment in another thread when asking, thanks for elaborating!

      I think I agree with your reasoning, but somehow still come to the opposite conclusion.

      First, agree that the JSON-RPC is a useless layer of abstraction. I’ve had to use it twice, and both times the value was negative. In this question, I am asking not about the jsonrpc 2.0, but about an RPC which works over HTTP, encoding payloads in JSON. I do have an RPC, rather than REST, style in mind though.

      I also couldn’t agree more about the issue with build-time dependencies is worth solving. Build time deps is exactly the problem I see at my $dayjob. We have an JSON-over-HTTP RPC interface and the interface is defined “in code” – there’s a bunch of structs with #[derive(Serialize)] structs. And this leads people to thinking along the lines of “I need to use the API. The API is defined by this structs. Therefore, I must depend on the code implementing the API”. This wasn’t explicitly designed for, just a path of least resistance if you don’t define API explicitly, and your language has derives.

      That being said, I think there has to be some dependency between producer and consumers? Unless you go full HATEOAS, you somehow need to know which method to call and (in a typed language, for ergonomics) which shape the resulting JSON would have. For stripe, I need to fetch https://stripe.com/docs/api/pagination/search to figure out what’s available. And, again, there is https://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.json.

      And, if we need at least an informal doc for the API, I don’t see a lot of drawbacks in making it more formal and writing, say, literal typescript rather than free-form text, provided that the formalism is lightweight. The biggest practical obstacle there seems to be absence of such lightweight formalisms.

      So, the specific “why”s for me wanting an IDL in an open decentralized ecosystem are:

      • reifying “this is the boundary exposed to outside world” in some sort of the specific file, so that it is abundantly clear that, if you are changing this file, you might break API and external clients. You could do that in the “API is informally specified in code” scenario, but that requires discipline, and discipline is finite resource, running out especially quickly in larger teams.
      • providing the low-friction way to document and publish promises regarding the API to the outside world. Again, with some discipline, documenting API in api.md file would work, but it seems to me that doc-comments require less effort for upkeep than separate guides.
      • making sure that details about the language the implementation is written in don’t accidentally leak. Eg, APIs probably shouldn’t use integers larger than 32 bits because they won’t work in JavaScript, but, if my impl language is Rust, I might not realize that as native serialization libraries would make u64 just work. More generally, in Rust just slapping derive(Serialize) always works, and often little thought is given to the fact that the resulting JSON might be quite ugly to work with in any non-rust language (or just to look at).
      • Maaaaybe generating stub clients and servers from the spec.
      1. 3

        That being said, I think there has to be some dependency between producer and consumers?

        Yeah! When a client calls a server there is definitely a dependency there. And as you note, there has to be some kind of information shared between server in client in order for that call to succeed. I suppose my point is about the… nature? of that dependency. Or maybe the efficacy of how it’s modeled?

        At the end of the day, you’re just sending some bytes over a socket and getting some bytes back. An IDL acts as a sort of filter, that prevents a subset of those byte-sets from leaving the client, if they don’t satisfy some criteria which is assumed will be rejected by the server. Cool! That reduces a class of risk that would otherwise result in runtime errors.

        1. What is impact of that risk?
        2. What benefits do I get from preventing that risk?
        3. What costs do I incur by preventing that risk?

        I suppose I’m claiming that, outside of a narrow set of use cases, and certainly for general-purpose public APIs, the answers to these questions are: (1) quite low, (2) relatively few, and (3) very many. (Assuming you’re careful enough to not break existing consumers by modifying the behavior of existing endpoints, and etc. etc.)

        reifying “this is the boundary exposed to outside world” in some sort of the specific file, so that it is abundantly clear that, if you are changing this file, you might break API and external clients. You could do that in the “API is informally specified in code” scenario, but that requires discipline, and discipline is finite resource, running out especially quickly in larger teams.

        I get that desire! The problem is that the IDL is a model of reality, not the truth, and, like all models, it’s a fiction :) Which can be useful! But even if you publish an IDL, some dingus can still call your API directly, without satisfying the IDL’s requirements. That’s the nature of open ecosystems. And there’s no way for you to effectively mandate IDL consumption with plain ol’ HTTP APIs, because (among other reasons) HTTP is by construction an IDL-free protocol. So the IDL is in some sense an optimistic, er, optimization. It helps people who use it — but those people could just as easily read your API docs and make the requests correctly without the IDL, eh? Discipline is required to read the API docs, but also to use the IDL.

        . . . document and publish … API [details] . . . making sure that details about the language [ / ] implementation don’t accidentally leak . . .

        IDLs convert these nice-to-haves into requirements, true.

        generating stub clients and servers

        Of course there is value in this! But this means that client and server are not independent actors communicating over a semantically-agnostic transport layer, they are two entities coupled at the source layer. Does this model of your distributed system reflect reality?

        I dunno really, it’s all subjective. Do what you like :)

        1. 1

          Convinced! My actual problem was that for the thing I have in mind the client and the server are implemented in the same code base. I test them against each other and I know they are compatible, but I don’t actually know how the JSON on the wire looks. They might silently exchange xml for all I know :-)

          I thought to solve this problem with an IDL which would be close to on-the-wire format, but it’s probably easier to just write some “this string can deserialize” tests instead.

          I’d still prefer to use IDL here if there were an IDL geared towards “this are docs to describe what’s on the wire” rather than “these are types to restrict and validate what’s on the wire”, but it does seem there isn’t such descriptive thing at the moment.

          1. 1

            docs to describe what’s on the wire [vs] types to restrict and validate what’s on the wire

            Is there a difference here? I suppose, literally, “docs” wouldn’t involve any executable code at all, would just be for humans to read; but otherwise, the formalisms necessary for description and for parsing seem almost identical to me.

            1. 1

              an IDL which would be close to on-the-wire format

              (Just in case you missed it, this is what Preserves Schema does. You give a host-language-neutral grammar for JSON values (actually Preserves, but JSON is a subset, so one can stick to that). The tooling generates parsers, unparsers, and type definitions, in the various host languages.)

              1. 1

                If your client and server exist in the same source tree, then you don’t have the problems that IDLs solve :)

                edit: For the most part. I guess if you don’t control deployments, IDLs could help, but certainly aren’t required.

                1. 1

                  They are the sever and a client — I need something to test with, but I don’t want that specific client to be the only way to drive thing.

                  1. 1

                    They are the sever and a client — I need something to test with, but I don’t want that specific client to be the only way to drive thing.

                    I test them against each other and I know they are compatible, but I don’t actually know how the JSON on the wire looks. They might silently exchange xml for all I know :-)

                    It seems you have a problem :)

                    If the wire protocol isn’t specified or at least knowable independent of the client and server implementations, then it’s an encapsulated implementation detail of that specific client and server pair. If they both live in the same repo, and are reasonably tested, and you will never have other clients or servers speaking to them, then no problem! It’s a closed system. The wire data doesn’t matter, the only thing that matters is that client-server interactions succeed.

                    If all you want to do is test interactions without creating a full client or server component, which I agree is a good idea, you for sure don’t need an IDL to do it. You just tease apart the code which speaks this custom un-specified protocol from the business logic of the client and server.

                    package protocol
                    package server // imports protocol
                    package client // imports protocol
                    

                    Now you can write a mock server and/or mock client which also import protocol.

                    1. 2

                      This (extracted protocol with opaque wire format) is exactly the situation I am in. I kinda do want to move to the world where the protocol is knowable indepedently of the specfic client and server, hence I am seeking an IDL. And your comment helped me realize that just writing tests against specific wire format is a nice intermediate step towards making the protocol specification – this is an IDL-less way to actually see what the protocol looks like, and fix any weirdness. As in, after writing such tests, I changed JSON representations of some things because they didn’t make sense for an external protocol (it’s not stable/public yet, so it’s changable so far).

        1. 1

          A colleague was asking me about fuzzing, perhaps I should send this to them along with the first part.

          Apropos: is there a good discussion of the differences between fuzzing and Property-Based Testing?

          1. 1

            I don’t think I’ve seen one, but in lieu of something more rigorous, here’s my take. It’s a bit of a fuzzy boundary, but property testing usually:

            • Allows building constrained, but richly typed values (maybe via a newtype)
            • Allows shrinking found examples
            • focusses on the relationionship between input and output (even if it’s it “it acts like this simplified model”)
            • Usually starts with random values from a distribution

            Wheras fuzzing:

            • Usually generates plain bytes
            • Doesn’t usually offer shrinking (but I’d be very happy to be wrong about that)
            • Is usually focussed on finding crashes of some kind
            • Sometimes starts from a set of known examples (eg: a jpeg file for an image loader).
            1. 2

              Doesn’t usually offer shrinking (but I’d be very happy to be wrong about that)

              Assuming this is the same thing I think libFuzzer does try to do that:

              -reduce_inputs Try to reduce the size of inputs while preserving their full feature sets; defaults to 1.

              https://www.llvm.org/docs/LibFuzzer.html

              1. 2

                The above maches my experience: usually that’s how the two setups look like.

                But I want to argue that these are accidental differences – property-based testing and fuzzing is the same technique, if implemented properly. Specifically, it is possible (and rather easy) to get all of:

                • coverage guided program exploration
                • by a state-of-the-art fuzzing engines (AFL or libfuzzer)
                • using highly structured data as an input (eg, the set of WASM programs which pass validation)
                • with support for shrinking

                The trick is to take raw bytes from fuzzer and use it as a finite PRNG you feed into property-style well-typed generator. This automatically gives you shrinking – by minimizing input raw bytes, you minimize the output well-typed struct.

                I don’t know of a good article describing this perspective abstractly. https://fitzgeraldnick.com/2020/08/24/writing-a-test-case-generator.html is a great concrete implementation.

                And I think the core idea of formulating property-based testing as generated structured outputs from unstructered inputs, getting universal shrinking for free was popularized (and probably invented?) by Python’s hypothesis: https://hypothesis.works/articles/compositional-shrinking/

                1. 1

                  But I want to argue that these are accidental differences

                  That’s fair. It’s pretty nebulous, really. I feel like the main difference is the perspective, or goals, rather than anything concrete.

            1. 8

              When more and more Rust seeps into the kernel, we might end up with the situation that the kernel takes longer to compile than Chromium.

              1. 17

                Not if Chromium itself doesn’t stand still!

                https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/security/rust-toolchain.md

                I would also expect the kernel to stick with fast-to-compile C-style subset of Rust, which is probably still quite a bit slower than C, but should be significantly faster than the typical monomorphisation-heavy userspace Rust code.

              1. 3

                I created an IDL with plain typescript interfaces. The tool parses Typescript AST, converts exported interfaces and meta to JSON. From there I use simple ES6 template literals to generate code.

                I started creating my own DSL, but why not just use TypeScript. It’s simpler than gRPC or GraphSQL for generating code. I would never write OpenAPI (feels like J2EE for APIs).

                Typescript has mostly everything you need:

                export interface AddArg {
                    a: number;
                    b: number;
                    // required is '?'
                    c?: number;
                }
                
                export interface Math {
                  // language specific instructions through tags
                  /** @_go { ident: 'Add' } */
                  add(arg: AddArg): number
                }
                

                I create strongly typed TypeScript requests in Next.js frontend and Go backend code from the JSON IDL.

                It’s not in a state where I want to share the repo yet. I’m iterating on how to process templates more intuitively, taking ideas from hygen.

                1. 1

                  <3 exciting, from the cursory look, it does seem that the world is missing something like this, thanks!

                1. 4

                  This is a weird question because you seem to already know the answer — JSON Schema — but don’t seem to like it.

                  I’ve used JSON Schema for “real world” things. It’s fine. I’m generally meh on the whole typed-RPC approach to web APIs, but if you’ve decided you’re going to do JSON-RPC, then JSON Schema is a tolerable way to do it.

                  1. 1

                    Yes, I believe both that JSON schema solves the problem I have, and that this solution, while technically working, is unsatisfactory. I don’t see a contradiction here: feels similar to, eg, inventing JSON while there’s already XML :)

                    1. 3

                      My takeaway would be less “I should look for an alternative to JSON Schema” and more “I should look for an alternative to doing this with an RPC protocol, because the things I dislike about JSON Schema are just a symptom”. But YMMV :)

                  1. 5

                    Honestly, graphql is perfectly reasonable. What type of simplicity do you think it violates?

                    Avro has a a json representation and is intended to define rpc interfaces. It’s also a type based idl.

                    1. 1

                      For my case, I think graphql solves to much — I think I want a straightforward request/response RPC, I don’t need graphql capabilities for fetching data according to a specific query.

                      This is kinda what protobuf does, but, impl wise, protobuf usually codgens a huge amount of ugly code which is not pleasant to work with (in addition to not being JSON). Not sure what’s the status with similar implementation complexity for graphql — didn’t get a chance to use it yet.

                      1. 2

                        Protobuf has a 1:1 json encoding. You could write your schema in Protobuf and then use the JSON encoding with POST requests or something to avoid all the shenanigans?

                      2. 1

                        Honestly, graphql is perfectly reasonable.

                        Doesn’t it still violate HTTP, badly? IIRC, it returns error responses with status code 200. I thought that it sent mutations in GET requests too, but from a quick look it looks like I misremembered that (shame on me!).

                        Regardless, REST is best.

                        1. 1

                          No, it returns errors inline in the body. There’s nothing non-restful about graphql. Indeed, the introspection and explicit schema, together with the ability to use it over a get request make it more restful than most fake rest rpc endpoints.

                      1. 15

                        I’m worried that ante may be trying to do too much at once. It is trying to innovate on at least:

                        • the syntax
                        • ergonomics static verification with SMT-checked refinement types
                        • low-level aspects / efficiency among functional languages, by playing on memory layout (hinted at but not described in the documentation) and memory management (lifetime/region inference)
                        • pervasive use of effect handlers as an effect-definition feature

                        This is a lot of topics (some of the research-level or at least very difficult) to tackle at the same time! And several of those goals are in tension, for example it is extra difficult to verify effect-handling programs or to make them efficient. (Efficient implementation of effect handlers that make them competitive with built-in effects for generators, etc. is an active research area).

                        If it all works out, great! But there is a good chance that the people (person ?) working on this end up spread much too thin and not clearly meet any of the goal that have been set.

                        1. 3

                          I didn’t want to say anything in particular about it but I’m glad to see my thoughts are pretty much what’s on other’s minds too.

                          This is why I’ve really come to love Standard ML. it hits the sweetest of spots: speed, syntax, low level, type checking, all achievable by one person. If Ante can achieve Standard ML levels of implementation simplicity it will go far… Otherwise, no one has any reason to use it over something like Rust which has huge developer buy-in.

                          1. 3

                            SML is not low level like this, surely, though?

                            1. 1

                              Why “surely”? It has references and FFI. I looked at Ante’s language reference quickly and it doesn’t look to do anything like C’s union or using bitfields. Actually it specifies references exactly like SML: ref keyword.

                              You could write a device driver in SML for sure with assembly and its FFI.

                              1. 3

                                Actually it specifies references exactly like SML: ref keyword.

                                it seems to me that, while the two features in SML and ante share name and syntax, they have completely different semantics. Different in the sense of orthogonal — these are completely independent language features.

                                Refs in SML are for storing mutable data. They are analogous to RefCell in Rust — something with a set operation, which nonetheless exists in pure functional world.

                                Refs in ante are pointers: values are unboxed, and you, as a programmer, have a choice between passing/returning stuff by value or by reference.

                                1. 1

                                  I didn’t see anything which signified being able to set the pointer to an arbitrary memory address and access it - could you please point me to it? (hehe) If so then for sure it’s not the same. ref in SML is as you said more or less: lets you box values.

                                  Otherwise the semantics between them is pretty much the same… as far as I can tell anyway

                                  1. 4

                                    ref in SML is as you said more or less: lets you box values.

                                    I am 0.9 sure this is wrong. In SML, refs denote mutability, they are orthogonal to in-memory representation.

                                    As for in-memory representation, as far as I know SML just doesn’t give you control over it. All values have the same size_of of a single pointer. In other words, everything is boxed anyway. As an observable effect of that, ints are 31 bit long, to implement small object optimization for them.

                                    (*) of course, compiler can optimize and stack-allocate and SROA stuff, especially in SML land where closed world whole-program compilers are popular. But in general everything’s a pointer, and programmer doesn’t have control over that (like Java, and in contrast to go)

                                    1. 3

                                      actually i think (0.9 sure) you’re right too ;)

                                    2. 1

                                      I didn’t see anything which signified being able to set the pointer to an arbitrary memory address and access it

                                      If it doesn’t have this then it’s not low level at all…

                                      OTOH, many high level languages can do this, such as Haskell.

                          1. 3

                            Nothing fancy but this is the persistent rope that I’m using in my next text editor project.

                            (Said text editor is going to have something really unusual, but that’s not ready to be shown yet.)

                            1. 2

                              Nothing fancy

                              Yup! While it’s possible to go overboard with fancy rope implementations, a basic thing which gets the job done is surprisingly simple. IIUC, this 500 lines of verbose Java is what powers text in IntelliJ: https://github.com/JetBrains/intellij-community/blob/master/platform/util/base/src/com/intellij/util/text/ImmutableText.java.

                            1. 1

                              I really like this article, because it’s a marked departure from the usual approach to Rust tutorials of showing you several version of the program, none of which compile, then basically telling you how to solve each compiler error. Instead, it focuses on building a model that leverages Rust’s data access model to safely implement things that are tough to get right.

                              It’s a little unfortunate that the mechanism of the solutions isn’t explored in more depth. Interior mutability is nicely introduced as a good approach to solving the problem of initialisation but then it’s used via the once_cell crate.

                              That’s obviously understandable given what the article is based on and, furthermore, I think it’s actually the right approach “in production”. This isn’t intended as criticism of the post, but as an encouragement for the readers who liked it to dig a little into once_cell and elsa because both make for very instructive readings of idiomatic Rust.

                              1. 1

                                Yeah, “how once_cell works?” would make for a good post one day… There’s a surprising amount of details to get wrong there!

                              1. 1

                                Data-oriented thinking would compel us to get rid of deserialization step instead, but we will not pursue that idea this time.

                                I found this statement interesting, but I’m finding it pretty difficult to imagine what this would look like. Could someone with more knowledge of data-oriented thinking explain this?

                                1. 3

                                  So if we think about this, the db interface itself, where it returns a freshly-allocated Vec of bytes, is redundant. It necessary includes a copy somewhere, to move the data from bytes on-disk to Rust managed heap.

                                  It probably is possible to get at those bytes in a more direct way: most likely db either mmaps the file, or includes some internal caches. Either way, you probably can get a zero-copy &[u8] out of it. And it should be possible to “cast” that slice of bytes to &Widget, using something like arkyv or abonotation. That is, if Widget struct is defined in such way that bytes-in-the-form-they-are-on-disk are immediately usable.

                                  1. 2

                                    Or something like https://capnproto.org/

                                1. 6

                                  It removes redundancy, because the function name should already be in the call stack.

                                  That implies that the only time you ever look at a test is when it fails. I spend a lot of time trying to navigate test organization to write new tests and review code. There’s no stack trace, so the tests just look jumbled together.

                                  Test organization is one of the most frustrating topics, to me. It seems that every language/framework has a different approach to it, none are perfect, and it’s impossible to aggregate all the good ideas, due to technical difficulties. For example, there’s good aspects to the author’s idea, but I can’t use it in pytest because test names have to start with test_. I’ve largely broken down to using doc strings to explain tests, and I don’t love that that’s the best I can come up with.

                                  1. 5

                                    Test organization is one of the most frustrating topics

                                    Yes. I haven’t seen satisfactory solutions either. One thing which helps a bit are explicit coverage marks which sometimes help you to identify where tests for specific area live.

                                    1. 2

                                      Oh, the coverage marks concept seems interesting!

                                    2. 4

                                      in pytest […] test names have to start with test_

                                      FWIW this is configurable.

                                      1. 3
                                        [tool.pytest.ini_options]
                                        python_functions = "should_*"
                                        
                                      2. 2

                                        Jeff Forcier (the author of Invoke and the maintainer of Fabric and Paramiko) wrote a plugin for Pytest that lets you specify tests in this more literate fashion as well: https://github.com/bitprophet/pytest-relaxed

                                        It’s not quite as expressive in this regard as rspec is, but it makes for nice readable test output when it’s the right model for your tests.

                                        1. 1

                                          oh wow, i’ve wanted this so badly. installing immediately. between this and this, my comment has been a great use of time as i use both rust and python :)

                                        2. 1

                                          Completely agree. The idea tests should have fixed prefixes/suffixes is outdated.

                                        1. 18

                                          /me raises hand.

                                          I do need a “ascii gui” platform which I can use to build my personal workflow on top of over the years. But Emacs doesn’t feel good enough to invest significant time into:

                                          • architecturally, it’s pretty horrible with asynchrony – things routinely block GUI. I think this got better recently with the introduction of the new async API, but I don’t think it’s possible to proprely retro-fit non-blocking behavior throughout the stack
                                          • it’s pretty horrible with modularity – I want to pull other people’s code as self-contained, well-defined modules. VS Code extension marketplace is doing much better job at unleasing uncoordinated open-source creativity. I feel Emacs kits like spacemacs are popular because they solve “composition” problem unsolved in the core
                                          • it’s pretty horrible with modularity – elisp single global mutable namespace of things is good to build your tool by ourself, but is bad for building a common tool together
                                          • it’s pretty horrible with modulariy and asynchrony – poorly behaved third party code can easily block event loop (cf whith VS Code, where extensions run in a separate process, and host/plugin interface is async by construction)
                                          • elisp is a pretty horrible language – modularity, performance, lack of types, familiarity
                                          • extension APIs make complex things possible, but they don’t make simple things easy. To compare with VS Code again, it’s well-typed and defiend APIs are much easier to wrap your head around: https://github.com/microsoft/vscode/blob/main/src/vscode-dts/vscode.d.ts.
                                          • Emacs user features make complex things possible, but they don’t make simple things easy. 90% of configuration should be editing config file with auto-completion. Emacs customize is overly complicated and unusable, so you end up writing code for 100% of config, not 10%.

                                          Still, Emacs is probably closer to a good platform than anything else I know :-)

                                          1. 5

                                            it’s pretty horrible with modularity – I want to pull other people’s code as self-contained, well-defined modules. VS Code extension marketplace is doing much better job at unleasing uncoordinated open-source creativity.

                                            I think a big part of this, though, is that compared to Emacs, the extension framework in VS Code isn’t very powerful (i.e., it exposes much fewer internals), a fact that has led to bad performance for widely-used extensions which had to be solved by bringing them into core. If all you want to do is install extensions, the current Emacs package system works well enough. I do agree about the lack of namespaces; honestly, 90% of the things you mention would be fixed if Emacs were simply implemented in Common Lisp (including both modularity and asynchrony). The problem being that there is too much valuable code in Emacs Lisp.

                                            1. 4

                                              it’s pretty horrible with modularity – elisp single global mutable namespace of things is good to build your tool by ourself, but is bad for building a common tool together

                                              On the other hand, having all of Emacs itself be available is part of what makes it such a malleable substrate for one to build their own, customized editor out of Emacs. Having things locked down, isolated contexts, all that good “modern” stuff would certainly be nicer from a software engineering perspective, maybe make it easier to write reusable packages (although there are an awful lot of packages out there now), but would, I fear, kill what is so powerful about Emacs. The ability to introspect on just about every aspect of what the editor is doing, see the code, hook into it and change it is nearly unique and is so powerful on its own that (for good and ill) most else falls by the way-side.

                                              1. 3

                                                I kind of wish Emacs lisp had a module system that worked like CL modules. Everything the package exports, and which is intended for other user code to interact with, you access with my-package:my-variable. But you can still get at all the internals with my-package::my-variable.

                                                1. 2

                                                  Honestly, Emacs would be strictly better were it just written in Common Lisp. Same, RMS doesn’t like it, so Emacs Lisp stayed its own separate dialect.

                                                  But hey, at least it’s a Lisp!

                                              2. 2

                                                poorly behaved third party code can easily block event loop (cf whith VS Code, where extensions run in a separate process, and host/plugin interface is async by construction)

                                                This works rather well in practice. There have been a couple of bugs in vsc (fixed quite quickly) over the last few years where something caused the event loop to lock up because of the main thread getting blocked, and it’s actually surprising when it happens because it’s rare.

                                                1. 1

                                                  I do need a “ascii gui” platform

                                                  What would the ideal such platform look like?

                                                  1. 4

                                                    It’s hard!

                                                    I don’t think I am capable of designing one, but I have some thoughts.

                                                    1. We need some model for character-grid based GUI. Sort-of how html + dom + event handlers allow programming the web, but simpler and with the focus on rectangular, keyboard driven interactions. While the primary focus should be character grid, this thing should support, eg, displaying images for the cases you need that.

                                                    2. With GUI model, we should implement the core of the application, the event loop. “nothing blocks GUI” should happen there, extension management and isolation should happen here, window/frame management should happen here.

                                                    3. Given the core, we should implement some builtin components which provide the core value, scaffold for extensions, and in general dictate the feel of the platform. I think this should be an editor, an shell, and a window/frame/buffer manager. And “shell” here is not the traditional shell+emulator of an ancient physical device, but rather a shell build holistically, from the first principles, like the thing that Arcan folks are doing.

                                                    4. In parallel, there should be a common vocabulary of “widgets” for user interaction. Things like command pallet, magit-style CLI-flavored GUI dialogs, completion popups, configuration flow.

                                                    5. These all should be implemented on top of some sane software component model, with physical separation between components. For scalable, community-driven, long-lived system I think stability should take preference over ability to monkey-patch internals, so we really want something like OS + processes + IPC here, rather than a bunch of libraries in the same address space. Good news is WebAssembly is I think building exactly that software components model which we need here. WebAssembly components are still a WIP though.

                                                    6. A non-insignificant part of the component model is the delivery mechanism. I think what Deno does might work! Otherwise, some centralized registry of packages also a tested solution.

                                                    7. Another important aspect of component model is stability culture. Here I think two-tier model makes sense – core modules are very pedantic about their interfaces, and essentially do vscode.d.ts thing – perpetually stable, thoroughly documented and reified-in-a-single file interface. Modules you pull from the Internet can have whatever guarantees their authors are willing to provide.

                                                    8. Hope this system doesn’t get obsoleted by the world of user-faced computing finally collapsing into “everything is HTML” black hole.

                                                1. 1

                                                  This seems even worse than the builder pattern, which although reads nicely in blog posts is usually not really adding much in the way of safety or maintainability.

                                                  1. 3

                                                    What would you use to solve the problem of “there’s a new function with ten parameters which is called in ten different places each of which needs to be updated when parameters are added or removed (which, with ten of them, happens frequently)”?

                                                    1. 1

                                                      I’m not sure what you’re getting at exactly, so apologies if I’ve missed your point by a mile, but you have to change all the same call sites whether you’re constructing something with a single call that’s got 10 args or 10 calls chained together. Almost all places you see the builder pattern would be a single call to a provider/constructor in languages with keywords args.

                                                      1. 1

                                                        but you have to change all the same call sites whether you’re constructing something with a single call that’s got 10 args or 10 calls chained together.

                                                        Not necessary: if you add a new argument to new, you have to add it to every call-site. If you add a with_ method, only call-sites with non-default version of the argument need updating.

                                                  1. 1

                                                    This advice is specific to some languages like Rust and C++, where you can’t use field access syntax to invoke getter and setter functions. This limitation (of Rust and C++) requires you to decide up front whether a user defined type is “data” or “abstract”. Many other languages don’t have this limitation.

                                                    1. 1

                                                      Yeah, it needs to be reformulated for language with setters: “expose all setters or none”, where “exposing a setter” is making a field public or writing a public setter function.

                                                    1. 2

                                                      Isn’t this just setters?

                                                      1. 4

                                                        It is indeed a setter in disguise :) There are two differences though:

                                                        • it needs T rather than &mut T, which prevents using it for modification unless you own the data (you kinda still can do that via mem::replace, but that’s very visibly a hack)
                                                        • intended semantics/naming is “only use during construction”
                                                        1. 1

                                                          Ahhh now I get it. Very nice!

                                                          1. 1

                                                            If you’re going to expose these setter methods like this, why not make all of the fields public? Then you could use the struct initialization syntax and save yourself a lot of boilerplate.

                                                            If struct initialization is insufficient (for example, if you want to call a more complex mutator method during initialization), then the most general solution seems something like Clojure’s doto macro:

                                                            clojure.core/doto
                                                            ([x & forms])
                                                            Macro
                                                              Evaluates x then calls all of the methods and functions with the
                                                              value of x supplied at the front of the given arguments.  The forms
                                                              are evaluated in order.  Returns x.
                                                            
                                                              (doto (new java.util.HashMap) (.put "a" 1) (.put "b" 2))
                                                            
                                                            1. 2

                                                              Sure, if you could make all fields public, just do that! The Shape example is illustrative, not motivational.

                                                              I think such “functional update” isn’t equivalent to just making the field public:

                                                              • it prevents mutation after construction
                                                              • it allows not exposing the getter
                                                        1. 2

                                                          I’m a little worried about someone talking about aligning code when they chose to full-justify their blog.

                                                          I am less worried about how editors interpret tabs for indentation, since I believe that people who hate how an editor does that will change the behavior or change editors.

                                                          1. 6

                                                            Meh, in a single-column layout like that the lines are wide enough that full justification is fine IMHO, Butterick says basically “it’s an individual aesthetic choice” on that page you linked to.

                                                            I do worry about tabs/indents, because of reading or editing other people’s code. If they use spaces I’m stuck with their indentation preference, and some people think 2-space or 8-space indents are OK (they’re wrong.) If they use tabs I see the indentation I prefer, but any spacing within the line, like marginal comments, gets badly messed up. And lines that have extra indentation to match something on the previous line, like an open-paren, come out wrong.

                                                            This elastic system seems like a nice idea for within-line spacing. Maybe it can be combined with a syntax-driven rule that generates the leading indentation automatically when the file is opened; then the editor can just ignore leading whitespace. (Except in indent-sensitive languages like Nim and Python, obvs.)

                                                            1. 3

                                                              The problem is that one often has to work with others, and they have different preferences.

                                                              By separating the semantics of the code (indented blocks, columns of code/comments) from the presentation of the code (amount of whitespace added), you provide the option for each reader to see a presentation that is most pleasing to parse without forcing a everyone to align their editor settings.

                                                              1. 3

                                                                The most recent versions of clang-format finally support my preferred style: tabs for indentation, spaces for alignment. Each line starts with one tab for each indent level. If you have whitespace beyond that (or anywhere other than the start of a line) then it is spaces. This means that the comment in the motivating example in this article looks correct for any tab width, and even with tabs not set to a uniform, but consistent, value (for example, in C++, it would be great if my editor could use a single 2-space tabstop inside a namespace but 4-space tabstops everywhere else, including the second tabulator in a line inside a namespace). This doesn’t need the editor to do anything special and works fine even with cat.

                                                              2. 2

                                                                I’m a little worried about someone talking about aligning code when they chose to full-justify their blog.

                                                                Any links I can read about justification concern?

                                                                  1. 4

                                                                    To piggyback off this – I consider Butterick’s Practical Typography a must read. I’ve learned a ton and improved my written communication thanks to his guidelines.

                                                                  2. 3

                                                                    HTML/CSS has few, if any affordances for hyphenation, so setting text to be fully justified runs the risk of getting huge gaps in some lines if there’s a long word that can’t be hyphenated correctly.

                                                                    It’s not a dealbreaker for most but it can look a bit unprofessional if you’re unlucky.

                                                                    1. 5

                                                                      CSS has the hyphens property, which gives coarse control over hyphenation, set to auto (aka, use hyphens) by default.

                                                                      1. 3

                                                                        It’s also simple to make it gracefully degrade to left-aligned text in browsers that don’t support hyphens, via @supports (hyphens: auto). In the modern browser world, I see no valid reason not to enable justification for browsers that support it, at least not for websites with significant amounts of text.

                                                                1. 23

                                                                  RAII is far from perfect… here are a few complaints:

                                                                  1. Drop without error checking is also wrong by default. It may not be a big issue for closing files, but in the truly general case of external resources to-be-freed, you definitely need to handle errors. Consider if you wanted to use RAII for cloud resources, for which you need to use CRUD APIs to create/destroy. If you fail to destroy a resource, you need to remember that so that you can try again or alert.

                                                                  2. High-performance resource management utilizes arenas and other bulk acquisition and release patterns. When you bundle a deconstructor/dispose/drop/whatever into some structure, you have to bend over backwards to decouple it later if you wish to avoid the overhead of piecemeal freeing as things go out of scope.

                                                                  3. The Unix file API is a terrible example. Entire categories of usage of that API amount to open/write/close and could trivially be replaced by a bulk-write API that is correct-by-default regardless of whether it uses RAII internally. But beyond the common, trivial bulk case, most consumers of the file API actually care about file paths, not file descriptors. Using a descriptor is essentially an optimization to avoid path resolution. Considering the overhead of kernel calls, file system access, etc, this optimization is rarely valuable & can be minimized with a simple TTL cache on the kernel side. Unlike a descriptor-based API, a path-based API doesn’t need a close operation at all – save for some cases where files are being abused as locks or other abstractions that would be better served by their own interfaces.

                                                                  4. It encourages bad design in which too many types get tangled up with resource management. To be fair, this is still a cultural problem in Go. I see a lot of func NewFoo() (*Foo, error) which then of course is followed by an error check and potentially a .Close() call. Much more often than not, Foo has no need to manage its resources and could instead have those passed in: foo := &Foo{SomeService: svc} and now you never need to init or cleanup the Foo, nor check any initialization errors. I’ve worked on several services where I have systematically made this change and the result was a substantial reduction in code, ultimately centralizing all resource acquisition and release into essentially one main place where it’s pretty obvious whether or not cleanup is happening.

                                                                  1. 3

                                                                    This is super informative, thanks! Probably worth it to turn this comment into a post of its own.

                                                                    1. 3

                                                                      The error handling question for RAII is a very good point! This is honestly where I’m pretty glad with Python’s exception story (is there a bad error? Just blow up! And there’s a good-enough error handling story that you can wrap up your top level to alert nicely). As the code writer, you have no excuse to, at least, just throw an exception if there’s an issue that the user really needs to handle.

                                                                      I’ll quibble with 2 though. I don’t think RAII and arenas conflict too much? So many libraries are actually handlers to managed memory elsewhere, so you don’t have to release memory the instant you destruct your object if you don’t want to. Classically, reference counted references could just decrement a number by 1! I think there’s a a lot of case-by-case analysis here but I feel like common patterns don’t conflict with RAII that much?

                                                                      EDIT: sorry, I guess your point was more about decoupling entirely. I know there are libs that parametrize by allocator, but maybe you meant something even a bit more general

                                                                      1. 2

                                                                        It may not be a big issue for closing files

                                                                        from open(2):

                                                                        A careful programmer will check the return value of close(), since it is quite possible that errors on a previous write(2) operation are reported only on the final close() that releases the open file description. Failing to check the return value when closing a file may lead to silent loss of data. This can especially be observed with NFS and with disk quota.

                                                                        so it’s actually quite important to handle errors from close!

                                                                        The reason for this is that (AFAIK) write is just a request to do an actual write at some later time. This lets Linux coalesce writes/reschedule them without making your code block. As I understand it this is important for performance (and OSs have a long history of lying to applications about when data is written). A path-based API without close would make it difficult to add these kinds of optimizations.

                                                                        1. 1

                                                                          My comment about close not being a big issue is with respect to disposing of the file descriptor resource. The actual contents of the file is another matter entirely.

                                                                          Failing to check the return value when closing a file may lead to silent loss of data.

                                                                          Is that still true if you call fsync first?

                                                                          A path-based API without close would make it difficult to add these kinds of optimizations.

                                                                          Again, I think fsync is relevant. The errors returned from write calls (to either paths or file descriptors) are about things like whether or not you have access to a file that actually exists, not whether or not the transfer to disk was successful.

                                                                          This is also related to a more general set of problems with distributed systems (which includes kernel vs userland) that can be addressed with something like Promise Pipelining.

                                                                        2. 1
                                                                          1. Note that in the context of comparison with defer, it doesn’t improve the defaults that much. The common pattern of defer file.close() doesn’t handle errors either. You’d need to manually set up a bit of shared mutable state to replace the outer return value in the defer callback before the outer function returns. OTOH you could throw from a destructor by default.

                                                                          2. I disagree about “bend over backwards”, because destructors are called automatically, so you have no extra code to refactor. It’s even less work than finding and changing all relevant defers that were releasing resources piecemeal. When resources are owned by a pool, its use looks like your fourth point, and the pool can release them in bulk.

                                                                          3/4 are general API/architecture concerns about resource management, which may be valid, but not really specific to RAII vs defer, which from perspective of these issues are just an implementation detail.

                                                                          1. 1

                                                                            I disagree about “bend over backwards”, because destructors are called automatically, so you have no extra code to refactor

                                                                            You’re assuming you control the library that provides the RAII-based resource. Forget refactoring, thinking only about the initial code being written: If you don’t control the resource providing library, you need to do something unsavory in order to prevent deconstructors from running.

                                                                            1. 1

                                                                              Rust has a ManuallyDrop type wrapper if you need it. It prevents destructors from running on any type, without changing it.

                                                                              Additionally, using types via references never runs destructors when the reference goes out of scope, so if you refactor T to be &T coming from a pool, it just works.

                                                                            2. 1

                                                                              The common pattern of defer doesn’t handle errors either.

                                                                              In Zig you have to handle errors or explicitly discard them, in defer and everywhere else.

                                                                            3. 1

                                                                              Using a descriptor is essentially an optimization to avoid path resolution.

                                                                              I believe it also the point at which permissions are validated.

                                                                              Entire categories of usage of that API amount to open/write/close and could trivially be replaced by a bulk-write API

                                                                              Not quite sure I understand what you mean… Something like Ruby IO.write()? https://ruby-doc.org/core-3.1.2/IO.html#method-c-write

                                                                              1. 3

                                                                                also the point at which permissions are validated

                                                                                I think that’s right, though the same “essentially an optimization” comment applies, although the TTL cache option solution is less applicable.

                                                                                Something like Ruby IO.write()

                                                                                Yeah, pretty much.

                                                                            1. 3

                                                                              RAII comes with its own set of interesting footguns. Not to say that it’s a bad feature, but it’s not perfect. Languages that don’t employ RAII have a right to exist, and not just in the name of variety.

                                                                              1. 11

                                                                                That particular example is not a problem with RAII though, it is specific to API of shared_ptr and C++ flexible evaluation order.

                                                                                1. 5

                                                                                  This should be fixed in C++17, at least partially. TL;DR – see the “The Changes” section.

                                                                                  https://www.cppstories.com/2021/evaluation-order-cpp17/

                                                                                  1. 5

                                                                                    As that article points out, this is solved in C++11 with std::make_shared: any raw construction of std::shared_ptr is code smell. This kind of footgun is not really intrinsic to RAII, but to the way that C++ reports errors from constructors: the only mechanism is via an exception, which means that anything constructing an object as an argument needs to be excitingly exception safe. The usual fix for this is to have factory methods that validate that an object can be constructed from the arguments and return an option type.

                                                                                    The more subtle footgun with RAII is that it requires the result to be bound to a variable that is not necessarily used. In C++, you can write something like:

                                                                                    {
                                                                                      std::lock_guard(mutex);
                                                                                      // Some stuff that expects the lock to be held
                                                                                    } // Expect the lock to be released here.
                                                                                    

                                                                                    Only, because you didn’t write the first line as std::lock_guard g{mutex} you’ve locked and unlocked the mutex in the same statement and now the lock isn’t held. I think the [[nodiscard]] attribute on the constructor can force a warning here but I’ve seen this bug in real-world code a couple of times.

                                                                                    The root cause here is that RAII isn’t a language feature, it’s a design pattern. The problem with a defer statement is that it separates the initialisation and cleanup steps. A language that had RAII as a language feature would want something that allowed a function to return an object that is not destroyed until the end of the parent scope even if it is not bound to a variable.

                                                                                  1. 5

                                                                                    Lots of “meh” on this one. Congrats, you re-invented adding a templating language on top of Markdown, then wove it a bit deeper so parsing both at once leads to your AST but lost the programmatic features of most templating systems. Hum, so? Who does this help over an SSG like zola (or your favorite SSG) built on Markdown with a templating engine on top?

                                                                                    1. 2

                                                                                      Their templating language, at first glance, appears to be using Liquid syntax, is it just a set of Liquid macros?

                                                                                      The existence of things like this really highlights for me the big limitation of Markdown: there’s no way of extending the semantic markup easily. For headings, lists, and links, Markdown is fine. GitHub-flavoured Markdown has per-language code blocks, but not inline per-language code spans and there’s no good way of adding them.

                                                                                      In contrast, DocBook makes this kind of thing easy. For the FreeBSD Handbook, for example, there are custom XML tags for things like man page entries. The down side is that DocBook is not human readable or human writeable.

                                                                                      I’d love to see something a bit more in the middle. For my own books, I tend to use TeX-style markup (e.g. \cppcode{some C++ code}, \ccode{some C code}, \keyword{semantic markup}, with a regular syntax that I can either implement as LaTeX macros or parse to generate something else.

                                                                                      1. 3

                                                                                        I’d love to see something a bit more in the middle.

                                                                                        You are looking for AsciiDoc(tor) I suppose? It is essentially a DocBook, whose surface syntax is almost a super-set of markdow. Specifically, AsciiDoctor is plain-text lightweight concrete syntax for HTML-like DOM with arbitrary nested nodes with attributes. It’s not directly HTML though – to get HTML, you implement a mapping from AsciiDoctor’s DOM to HTML. Ditto for DocBook XML.

                                                                                        Syntax for nesting + syntax for attributes + user-controllable transformation steps give enough of flexibility to do whatever, without resorting to inserting raw HTML. And the surface syntax is rather tastefully design.

                                                                                        1. 1

                                                                                          AsciiDoc does look close, but it also looks pretty verbose for custom markup. The thing that I like about something TeX based is that it’s three extra characters on top of the name of the macro. If I want syntax-highlighted C++ text, I write \cppcode{virtual}. The macro name is cppcode, the only extra typing that I need to do is \, {, and }, to indicate the start of a macro name, and the start and end of the argument. With XML, I'd write something like virtualor possiblyvirtual`, which I wouldn’t want to type without an XML-aware editor (and which is annoying to read). I think AsciiDoc may be similar to TeX, but skimming the manual I couldn’t find how to write custom macros (the ‘inline macros’ section just tells me about the predefined ones).

                                                                                          1. 4

                                                                                            In AsciiDoctor, you’d write this as [.cppcode]`virtual` – this is built-in monospace with cppcode role attached. Or, if you want to decorate non-specific inline element, [.cppcode]#virtual#. Curiously, this won’t be a macro – cppcode would be attached as an attribute to the relevant inline element. It would be up to convertor into the specific output format to interpret this role.

                                                                                            AsciiDoctor also has macros (bits of code which are run during construction of dom during parsing). With a macro, that would look like cppcode:[virtual] (matches TeX in the number of characters!). macro surface syntax is a cute hack: in http://example.com[this is a link], the http: is a name of the macro which receives //exaple.com as an argument. And image:/path/to/file.png is an image, which is way easier to remeber than markdown syntax. Although I like macro syntax, I hate the semantics – I think I wish that everything were just inert attrs in the dom, and that all the logic were in the convertor from dom to a particular format.

                                                                                            TeX’s syntax does look nice for inline elements, but things like

                                                                                            \begin{itemize}
                                                                                              \item one
                                                                                              \item two
                                                                                              \item three
                                                                                            \end{itemize}
                                                                                            

                                                                                            are pretty horrible in comparison to

                                                                                            * one
                                                                                            * two
                                                                                            * three
                                                                                            

                                                                                            That’s I think is the reason I like asciidoctor – it has (admittedly, poorly specified) a sane general tree-shaped document model inside, but enough syntax sugar (well, maybe a bit too much) and syntactical variety on top of it to make authoring pleasant.

                                                                                            1. 1

                                                                                              In AsciiDoctor, you’d write this as [.cppcode]virtual – this is built-in monospace with cppcode role attached. Or, if you want to decorate non-specific inline element, [.cppcode]#virtual#. Curiously, this won’t be a macro – cppcode would be attached as an attribute to the relevant inline element. It would be up to convertor into the specific output format to interpret this role.

                                                                                              That does look pretty nice, thanks.

                                                                                              TeX’s syntax does look nice for inline elements, but things like [list examples]

                                                                                              The TeX example looks bad here but it has the nice property that it generalises. Itemised lists, enumerated lists, description lists, and any kind of user-defined collection can have the same syntax. The \begin{} / \end{} syntax is a bit verbose, but it’s used for so many things that I just have F2 in vim bound to a small macro that inserts a block with the token under the cursor used in the begin and end parts and switches to insert point with the cursor between the two.

                                                                                              1. 2

                                                                                                Asciidoctor also allows general tree-shaped things. For example, you can do something like

                                                                                                [my-list]
                                                                                                --
                                                                                                [item]
                                                                                                one
                                                                                                
                                                                                                [item]
                                                                                                two
                                                                                                
                                                                                                [item]
                                                                                                three
                                                                                                --
                                                                                                

                                                                                                This isn’t exactly equivalent to * (lists are first-class in the AST), but allows expressing arbitrary structure. Subjectively, AsciiDoctor scores high on make simple (common) things simple, and complex things possible. Though, it perhaps has too many general mechanisms for complex things.

                                                                                                1. 1

                                                                                                  Nice, thanks! It looks as if learning AsciiDoctor should be quite high up my TODO list.

                                                                                                  1. 1

                                                                                                    My current plan is to wait until standardization effort proceeds to a meaningful spec with the grammar: https://projects.eclipse.org/proposals/asciidoc-language. Quality-of-the-implementation is a weak link today (there’s essentially CPython situation – one dominating featureful impl, which leads to a lot of impl-defined corners). With the spec, I hope to see an embedable asciidoc parser, and a bigger variety of converters.

                                                                                        2. 1

                                                                                          My ducks aren’t quite in a row to write about this, but carpe diem…

                                                                                          There is at least one language here: D★Mark. It’s basically just the parser, but the author has at least one tool built atop it called MarkBook. The original implementation is in Ruby, and the author also has a Rust implementation (though I haven’t used that one). The author notes that its syntax is TeX-inspired.

                                                                                          I’ve wanted something similar for a while, so this Fall I bit the bullet and started prototyping something of my own (wordswurst) atop D★Mark (I wrote a Python parser for D★Mark in the process). I’m trying to scratch a specific itch: using a single documentation source to generate outputs for formats with differing whitespace semantics (~normal template languages fall over, here). To that end, I also want something that is both lightweight and as-purely-semantic as possible.

                                                                                          I’m not 100% certain wordswurst will use D★Mark in the long term, but it’s helped me focus while prototyping.

                                                                                          1. 1

                                                                                            D★Mark looks nice. For me, the big motivator was wanting semantic markup in code snippets. For the ePub editions of my second Objective-C book, I was able to use libclang to parse all of the example code and then use the clang token kinds as classes in the HTML so that I could use CSS to style local variables, class names, macros, and so on. This was motivated by the previous book, where the publisher had generated the HTML from the PDF that LaTeX produced and mangled a lot of the code.

                                                                                            I also had a lot of custom macros for things like keywords (should be italicised and appear in the index), or abbreviations (should appear as a cross-reference in the index and be expanded on first use in a chapter).

                                                                                            The clang integration isn’t something I’d expect to have out-of-the-box for any system, but it’s something that I’d like to be able to easily write a plugin for. Writing my own parser for the subset of TeX that I used for semantic markup was very easy (though I didn’t use TeX’s math mode for anything). As a side effect, libclang is reliable only with complete valid compilation units and so all of the code snippets in the book had to be pulled from a real source file, which guaranteed that they actually compiled and worked.

                                                                                            As we finalise the design for Verona, I’m going to need to start writing a Verona book soon and I’d like to be able to use more modern tooling (and not reinvent the wheel).

                                                                                      1. 5

                                                                                        This is a great testing approach! Without going into details, the bulk of tests indeed should be just data, with just a smidge of driver code on top.

                                                                                        Another important consideration is that the existence of a package for this is mostly irrelevant— you should just build it yourself (about 200 loc for MVP I suppose?) if there isn’t one for your ecosystem. It’s not a hidden gem, it’s more of an leftpad (considering the total size of tests you’d write with this).

                                                                                        One thing I would warn about here is that these specific tests, which work via process spawning, are quite slow. If you have hundreds/thousands of them, process spawning would be a major bottleneck. Unless the SUT spawns processes itself, you’d better of running a bunch of tests in process.

                                                                                        I think the example from the post shows this problem:

                                                                                        https://github.com/encoredev/encore/blob/main/parser/testdata/cron_job_definition.txt

                                                                                        Rather than invoking parse process via testscript and asserting stdout/stderr, this should have used just the txtar bit, and the parse function for an order of magnitude faster test.

                                                                                        Some real world experience with this problem:

                                                                                        • in Kotlin native, switching from many processes to single process for tests reduced the overall time massively (don’t remember the number)
                                                                                        • rustc testsuite is very slow due to this problem. However, as rustc build times are just atrocious due to suboptimal bootstrapping setup (both compiler and stdlib/runtime are magic code which need to be bootstrapped; a better setup is where compiler is just “normal Rust crate”), the testing time overall isn’t a pain point.
                                                                                        • cargo testsuite is very slow, but, as cargo is fundamentally about orchestrating processes, there isn’t much one can do there.
                                                                                        1. 5

                                                                                          Author of the post here. Note that “parse” here does invoke a function, not a binary. You have to specify “exec” to execute a subprocess.

                                                                                          1. 2

                                                                                            Ah, I see now. Yeah, then this is just the perfect setup!

                                                                                          2. 2

                                                                                            Another important consideration is that the existence of a package for this is mostly irrelevant— you should just build it yourself (about 200 loc for MVP I suppose?) if there isn’t one for your ecosystem. It’s not a hidden gem, it’s more of an leftpad (considering the total size of tests you’d write with this).

                                                                                            You seem to be saying (or implying) two different things here: (1) this is not a hidden gem: it’s like leftpad, and therefore you should write this yourself (the last clause is mine, but it seems—maybe?—implied by the first clause); (2) if this didn’t already exist, you could write it yourself. (2) seems fine, though—for the record—not true for many values of yourself. (I doubt I would write this myself, to be explicit.) (1) seems like a terrible case of NIH syndrome. This package is significantly more than 200 lines of code and tests. Why would I want to reinvent that?

                                                                                            Finally my recollection is that left-pad was four lines, maybe less. There’s simply no comparison between the two projects. (I checked, and the original version of left-pad was 12 lines: https://github.com/left-pad/left-pad/commit/2d60a7fcca682656ae3d84cae8c6367b49a5e87c.)

                                                                                            1. 1

                                                                                              You seem to be saying (or implying) two different things here: (1)

                                                                                              Tried to hedge against that reading with explicit “if there isn’t one for your ecosystem”, but apparently failed :-)

                                                                                              I doubt I would write this myself, to be explicit This package is significantly more than 200 lines of code

                                                                                              So this line of thinking is what I want to push back a bit, and the reason why I think that part of my comment is useful. It did seem to me a bit that this was presenting as a complicated black box which you are lucky to get access to, but implementing which is out of scope. I do want to say that it simpler than it seems, in the minimal configuration. the txtar part is splitting by regex and collecting the result in Map<Path, String>. The script part is splitting by lines, running each line as a subprocess and comparing results. This is simple, direct programming using basic instruments of the language/stdlib – no fancy algorithms, no deep technology stacks, no concurrency.

                                                                                              Now, if you make this the main driver for your testsuite, you’d want to add some fanciness there (smarter diff, colored output, custom commands). But such needs arising mean that you testsutie is using the tool very heavily. If you have 10_000 lines of tests, 1000 lines of a test driver are comparatively cheap. left-pad is small relative to its role in the application – padding some strings here and there. Test driver is small relative to its role in the application – powering majority of tests in a moment, supporting evolution of the test suite over time, and being a part of every edit-compile-test cycle for every developer.