Threads for ar-nelson

  1. 13

    https://danieljanus.pl/autosummarized-hn/previously/2022-08-02T08%3A00%3A25.227279Z.html

    “The article is informing the reader that they need to have JavaScript enabled in order to use twitter.com.”

    At least it made an attempt to summarize it?

    1. 4

      I really want to like this—it’s a lot better designed than Markdown—but nested sublists without blank lines between them are non-negotiable for me. That’s how I write all of my notes, in text files that are kinda-sorta-Markdown but never parsed. I get how the newline removes ambiguity, but it will constantly trip me up. Nested bulleted lists are my spacebar heating, I guess.

      1. 4

        Excellent, this is very close to something I was planning to attempt, you may have saved me a whole gang of time :)

        Question - in the personSchema example, is there a way I can extract and expose the type being described? Like export type Person = something<personSchema> ?

        1. 6

          Yes, the MatchesSchema type is described in the API but I didn’t include an example of using it this way. I probably should. It works like this:

          import { MatchesSchema } from 'spartan-schema';
          
          export type Person = MatchesSchema<typeof personSchema>;
          
        1. 3

          Well considering I had only just discovered Zod, this is really interesting.

          1. 2

            Interesting, I hadn’t heard of Zod before. I guess the main difference between it and Spartan Schema is that Zod is just a Typescript API (using function calls), whereas Spartan Schema is a JSON language with a Typescript library.

          1. 4

            Finally getting around to finishing my rewrite of Osmosis.

            It went through several rounds of potential rewrites, in a few different languages, with varying degrees of added complexity. I’ve been trying to solve two problems: portability (it should run on just about any platform) and flexibility (how to define a CRDT that can do enough things for every use case).

            What I finally settled on was to edit the existing Typescript version into something much more like Redux. Get rid of the Actions listed in the current README, and instead allow users to write their own reducers for their own actions. An embeddable version based on QuickJS (or JavaScriptCore) could be used in other languages, and it could even support writing reducers in the host language!

            This neatly sidesteps the “how many features should my CRDT support” problem, because the CRDT is now just a list of actions, garbage-collected once all nodes have seen all actions up to a point, which is easy to keep in sync.

            1. 5

              Looks useful, it’s a nice compromise between XML and something more like TOML. It seems very similar to Tree Notation; it could even be an alternate, indentation-insensitive way to write Tree Notation.

              I agree that SML is a bad name. Standard ML is a well-known language, and confusion is inevitable. You could keep “Simple Markup Language”, and use a different acronym, maybe “SiML”?

              1. 2

                I was going to make the same comment on Tree notation, although I find the unfortunately named SML clearer to understand based on its documentation and easier to read. I’d let go of the End, though.

              1. 2

                Finally getting multiplayer working in my game engine project, I hope.

                Ever since January I’ve been on a sabbatical from work, and I’ve used it to start a game project. Right now it’s a Terraria clone, but I plan to add a lot of features that I’ve never seen before in a procedural block-building game… once I finish the basics of making it a Terraria clone.

                I’ve been writing the engine from scratch in C and SDL. I knew that I had to get multiplayer figured out very early, because rewriting a singleplayer game into multiplayer can be nearly impossible. And I knew I wanted a sophisticated sync system–ideally, rollback netcode–to avoid the mistakes I’d seen in Starbound, which has a multiplayer implementation so naive that one slow client will bog down the whole server.

                I thought I’d have at least basic client-server multiplayer after 2-3 months, but it’s now been over 4 and I maybe, just maybe, will have a playable demo of a multiplayer server done this weekend. Network and multithreaded programming in C is hard!

                1. 2

                  It looks like this only has a TCP interface, not an in-process Rust API, but adding a Rust API would be very useful. I can think of several app ideas that could use this as a local, in-process search index, but it would be too cumbersome to run Sonic as a separate process when it could just be loaded as a library.

                  (Or is there a way to expose the TCP port to other functions in the same process, without binding a loopback port that other processes can access?)

                  1. 12

                    Is this browser going to be usable outside of SerenityOS? If it can be separated out as a library, in a few years this could finally be the new browser engine that breaks the WebKit/Gecko duopoly.

                    1. 4

                      I honestly doubt they’ll reach production quality, because that would require a lot more like sandboxing, speed & performance optimization etc. Maybe they’ll reach that, but I doubt they will reach the feature completeness of a chromium or gecko engine.

                      1. 1

                        I am pretty sure it already includes some sandboxing, it seems to do multiprocess tabs already.

                        1. 1

                          interesting, though what I meant with sandboxing is a little more sophisticated than spawning one worker per process

                          1. 3

                            They also use pledge and unveil to disable system calls and filesystem access in the sandboxes

                            1. 1

                              *one process per tab

                        2. 2

                          You forget Blink? The fork was in 2013, so it’s not exactly news.

                          1. 2

                            Blink/Gecko duopoly then :)

                          2. 1

                            I wouldn’t expect so, that seems to cut against the whole idea of the project.

                          1. 12

                            This is seriously, seriously impressive. The whole time I was reading “HOW.md”, I kept thinking it was too good to be true. There has to be a catch. And there is, kind of?

                            If a lambda that clones its argument is itself cloned, then its clones aren’t allowed to clone each-other.

                            For example, this term is not allowed:

                            let g = λf(λx(f (f x)))
                            (g g)
                            

                            That’s because g is a function that clones its argument (since f is used twice). It is then cloned, so each g on the second line is a clone. Then the first clone attempts to clone the second clone. That is considered undefined behavior, and a typed language that compiles to HVM must check that this kind of situation won’t happen.

                            Runtime undefined behavior that has to be detected and forbidden by a compiler. What I can’t tell from the documentation is: how difficult is it to statically prove that this can’t happen? How much of a burden will that be on compiler implementers? The documentation claims that the undefined behavior is a very rare situation that won’t happen unless you’re doing Deep Functional Magic, which is fine, so long as you can know it won’t happen in a given program.

                            This is something I’m mildly concerned about, but it doesn’t cancel out my enthusiasm. HVM is an idea with a lot of potential, and I can already see a lot of ways to use it in my own projects.

                            1. 4

                              Terms like this seem to rely on impredicativity. So I think one easy way a type system could rule this out is by ruling out impredicativity altogether (as does Hindley–Milner, for example), but that would also rule out some useful terms like those described in [1].

                              [1] Alejandro Serrano, Jurriaan Hage, Simon Peyton Jones, and Dimitrios Vytiniotis. 2020. A quick look at impredicativity. Proc. ACM Program. Lang. 4, ICFP, Article 89 (August 2020), 29 pages. DOI:https://doi.org/10.1145/3408971

                            1. 5

                              Kitten-proofing the apartment. We got two kittens this week, and every day they discover something new to get into.

                              Also, completely taking apart my NAS box and trying to figure out if it has a short, and where it is. I tried moving it (kitten-proofing), and, when I laid it on its side, its PSU died. It smelled like it completely burned out. So I bought a new PSU, and it worked briefly… until I flipped the tower on its side again, and now it won’t start anymore, again. Hopefully this one’s not dead.

                              1. 6

                                You can’t just mention kittens and not post a picture!

                                1. 1

                                  ^ this.

                                  cough up them pix, ar-nelson 😊

                              1. 12

                                The title of this article really buries the lede: it’s about building a self-modifying unit test library with Janet macros, which is actually really cool. Running a test generates a new file, with expected values replaced with actual ones. Writing tests this way is similar to working in a REPL.

                                Are there any other languages with a test system like this?

                                1. 10

                                  (author here)

                                  Probably! Lots of languages have some kind of snapshot test library, but I don’t know of any framework that works exactly like Judge. I assume that is due to ignorance, though; I can’t claim to have any particular knowledge of the domain.

                                  The .corrected file just-look-at-a-diff thing is taken straight from Jane Street’s expect test framework, which I think got the idea from Mercurial’s unified tests – see cram for a generic description. cram uses .err instead of .corrected, but whatever.

                                  But ppx_expect doesn’t serialize values or work on arbitrary expressions. The way it works is by redirecting stdout for the duration of the test, and by inserting string literals into the source code.

                                  This makes some sense in OCaml because there is not really a canonical way to serialize values – even though there is a de facto standard at Jane Street – so having the user produce strings is reasonable.

                                  (The stdout capture is also extremely useful, because it allows you to use literal printf debugging deep in your application code, and see the output show up as you run your tests. It’s also very useful to have tests that produce ASCII tables or whatever. It’s good for a lot of reasons, but of course it’s easy to do stdout redirection yourself, and you can easily implement the ppx_expect approach on top of the “view a value” primitives.)

                                  But apart from that difference, the workflow you see in the post matches the ppx_expect workflow very closely. Here are some other examples:

                                  But I don’t know what the ergonomics of those are like; I don’t know how easy it is to write tests in this REPLy-style using those libraries. The only one I’ve actually tried is k9, and while it did support “inline snapshots,” at least at the time it didn’t write .corrected files. It will still show you failing tests, but… I really like being able to bring my own diffs to the table. Let me use emerge or delta or vimdiff or whatever else.

                                  1. 2

                                    The first place I heard of this approach was Jane Street’s expect test framework for OCaml.

                                  1. 7

                                    I like the retro style of dithered images. It’s a stylistic choice with the side benefit of performance. Which is a nice change, given that most web design trends of the last decade or two have only increased file size. But it’s a stylistic choice first and a performance one second, since there are much less noticeable ways (WebP) to achieve similar compression.

                                    1. 2

                                      This is going to make Osmosis development much more difficult… I was already considering these two changes, but now I’m forced to make one or both of them:

                                      1. Use Bonjour for service discovery instead of a custom protocol. (I don’t know the tradeoffs here, but most Bonjour libraries seem complex and outdated.)
                                      2. Make Osmosis a separate app that other apps communicate with via IPC (or a localhost HTTP port), rather than a library to be included in other apps.

                                      iOS is always the most complex, limiting part of cross-platform app development, but it has too large of an install base to ignore. :/

                                      1. 3

                                        Bonjour is pretty simple to implement IIRC, and not really proprietary. Go that way.

                                        1. 1

                                          For service discovery NSNetService works fairly well. mDNS just isn’t that dependable under packet loss scenarios.

                                          1. 1

                                            It looks like NSNetService is deprecated, does that matter? And is there a cross-platform library for it?

                                          2. 1

                                            #2 isn’t going to work because iOS doesn’t support arbitrary background apps.

                                            Using mDNS (Bonjour) is probably the right way to go. It generally works and where it doesn’t your custom protocol is probably also blocked or broken.

                                            You could support iOS devices by going via a server rather than p2p I guess.

                                            1. 1

                                              Hmm… I don’t have any experience with iOS apps, and this kind of information seems hard to find, but it looks like there might be loopholes for background apps? Based on some quick Googling, it seems like iOS apps are allowed to run continually in the background if they use the Location or Bluetooth permissions. And one (long shot) idea I had for Osmosis was to allow Bluetooth as another transport for p2p sync, to sync devices without a router present. So maybe the app could cheat by requiring Bluetooth LE to be on even if it’s not actively using it, allowing a server to run in the background…

                                              Of course, an app that exists only as a service to be used by other apps is pretty weird in the mobile space anyway, so Apple reviewers might not allow it no matter how I set it up.

                                              1. 1

                                                In my (limited) experience background access is quite tightly controlled. It would probably be easier to have apps that use Osmosis apply for multicast access from Apple.

                                          1. 3

                                            I tried it out briefly, as a terminal font. A few observations:

                                            • The alphanumeric characters are beautiful, and I wish there were more programming fonts in this style. I already use Go Mono as my default programming font, for similar reasons (aesthetic, and I find serif fonts more readable).
                                            • Unfortunately, it’s not well suited to dark themes. The font contains some extremely thin lines, which look decent as black-on-white text but become hard to read as white-on-black. Maybe this is a font rendering problem? (I tried it in iTerm 2 on a Mac).
                                            • The non-alphanumeric characters are not well-suited to programming. []{} in particular are very hard to see, even at large font sizes, and look more like lines than brackets/braces. These probably need to be completely redesigned if this is going to work as a programming font.
                                            1. 1

                                              Maybe this is a font rendering problem? (I tried it in iTerm 2 on a Mac).

                                              I can’t replicate your setup (no Mac) but I took a comparison with vscode, and the font seems to look alright light-on-black. In fact, I’d say it even looks a bit better. But I like big fonts https://i.imgur.com/pxPY4cI.png https://i.imgur.com/Vw2FGnA.png

                                              I’m going to leave up as default for at least two weeks, nice find, OP!

                                            1. 3

                                              Wouldn’t it make more sense to display iframe alerts inside of their iframes? Render them as though they’re a <dialog> inside of that iframe’s <body>, and that will prevent embedded ads from displaying scam popups, while leaving some use cases (CodePen) unchanged.

                                              It would still break code that’s using invisible iframes to display alerts, but I’d imagine that’s a much smaller proportion of legitimate web apps.

                                              1. 10

                                                What impresses me most about this language is its syntax. There are so many small, clever innovations that I’d love to see more widely used. For example:

                                                • Simple macro expressions: IO { ... } makes everything inside the { ... } a do expression in the context of the IO monad.

                                                • Sequenced let expressions that just look like variable assignments. (The only other place I’ve seen this is ReasonML)

                                                • C-style function calls that are actually curried.

                                                • Using ! to infer arguments (usually type arguments), making inference explicit without boilerplate.

                                                • Lots of syntax sugar for common FP patterns: <- functional-update setters, <> as shorthand for Option.getOrElse, and an abort operator that’s similar to Rust’s ?.

                                                • And, most importantly: the cleanest, most intuitive pattern matching syntax I’ve seen.

                                                  What would be opt match { case Some(x) => x; case None => "foo" } in Scala becomes something like case opt { some: opt.value; none: "foo" } in Kind. (That might not be the right names for Kind’s option type.) It knows that opt is a variable and that its type can be narrowed, so in the some branch, its type is some and opt.value is accessible. There’s no destructuring, no ordered arguments, and no special pattern sublanguage.

                                                1. 5

                                                  I completely agree. Having more ways of making the whole milieu of FP/dependent types accessible is an area that I think is underexplored. Rust starts down this path, but I don’t particularly think it’s as ‘nice’ as it could be.

                                                  And accessibility is the game. Rust does well enough by having freaking phenomenal compiler errors. You can learn by doing. Go goes the other way, accessibility by minimalism.

                                                  It’s an art, not a science.

                                                1. 3

                                                  So this raises the question: is it possible to implement SQLite-on-bare-metal as a filesystem driver? Would there be any performance benefit? I’m pretty sure there’s already a SQLite FUSE driver, but that still requires a file inside another filesystem to contain the DB.

                                                  1. 4

                                                    You could use SQLite’s virtual-filesystem API to wire up a raw block-storage device as a back end for a database. It would be a bit faster than going through the filesystem, dunno how much, and of course it depends on the filesystem and the OS.

                                                    But by putting FUSE on top of that you’d just be reintroducing the exact same causes of slowdown that made the filesystem slower in the first place.

                                                    The reason SQLite is faster at reading blobs is that it doesn’t have to allocate heavyweight data structures like a file descriptor, nor go through extra system calls to open/close them.

                                                  1. 4

                                                    I’ve been working on a similar project, Osmosis. I’m currently working on a rewrite that adds strong typing (based on a simplified JSON schema language) to make it truly conflict-free.

                                                    The main thing Osmosis adds on top of Automerge or Y is a full network stack, using UDP service discovery to find other copies of an app on the same network and automatically sync in the background.

                                                    1. 2

                                                      The pairing stuff sounds cool! Is there something about your service discovery layer that makes it difficult to re-use Yjs or Automerge for the data layer?

                                                      1. 2

                                                        Honestly I started working on it before I was aware of Automerge or Y.js, and so I haven’t seriously considered reworking it to replace my own CRDT with one of those.

                                                        I haven’t looked deep enough into other CRDTs to know if they fulfill this requirement, but one of my goals for Osmosis is to support at-rest encryption, and the CRDT was designed to make this easy. For example, the Osmosis CRDT can be easily stored in a LevelDB-like key-value store: each action is stored at a key named with the action’s ID (UUID + Lamport number), while the JSON structure itself is split up into one entry per node, where the keys are the node paths (a sequence of integers and/or strings, encoded in MessagePack). Queries and insertions use JsonPath, and the JsonPath parser emits a set of these binary path strings. And this (in theory) makes at-rest encryption really easy: path strings can be hashed (with a unique salt), values and actions can be encrypted with a key, and the action IDs can be left unencrypted.

                                                        This allows an app like a password vault to run in “locked” mode by default (without the encryption key in memory), but still receive and propagate encrypted CRDT actions. As soon as it’s unlocked, it can apply all newly-received actions to the database.


                                                        Another thing the Osmosis CRDT was designed for: an extremely uniform JSON-based API. I expect the final version to be usable and callable from several different languages. So I tried to base the entire API around JSON, JsonPath, and subscribing to query events. Osmosis’s internal state (“metadata”) is treated as a read-only part of the database, and is accessible using the same API. So, if you want to ask a question like “which peers am I connected to?” or “which actions were performed by peer X?” or “what was the state of the database 3 days ago?”, you can get this information through JsonPath queries that look exactly like queries for ordinary data.

                                                        1. 2

                                                          Sounds like a good set of principles! I look forward to the fruits of your effort.

                                                    1. 3

                                                      How on earth are you supposed to pronounce “≱”?

                                                      1. 1

                                                        “not greater than or equal”?

                                                        1. 1

                                                          In my head I was just calling it “blob”, given that it’s supposed to be an opaque symbol to begin with.