1. 13

    Copper is the language used to implement that same author’s excellent Code Browser folding text editor.

    Copper’s predecessor language, Zinc has the unusual property of allowing whitespace in identifier names, which makes for a very readable language once you get the hang of it.

    1. 7

      Yes, I found out about it in this thread: https://news.ycombinator.com/item?id=25561688

      It’s pretty hidden! Big piece of work with apparently no source repo, just tarballs.

      Reminds me a little of Virgil, although this language has a source repo and paper. It’s a very significant self-hosted compiler.

      https://github.com/titzer/virgil

      1. 3

        has the unusual property of allowing whitespace in identifier names

        this was big in certain Algol dialects, and was one of the interesting properties that Algol (and language experiments like ISWIM) were separated from their representations (like the ISWIM paper defines “physical,” “abstract,” “logical,” and the intermediary representation in what Landin called “applicative expressions.”

        It’s neat because Algol dialects had all sorts of tricks to delineate keywords from identifiers to allow spaces in names or more free-form naming. From some of the ones I’ve seen, there have been dots before keywords (like .INT and .MODE), just capitalizing, special character sets… it was a super fascinating time. It’s also neat to see what languages like PL/I and COBOL allow with just - within reserved names, it’s pretty nice

        1. 3

          I don’t know if this is a SQL standard thing, or a MS SQL Server/T-SQL extension, but in that dialect one can have table- and field names with whitespace as long as they’re enclosed in square brackets.

      1. 1

        This is really interesting; Clint just spoke at a conference I ran (direct link to his time slot, we’ll slice his video out this week too), and it’s interesting to see what the r2c folks are doing

        1. 3

          I have two computers setups for what I call “ascetic computing,” which are minimal interaction machines; they:

          • don’t have slack
          • aren’t logged into lobste.rs or twitter generally
          • aren’t used for banking or other things I need to do
          • generally aren’t logged into anything

          I just use git, whatever compilers I need, and the man pages for the most part. Getting a good workflow with my editor & version control system was pretty big, and being disciplined enought to make sure I pull before starting to code and push after I come back is key, but otherwise it’s pretty natural for me.

          It really reminds me of when I was young, and would code just by man pages and local version control…

          1. 5

            I don’t believe the core point is language-specific at all.

            I disagree with this thesis. In particular, I think that there are some properties of Rust and Haskell which are not universal among languages:

            • I/O can block
            • Exceptions can be inspected by whoever catches them
            • Exceptions can be masked
            • Actions cannot be deferred to a later time

            I’d like to reinspect these claims with a language which forces I/O to be distinct from computation, hides exception information from unauthorized catchers, and has syntax for deferring actions until some later time. Dart, Deno, or Go might be reasonable mainstream languages; I’m going to use Monte, my flavor of E.

            First, since I/O cannot block, how do we read from files? Monte has an unsafe object, makeFileResource, which constructs objects which have a method, .getContents(), which returns a promise for a bytestring. So, putting this together, we have:

            ⛰  def bs := makeFileResource("yes.txt")<-getContents()
            Result: <unsafely-printed promise>
            

            (These examples are at the REPL, which has access to unsafe objects and can print promises too quickly for the runtime. Error handling is not just hard, but fractally hard.) If we wait a moment, and then ask the REPL:

            ⛰  bs
            Result: b`test$\n`
            ⛰  bs.size()
            Result: 5
            

            Then it seems that there is a test file which has some bytes in it. (And a bit of Monte’s philosophy about syntax; that hard-to-see newline is decorated to make it obvious, but it is a plain old \n newline.)

            If you’re Rust-fluent, that .unwrap() may stick out to you like a sore thumb. You know it means “convert any error that occurred into a panic.” And panics are a Bad Thing. It’s not correct error handling.

            Similarly, if one is E-fluent, then the direct usage of this promise sticks out quite badly. We know that it is like, “convert any error which occurred into a broken promise.” And broken promises are a Bad Thing if we try to use them directly; errors aren’t handled. Suppose we open a non-existent file:

            ⛰  def err := makeFileResource("no.txt")<-getContents()
            Result: <unsafely-printed promise>
            ⛰  err
            Result: <ref broken by Couldn't open file fount for /home/simpson/typhon/no.txt: no such file or directory (ENOENT)>
            ⛰  err.size()
            Exception: Couldn't open file fount for /home/simpson/typhon/no.txt: no such file or directory (ENOENT)
            File '/nix/store/8s05yq4h1cx6w9nf20yvigcsbir57hv9-mast-full/prelude/m.mast' 41:30::42:80, in object _$eval:
              <eval>.evalToPair(m`err.size()`, ["&&null" => <binding <FinalSlot(null)>…)
            …
            

            The REPL is saving us from ourselves on a direct access to a broken promise, mostly for legibility. But when we try to invoke a behavior on the promise by calling a method, then an exception is thrown. (I’ve omitted most of the traceback; you can see that the most recent frame is where the REPL calls eval.)

            Let’s see the beautiful error message generated by my program: … Huh… that’s thoroughly unhelpful.

            Yes. This is the worst part of error handling: Errors must be legible to the computer, so that the computer can recover algorithmically and automatically; but also legible to the programmer, so that the programmer can imagine possible failures and remediations in the mechanisms of the code. And it’s never good enough.

            Finally, since I implicitly claim that Monte’s error handling is better than Rust’s or Haskell’s, let’s see how to implement graceful high-level control-flow for promises, along the lines of the examples with the error-context libraries. These sorts of libraries aren’t possible in Monte due to how modules are isolated, so let’s hope that what’s built-in is sufficient!

            ⛰  def finish(p) { return when (p) -> { `Opened file, got ${p.size()} bytes` }
            …                   catch problem { `Couldn't open file: $problem` } }
            Result: <finish>
            ⛰  def p1 := finish(bs)
            Result: <unsafely-printed promise>
            ⛰  def p2 := finish(err)
            Result: <unsafely-printed promise>
            ⛰  p1
            Result: "Opened file, got 5 bytes"
            ⛰  p2
            Result: "Couldn't open file: Couldn't open file fount for /home/simpson/typhon/no.txt: no such file or directory (ENOENT)"
            

            when blocks are the powerhouse of E. In some later period of computation (some later “turn”), once the given promises are resolved enough, then the first clause runs; however, if the promises break, then the second clause is run. The block evaluates to a promise itself, allowing for promise chaining and promise pipelining.

            I should point out that there are libraries which bring this power to other languages. I have personally used Haskell’s async, and I believe that similar crates are available for Rust. However, their usage is not automatically hygienic, and care is still required to handle the caveats I listed at the beginning.

            I’ll continue with my general advice of using your language’s preferred mechanisms for error handling.

            Excellent conclusion. In Monte, use ejectors and catch blocks, try to avoid unsealException or other unsafe exception handlers, and use broken promises to propagate asynchronous errors, and ABCDEF: Always Be Calling Deferred Ejector “FAIL”.

            1. 3

              I’m going to use Monte, my flavor of E.

              this is a great post, but excuse me? WHAT? You have a dialect of E floating around? that’s amazing! I’ve experimented with E and such, as well as W7, in my Scheme dialect. It’s really great to see someone experimenting! If you’re curious like me, I think it’s this page

              1. 3

                Thanks! Our living documentation is at monte.rtfd.io, and we have #erights and #monte on Freenode. Our progress is very slow; we are hesitant to make wrong steps and add too much authority.

                Recently there is something of a confluent renaissance in CapTP, the capability transport protocol. Goblins, Agoric, and Monte have all recently demonstrated various flavors of working CapTP, and we are all slowly coalescing around using Capn Proto RPC as common lingua franca.

                1. 2

                  This is super interesting; I’ve seen CapTP from some client work, as well as Capn Proto, but it’s very interesting to see it coalesce as a lingua franca!

                  If someone (e.g. me) wanted to help, where would they get started?

                  1. 2

                    Right now it would be good to both get a sense of the technical portions of CapTP and the current politics, like here and here. Idling in #erights or reading cap-talk will be good for talking to folks.

                    Your main goal should be to find a capability-aware community which fits your needs. There are many different groups, and you shouldn’t feel pressure to find a flavor of E which you like.

                    If you want to contribute to Monte specifically, then read our docs and idle in #monte. There is much to do.

              2. 3

                I/O can block

                GHC Haskell can only have blocking IO when using the FFI. Not saying this assuming you don’t know, but in case the direct comparison to Rust confuses another reader without this clarification. Rust I/O is blocking by default. Haskell I/O is non-blocking by default.

                1. 2

                  This is an excellent point and helps expose how deep the language-design differences can go. In particular, Monte doesn’t have FFI, and one reason is because FFI would have to be run in a very special way in order to not block anything else. (And, of course, FFI can still break other various runtime invariants at will. It’s simply too powerful to load Somebody Else’s Code into an address space with C linkage and then transfer control to it.)

                  We are also likely nuanced on what blocking means. Distilling an example from Coroutines Reduce Readability, let us consider a Python class which has a transactional behavior; its method meth wants to first enter a critical section, and then call its argument. In the following straight-line Python code, it is possible for our object self to fail its internal invariants precisely when c is a coroutine:

                  def meth(self, c):
                      self._beginCriticalSection()
                      self.status = c()
                      self._endCriticalSection()
                  

                  This same problem can occur in Haskell and Rust. It might not be obvious, but in an extended example in Java, we can see the problem. We call this class of bugs plan interference, and it is possibly the most important thing addressed by E. In Monte, it is not possible for c() to block indefinitely but also for meth to be called again, unless c itself is re-entrant. (Another flavor of E, Joule, works around this caveat by requiring most method invocations to create promises and delay their work, as if everything were wrapped in when-blocks. Joule’s core language is strange and elegant.)

              1. 40

                Something other than “everything is bytes”, for starters. The operating system should provide applications with a standard way of inputting and outputting structured data, be it via pipes, to files, …

                Also, a standard mechanism for applications to send messages to each other, preferably using the above structured format when passing data around. Seriously, IPC is one of the worst parts of modern OSes today.

                If we’re going utopic, then the operating system should only run managed code in a abstract VM via the scheduler, which can provide safety beyond what the hardware can. So basically it would be like if your entire operating system is Java and the kernel runs everything inside the JVM. (Just an example, I do not condone writing an operating system in Java).

                I’m also liking what SerenityOS is doing with the LibCore/LibGfx/LibGui stuff. A “standard” set of stuff seems really cool because you know it will work as long as you’re on SerenityOS. While I’m all for freedom of choice having a default set of stuff is nice.

                1. 21

                  The operating system should provide applications with a standard way of inputting and outputting structured data, be it via pipes, to files

                  I’d go so far as to say that processes should be able to share not only data structures, but closures.

                  1. 4

                    This has been tried a few times, it was super interesting. What comes to mind is Obliq, (to some extent) Modula-3, and things like Kali Scheme. Super fascinating work.

                    1. 3

                      Neat! Do you have a use-case in mind for interprocess closures?

                      1. 4

                        To me that sounds like the ultimate way to implement capabilities: a capability is just a procedure which can do certain things, which you can send to another process.

                        1. 5

                          This is one of the main things I had in mind too. In a language like Lua where closure environments are first-class, it’s a lot easier to build that kind of thing from scratch. I did this in a recent game I made where the in-game UI has access to a repl that lets you reconfigure the controls/HUD and stuff but doesn’t let you rewrite core game data: https://git.sr.ht/~technomancy/tremendous-quest-iv

                      2. 1

                        I would be interested in seeing how the problem with CPU time stealing and DoS attacks that would arise from that could be solved.

                      3. 17

                        Digging into IPC a bit, I feel like Windows actually had some good stuff to say on the matter.

                        I think the design space looks something like:

                        • Messages vs streams (here is a cat picture vs here is a continuing generated sequence of cat pictures)
                        • Broadcast messages vs narrowcast messages (notify another app vs notify all apps)
                        • Known format vs unknown pile of bytes (the blob i’m giving you is an image/png versus lol i dunno here’s the size of the bytes and the blob, good luck!)
                        • Cancellable/TTL vs not (if this message is not handled by this time, don’t deliver it)
                        • Small messages versus big messages (here is a thumbnail of a cat versus the digitized CAT scan of a cat)

                        I’m sure there are other axes, but that’s maybe a starting point. Also, fuck POSIX signals. Not in my OS.

                        1. 5

                          Is a video of cats playing a message or a stream? Does it matter whether it’s 2mb or 2gb (or whether the goal is to display one frame at a time vs to copy the file somewhere)?

                          1. 2

                            It would likely depend on the reason the data is being transferred. Video pretty much always fits into the ‘streaming’ category if it’s going to be decoded and played, as the encoding allows for parts of a file to be decoded independent of the other parts. Messages are for atomic chucks of data that only make sense when they’re complete. Transferring whole files over a message bus is probably a bad idea though, you’d likely want to instead pass a message that says “here’s a path to a file and some metadata, do what you want with it” and have the permissions model plug into the message bus so that applications can have temporary r/rw access to the file in question. Optionally, if you have a filesystem that supports COW and deduplication, you can efficiently and transparently copy the file for the other applications use and it can do whatever it wants with it without affecting the “original”.

                            1. 5

                              Which is why copy&paste is implemented the way it is!

                              Many people don’t realize but it’s not actually just some storage buffer. As long as the program is running when you try to paste something the two programs can talk to each other and negotiate the format they want.

                              That is why people sometimes have odd bugs on linux where the clipboard disappears when a program ends or why Powerpoint sometimes asks you if you want to keep your large clipboard content when you try to exit.

                        2. 13

                          Something other than “everything is bytes”, for starters. The operating system should provide applications with a standard way of inputting and outputting structured data, be it via pipes, to files, …

                          It’s a shame I can agree only once.

                          Things like Records Management Services, ARexx, Messages and Ports on Amiga or OpenVMS’ Mailboxes (to say nothing of QIO), and the data structures of shared libraries on Amiga…

                          Also, the fact that things like Poplog (which is an operating environment for a few different languages but allows cross-language calls), OpenVMS’s common language environment, or even USCD p-System aren’t more popular is sad to me.

                          Honestly, I’ve thought about this a few times, and I’d love something that is:

                          • an information utility like Multics
                          • secure like seL4 and Multics
                          • specified like seL4
                          • distributed like Plan9/CLive
                          • with rich libraries, ports, and plumbing rules
                          • and separated like Qubes
                          • with a virtual machine that is easy to inspect like LispM’s OSes, but easy to lock down like Bitfrost on one-laptop per child…

                          a man can dream.

                          1. 7

                            Something other than “everything is bytes”, for starters. The operating system should provide applications with a standard way of inputting and outputting structured data

                            have you tried powershell

                            1. 4

                              or https://www.nushell.sh/ for that matter

                            2. 4

                              In many ways you can’t even remove the *shells from current OS’s IPC is so b0rked.

                              How can a shell communicate with a program it’s trying to invoke? Array of strings for options and a global key value dictionary of strings for environment variables.

                              Awful.

                              It should be able to introspect to find out the schema for the options (what options are available, what types they are…)

                              Environment variables are a reliability nightmare. Essentially hidden globals everywhere.

                              Pipes? The data is structured, but what is the schema? I can pipe this to that, does it fit? Does it make sense….? Can I b0rk your adhoc parser of input, sure I can, you scratched it together in half a day assuming only friendly inputs.

                              In many ways IPC is step zero to figure out. With all the adhoc options parsers and adhoc stdin/out parsers / formatters being secure, robust and part of the OS.

                              1. 3

                                I agree wholeheartedly with the first part of your comment. But then there is this:

                                If we’re going utopic, then the operating system should only run managed code in a abstract VM via the scheduler, which can provide safety beyond what the hardware can.

                                What sort of safety can a managed language provide from the point of view of an operating system compared to the usual abstraction of processes (virtual memory and preemptive scheduling) combined with thoughtful design of how you give programs access to resources? When something goes wrong in Java, the program may either get into a state that violates preconditions assumed by the authors or an exception will terminate some superset of erroneous computation. When something goes wrong in a process in a system with virtual memory, again program may reach a state violating preconditions assumed by the authors, or it may trigger a hardware exception, handled by the OS which may terminate the program or inform it about the fault. Generally, it all gets contained within the process. The key difference is, with a managed language you seem to be sacrificing performance for an illusory feeling of safety.

                                There are of course other ways programs may violate safety, but that has more to do with how you give them access to resources such as special hardware components, filesystem, operating system services, etc. Nothing that can be fixed by going away from native code.

                                No-breaks programming languages like C may be a pain for the author of the program and there is a good reason to switch away from them to something safer, in order to write more reliable software. But a language runtime can’t protect an operating system any more than the abstractions that make up a process, which are a lot more efficient. There are of course things like Spectre and Meltdown, but those are hardware bugs. Those bugs should be fixed, not papered over by another layer, lurking at the bottom.

                                Software and hardware need to be considered together, as they together form a system. Ironically, I may conclude this comment with an Alan Kay quote:

                                People who are really serious about software should make their own hardware.

                              1. 4

                                It’s impressive how much StandardML has resisted extensions over the years, especially in a world where people believe that adding a feature improves a language.

                                I’ll always be fond of ML and ML-like languages (except Ocaml and Reason).

                                1. 3

                                  SML/NJ (probably the most used, most popular) does have a bunch of extensions and MLton has adopted some of them.

                                  But there’s nothing like camlp4.

                                  1. 2

                                    I’ll always be fond of ML and ML-like languages (except Ocaml and Reason).

                                    Why the exclusion?

                                    1. 1

                                      Lack of taste.

                                      1. 3

                                        You could also call that “pragmatism”. Look at the respective size of communities :-)

                                        I find OCaml mostly tasteful. Compared to SML it has better pattern matching and applicative functors, for example.

                                        1. 2

                                          I have to agree; as much as I like SML, it’s not always the easiest to work with. iirc, Okasaki even mentions this in some commentary about Purely Functional Data Structures, that he had to flub some of the SML code because it wasn’t working the way he needed it to, and Haskell was easier to work with.

                                          Personally, I prefer F# to OCaml proper, but either is probably fine. There’s definitely a siren’s song for the simplicity that SML provides, and ML for the Working Programmer was definitely a foundational book in my thinking.

                                    2. 2

                                      how much StandardML has resisted extensions over the years

                                      This is discussed in section 9.3 of the HOPL IV paper from this year; Milner and Tofte expressly stated in a mailing list post from 2001 that there would be no further revisions to the Definition, and that any future work would not be “Standard ML”.

                                      An amusing (but entirely relatable!) quote from an interview with Milner in 2010 suggests that even from the early days, there was a desire to prevent too much meddling:

                                      By the way we called it `Standard’ ML to prevent some organisation stepping in to standardise it.

                                      1. 1

                                        Look up the successor ml group on github. Many existing SML compiler authors are definitely interested in a next version of direct descendent.

                                    1. 2

                                      Interesting post, and really neat to see so much F#!

                                      I’d probably recommend adding the tags +ml and +dotnet as well.

                                      Wrt the crypto, I’d probably recommend Argon2id over BCrypt, as it is more modern and resistant than BCrypto itself, and I’d definitely avoid RSA as much as possible. Also, many voting systems follow NIST 800 series, NIST 1800 series, and things like NIST IR 7711 and the various UOCAVA studies. Because of this, you may be limited to things like PBKDF2 and such as well.

                                      Additionally, these are things we discussed in the Voatz Security Review and Threat Model. It’s definitely an interesting space, I wish we could see more innovation in it.

                                      Also, if you’re interested in this space too, looking at how the Swiss Post voting system was broken is fascinating, as well as things like the Moscow Parliamentary elections flub with ElGamal

                                      1. 2

                                        Neat. Great stuff here.

                                        15-20 years ago, I coded a secure voting system for the Board of Governors of the Federal Reserve. You know, the guys that set interest rates. Back then it was all bubblegum and bailing wire. But hell, it was better than Lotus Notes, which it replaced.

                                        I appreciate this update. If I continue to play with the idea I’ll look into this information.

                                        I’m a conceptual/intuitive thinker. That works great for some things and not others. For stuff like crypto, it pays to be the other way, both detail-oriented and methodical. There are a lot of really cool technical folks out there that are working in this space, mostly because of crypto-currencies. Things like information leakage or SIGINT can come at you from so many dimensions that it is impossible to track it all, even conceptually. The old joke is why go to all of the trouble to hack a guy’s system when you can just stick a cam in his office?

                                        Frankly I love this space. I also love the AI/ML/GAN stuff I’m seeing. So many areas of cool tech and so little time!

                                        1. 2

                                          It’s an extremely interesting space! The thing that gets me is most of it is focused on blockchain, which I don’t think is the correct direction, it’s more the fundamentals of cryptography, provenance, authenticity, confidentiality, integrity, &c. that are the real killer features needed.

                                          There are a lot of really cool technical folks out there that are working in this space, mostly because of crypto-currencies.

                                          Spot on, and it’s more than just cryptography too! As much as I dislike the cryptocurrency space, there have been huge inroads in Program Analysis, Cryptography, &c that have been paid for by the likes of Bitcoin, Ethereum, and so on.

                                          Things like information leakage or SIGINT can come at you from so many dimensions that it is impossible to track it all, even conceptually. The old joke is why go to all of the trouble to hack a guy’s system when you can just stick a cam in his office?

                                          exactly right; it’s like that old XKCD about what cypherpunks want to happen, versus what will happen.

                                          Frankly I love this space. I also love the AI/ML/GAN stuff I’m seeing. So many areas of cool tech and so little time!

                                          There’s also some interesting inroads that tie AML/KYC to ML and what not. There’s a lot of fascinating things going on here…

                                          1. 2

                                            I completely agree, and I agree that there’s some crossover here.

                                            When I think of the things I would like to do in this space, I end up thinking about stuff like code-to-hardware, where the code and hardware both are digitally-signed and only does the one thing the coder programmed it to do. To me, this guarantee of provenance and behavior is what’s missing in modern computing. I have no idea what the hell my phone or PC are doing, and that’s way f*cked up. We’ve somehow gotten upside-down in relationship to our technology. Ctypro has a big place at the table in fixing that.

                                            But I also wanted to be an astronaut, master Kung Fu, and become a secret agent. Conceptually I can understand many things, and I can even roughly spec it out and work on accomplishing them, but there’s no way around investing the money and doing the hard work, even if it’s possible to get there from here. (And it might not be. Most things like this you don’t understand until you do them. As we marines say, we learn by engaging with the enemy, not planning)

                                            I’ve seen a lot of money coming from various folks to make progress here, but in the end it all kind of boils down to “Here’s yet another app to identify hot dogs, only it’s uses our tech, it’s a dapp and it works zero trust!”

                                            I ain’t got no time for that stuff. Life’s too short.

                                      1. 23

                                        the probability that changeme is actually valid base64 encoding must be very low

                                        On the contrary, any 8 char alphanumeric string is valid base64. All such strings with a length divisible by 4, in fact, as those strings are guaranteed not to need trailing padding.

                                        1. 3

                                          I remember with distinct horror the first time I realize that rot13(base64(data)) is also valid base64, and that there’s probably a developer out there that was using this…

                                        1. 3

                                          I wonder what was used to visualize the SQL statements in there, that’s pretty sweet!

                                          edit Thinking about it further, I’ve never written a malicious application for a tester to run. I have done the following during CTFs and such tho:

                                          • wrote a middleware for Flask that exploited the API Shutdown function in ZAP
                                          • given teams vulnerable routers and then exploited them one by one
                                          • hosted back doors & other unknown vulns in applications that I could exploit as part of either a white or black team
                                          • run while true; do killall -9 $server; sleep 3; done on the host via various mechanisms
                                          • written a programming language and then inserted a backdoor/bug/key into its prelude
                                          1. 8

                                            If your DB exists inside one org - sure. If your DB is shared between multiple orgs and you want to concentrate on tech, not on legal… Maybe time-series database won’t cut it.

                                            1. 10

                                              What attack, exactly, do you fear that is allowed by giving your partner orgs append-only access to a DB, but is not possibly by giving them append access to your blockchain?

                                              I note that the article explicitly addresses the ‘partners who don’t trust each other’ use case.

                                              For example, a private ledger allows data to be shared and seen by different parties who do not need to trust each other because of the software’s rules – this could be a bank and a regulator sharing data access to customer trades data. I would argue that such data access could be done via the business logic layer sitting on top of the database layer to provide data to outside parties.

                                              1. 6

                                                This

                                                ‘partners who don’t trust each other’

                                                is not compatible with this:

                                                giving your partner orgs append-only access to a DB

                                                Because it requires your partner to trust that you do not do funny stuff on DB yourself. Simplest attack - place a buy before appending a large incoming buy order, and place a sell just after it. Free money. Happens on public blockchains all the time.

                                                BTW, this article is a dumpster fire. It is full of false claims, half-truths and just irrelevant bullshit. The guy who wrote it knows his time-series databases and knows almost nothing about blockchain design space.

                                                Blocks are added to the blockchain at regular time intervals. For each block of data, there is an associated timestamp.

                                                Yes. Trains also arrive at different time points. Is Amtrak a blockchain? How is this even relevant?

                                                Data replication: Each node in the blockchain holds the entire history of transactions. If one node is compromised, we rely on the others to provide the full history. Again, this concept has been in effect for decades with traditional databases: if one database fails, we may want another as a backup.

                                                False. There are multiple types of nodes. There are chains where history can be truncated - chains with snapshots. Reason for nodes to have the full state is an ability to validate every state transition. Data availability is an important concern, but it is secondary since the need to share some data can be removed with the help of zksnarks / bulletproofs.

                                                full history of all individual transactions ordered by time; this is how blockchain nodes work

                                                No, this is not how blockchains work. They implement total global ordering of events. But the order is logical, not time based. E.g. both Bitcoin and Ethereum include transactions in the order defined by the fee paid - from txes paying high fees, to txes paying low fees. Total global order of transactions plus deterministic execution equals ability to arrive to the same final state. It has very little to do with time.

                                                Blockchains would have multiple parties (i.e., nodes) to agree for a specific transaction. There are consensus algorithms such as Raft and Paxos in traditional databases akin to a voting mechanism.

                                                This bit is just lazy. Consensus determines the order of inclusion of transactions. Nothing more.

                                                Long 256 format: This is the format of crypto public addresses. At QuestDB we have built a data type that is better than a string to efficiently write and read Long 256 blockchain addresses.

                                                Irrelevant. This is a wrong layer. See Tendermint. It provides a total ordering on a set of binary blobs. Binary blobs are untyped, and Tendermint knows nothing about details of business logic of the application it is running.

                                              2. 8

                                                You are not avoiding any “legal”.

                                                This reminds me how a bunch of blockchain peddlers that made their way into a meeting of a municipal IT steering committee, trying to sell their “block-chain based authentication mechanism”. They did not read eIDAS and GDPR (nor the related local laws) did not read the public service information systems law, nor the cybernetic security law and showed that they had zero understanding of the fact, that if public administration screws up proving someone’s identity and that someone is harmed by that, the administration is held liable.

                                                It is several times easier to write a contract between couple of parties detailing how you share data and who is responsible for what, adjust your respective privacy policies and use a couple of regular databases than “putting it all onto a blockchain”, potentially being liable for publishing personal information that cannot ever be deleted.

                                                And frankly, I am pretty confident writing small-scale contracts myself, without being a lawyer. Continental Law tends to provide mostly sane defaults (to a point it is safe to sign most of the common contracts without any additional clauses apart from the mandatory ones) and since overly unfair contracts are invalid and courts are expected to read into what the signatories meant, you only need lawyers in high-risk situations or once something blows up.

                                                And if you need it to scale, just use adhesive contract (terms of service) with one organization acting as a steward. If you need it to be neutral, association is both the simplest of corporations to fund and also the simplest to administer (almost no accounting) providing democratic (one member = one vote) decision-making by default.

                                                1. 6

                                                  How do you realistically prevent a 51% attack though…?

                                                  1. 8

                                                    What you decide to do will depend a lot on the specifics of your use-case. You might decide to run your own proof of authority blockchain, or to run some other BFT protocol such as HotStuff. You also might communicate using a public blockchain. However, what you must not do is run your own proof of work blockchain. Your comment correctly identifies one of the reasons why doing so is a bad idea.

                                                    1. 3

                                                      WoT?

                                                      1. 4

                                                        Web of Trust style, ala Proof of Authority, doesn’t actually handle adversarial consensus, only who can publish to the blockchain, so often you’ll see another consensus algorithm underneath that layer, like IBFT, Raft, what-have-you, if you’re concerned with adversaries within the trusted nodes

                                                        1. 2

                                                          Thank you, that’s helpful.

                                                          1. 2

                                                            of course! It’s a really interesting space, and there’s lots of nuances to it all. We obviously deal with it a lot at work, more than other folks may in their day to day

                                                      2. 2

                                                        Enterprise organisations are also vulnerable to 51% attack. You can buy 51% of the shares, then burn the whole thing to the ground.

                                                      3. 4

                                                        Yeah this is my understanding of the value of enterprise blockchains. It’s more about getting diverse organisations to run a database together in an interoperable manner without one of them becoming the “owner” of the database or similar. I’ve never actually worked in such a large organisation so I have no idea if this rings true.

                                                        1. 4

                                                          This is generally the case; my company reviews blockchains quite frequently, as well as their installations, and we’ve seen this comment often. Having said that, I haven’t seen as much success from that sort of thing; very often it’s a pilot, and doesn’t go much further than that.

                                                      1. 8

                                                        There are a few “blockchain”-like systems that use Merkle trees or some other consensus algorithm that we use every day that are “easier” than blockchains proper.

                                                        Certificate Revocation Lists (CRLs) are a signed tree, similar to a Merkle (there’s some complexity, but we’ll go with it) that is/was used to distribute when certificates should no longer be trusted, from a central source. Nodes in the tree were signed by a signing key, similar to how we process hash trees.

                                                        For the use in the article, you definitely can have signed time-series lists, these are linked timestamping data structures, and you can use them to validate that an article or node is definitely from the source you expected in the order that you expected.

                                                        Also, things like etcd make use of RAFT for consensus, allowing multiple nodes in a cluster to keep a consistent state.

                                                        Generally, when customers ask me if they need a blockchain, I refer to NIST IR 8202, which has this hilarious flowchart.

                                                        1. 1

                                                          Etcd is a blockchain in the same way Hyperledger Fabric is one. The only true blockchains are public ones. The rest is marketing.

                                                          1. 2

                                                            Hence why I said “blockchain”-like in my comment :)

                                                            Fabric, the various other private permissioned blockchains (Substrate in various modes, &c.) have their purposes too, and there are (D)PoS blockchains that are in use for private usage.

                                                            I wouldn’t say it’s “marketing” per se, it’s more that people don’t really need blockchains generally when they think they do.

                                                        1. 3

                                                          How does this compare to Paseto? Branca seems just more limited and opinionated.

                                                          1. 2

                                                            I think this is meant to be more like itsdangerous than Paseto; similar to Paseto, it is meant to be algorithmically simple (XChaCha20-Poly1305 for authenticated encryption authenticated data, whereas Paseto supports XChaCha20-Poly1305 + Blake (for nonce misuse resistance) and Edwards-curves via Ed25519), but unlike Paseto, it doesn’t have a tie in to a JSON-like structure.

                                                          1. 5

                                                            Warning: This comment may yuck your yum. If you don’t want your yum yucked, please avert your eyes now. Also, I welcome counterarguments heartily :)

                                                            I don’t understand why people use macOS who don’t want a system imposed on them.

                                                            If you want to do whatever you want, you could just run some other BSD, or even GNU+Linux. If you want a pretty UI, things like Elementary OS are available for you to use for free! And if you’re picky (I know I am), and want the benefits of a closed-down ecosystem without being beholden to that ecosystem… I suggest you just suck it up and install your other OS on a different partition. (Or even better yet, run more than one PC: one that you mess around with to your heart’s content, and one whose default configuration you leave alone for God’s sake.)

                                                            When you constantly reconfigure your primary system, all you’re really doing is carving out nice little nooks and crannies for bugs to hang out. I don’t know very many professionals who can afford to play with fire like that, it seems like more of a hobbyist thing to me. Don’t get me wrong, I love doing it all the same. But I find it strange to have the goal of messing around with your computer, and then decide that buying a Mac is the best way to accomplish that goal. You, the consumer, are choosing a producer that is actively working against your goal. Think about it for a second: why would you intentionally do that?

                                                            1. 3

                                                              Well I got this computer from work that’s why I am using it. I considered installing another operating system but learned that I should expect the battery life to decrease by ~3x after that, and some hardware might not work optimally, so dropped the idea.

                                                              A lot of people with macOS use homebrew or macports, which seems like a lot bigger intrusion into the defaults. This is just changing my $HOME.

                                                              1. 2

                                                                Eh. Plenty of people want some of both cakes. Some people just want Linux with stable WiFi*. Some people just need Xcode. Some companies issue MacBooks with no Linux option. Some people want to run i3, but also Adobe Photoshop.

                                                                * I know stable WiFi is perfectly achievable on Linux, but certainly not easier, simpler, or faster than buying a MacBook.

                                                                1. 2

                                                                  To play devil’s advocate, I think paying Mac prices just for decent Wi-Fi is a waste of money, even if it saves a bit of time. But I’ll be the first to admit that I have no first-hand experience with Linux Wi-Fi troubles.

                                                                  Also, i3 on Mac? I had no idea that was even possible!

                                                                  1. 6

                                                                    That’s a question of how much you value your time. I spend thousands of hours per year doing professional work on my laptop. If a top quality laptop cost $10,000 instead of $3,000, it would still be worth it for me. On the other hand, when I was an unemployed student I used a 5 year old Thinkpad I bought for $400 on eBay. So what is and isn’t a waste of money is completely subjective.

                                                                    To my knowledge, i3 on MacOS isn’t possible short of virtualizing a Linux desktop. But someone who needs Adobe Photoshop might have MacOS imposed on them for that reason, even if they would otherwise prefer to run Linux with i3.

                                                                    1. 5

                                                                      I think paying Mac prices just for decent Wi-Fi is a waste of money

                                                                      Just to check this: for my work load and working style, a $1000 MacBook Air is roughly the same price as the equivalent not-MacBook computer, but has easier internationalization and accessibility (I have hearing loss) than the equivalents for Linux. Windows has mostly caught up, but it’s not really a Unix by default underneath. I also just now replaced my previous MacBook Air that I purchased in 2012 and used as a daily driver, so I’ve been pretty happy with them overall.

                                                                      You obviously can go way cheaper than the Air or equivalent machines if you don’t need some of the features I use (esp. accessibility), but it’s not like non-Apple laptops are all that much cheaper most of the time. The Dell XPS 13 is right around the same price, for example.

                                                                      1. 2

                                                                        Thank you for sharing your experience, I found it interesting! I’m curious though, what concrete features would Linux need in order to be competitive with other OSes for your use?

                                                                        1. 2

                                                                          Generally, the things that are killer for me are internationalization support across the languages that I use (Bulgarian being the primary for personal stuff, Russian for work, some Turkish, Croatian) and adaptive technologies. The biggest one of the latter is telling me when my phone rings; I actually can’t hear my phone ring most of the time, and often I can’t hear it vibrate if it’s on my right hand side, so my devices lighting up with incoming calls/texts are huge for my ability to respond when clients or family are trying to get a hold of me.

                                                                          Personally, I sort of hate that I’m tied to MacOS in this way, because it means when Apple changes something, it means I don’t have as much to “respond” to it with, but it’s way easier than attempting to rig up a setup that works with all devices or setups, and infinitely less error prone.

                                                                          Does that make sense?

                                                                          1. 2

                                                                            Yes, that makes sense. I hadn’t thought about those concerns. Thank you for sharing :)

                                                                            Have you found that people in the communities for other, open-source projects have been hostile, dismissive, or complacent when it comes to fixing these problems? For instance, have you perhaps tried the issue-reporting systems and found they also have a11y/i18n issues?

                                                                            1. 2

                                                                              Btw, I saw this post on Lobster.rs that reminded me of our discussion. If you have any thoughts, I’d love to hear them!

                                                                              1. 2

                                                                                oooo yes! This looks super interesting, thank you for the link!

                                                                                Ergonomics & user experience was actually something the CNCF asked us to review Kubernetes for; we wrote a whole “security ergonomics” paper about our experiences with configuring k8s with this and mind.

                                                                                Wrt what you linked:

                                                                                Time, investment and respect. Skilled UX visionaries and employees are hard to find (limited in quantity), well-paid, and sometimes removed from the technical aspects that encourage developers to become entwined with open source. Encouraging the quantity of UX expertise to increase, and constantly pushing the importance of our end users direct input is key to growing the field. Industry-wide investment of money and resources into hiring and involving UX experts on open source projects is a key tipping point.

                                                                                this is something that’s incredibly true; I talk to my designer all the time about how security tools fail at UI/UX, and how we need folks like him to help build it up. I’m actually going to send him this article…

                                                                                let’s build-in User eXperience by default, from the beginning, every day. Instead of rebuilding afterwards, UX is involved early.

                                                                                I think, personally, this is similar to security (the field I work in), and more generally to “correctness;” builders & defenders are very frequently just sorta… tossed requirements, and they satisfy what they can in the terms they have. We see this doubly so in FLOSS: we don’t build things for general use per se, but to satisfy our own needs. This is great! It’s democratic and very liberating; the downside is that sometimes it’s very hard to understand what may make things more accessible to others. However, if we think about our requirements ahead of time, we can often at least prepare ourselves for what’s needed going forward.

                                                                    2. 2

                                                                      It seems like a bit of a futile effort. I’m not opposed to futile efforts, myself, but honestly, it just seems exhausting to me at this point. I no longer really try to fight the platform, but I do wish I could change just … one … more … thing …

                                                                    1. 7

                                                                      Using blowfish is a bit of a concern; I would recommend using something else, like libsodium or something that is a bit less malleable. Looking through the source of blowfish.js it looks ok, even tho it only supports ECB or CBC mode. Note however that it defaults to ECB, which is problematic. If you consider keeping this, I’d switch to CBC (based on what your library supports), but it would be easier to swap for another library, like libsodium.js or the like.

                                                                      Additionally, if you use anything in CBC mode, you do need some sort of HMAC against the cipher to avoid padding oracles and the like. You also want to avoid ECB mode, as that can be hugely problematic. Again, something like libsodium or another library that handles some of these “cryptographic right answers” in an opinionated way would be great.

                                                                      Lastly, you probably want to do some sort of key stretching, esp if you’re going to keep passwords as keys; even the 10 character limit is pretty short (10 chars == 80 bits); there are various algorithms here, but Argon2id is p good in the JS space. There’s quite a few others as well; PBKDF2 is terrible in many ways, but 100k iterations of PBKDF2 would be ok.

                                                                      note: I’m not saying “blowfish” is a concern per se, although I generally would recommend more standard ciphers to clients, it’s more along the lines of “typing the letters ‘A-E-S’ into your program is a code smell:” most developers don’t understand the intricacies of cryptography, and it’s easier to use something opinionated that does “the right thing” for you.

                                                                      1. 3

                                                                        I just looked at libsodium.js library. It seems to be safer than blowfish.js, but more complicated, too. I’ll try to implement it as soon as I can. Thank you for your recommendation!

                                                                        1. 2

                                                                          absolutely happy to help!

                                                                          , but more complicated, too.

                                                                          this is true; most of this is because it’s doing some of the things that you would have to do to implement some of the cryptography in a safe way, but also because it’s a different set of algorithms and libsodium.js is implementing some other moving parts surrounding it. Luckily, getting things right is mostly easier with libsodium.

                                                                          Oh, one other benefit to using libsodium I neglected to mention: it includes an Argon2id implementation if you want to use that for key stretching.

                                                                      1. 7

                                                                        I switched to Kakoune for all my ad-hoc editing although for coding I like to have something more mouse driven for working on multiple files in the same session. I rarely remember the names of source files so I like to have the source tree permanently visible and accessible.

                                                                        1. 1

                                                                          I tried this today based off your comment and I’m pretty impressed with it, esp. as a vi/vim user for the last 25+ years…

                                                                        1. 9

                                                                          A BIG oof about this. I really loved actix for what it was doing for the rust community. But I also shaked my head about how ignorant some of the creators replies seemed to be, when people tried their hardest to point out why a specific usage of unsafe wasn’t right. Still I always kept using actix, because I think that in itself it was a nice project and improving it will eventually remove all its UB. This reminds me of systemd for some reason.

                                                                          Obviously there was also too much shitstorm about actix itself, so I can’t blame them for complaining about how people started to treat actix.

                                                                          I’ll have to transition some projects. Guess I won’t have to do the async/await switch for the actix code anymore.

                                                                          1. 3

                                                                            Even if the code was unsafe and the author impermeable to the remarks, it’s his code and therefore his right. Users of his code are not entitled to anything. If the users don’t like it, they can fork it and do as they please. Expecting anything more is preposterous.

                                                                            1. 2

                                                                              Are there actual exploits for any the unsafe code? Or is it the usual Rust cargo cult over reaction?

                                                                              1. 5

                                                                                Both. Issues got proven to be exploitable from userland.

                                                                                1. 9

                                                                                  I wouldn’t go as far. It was proven that there’s a usage pattern that would trigger UB if the user used it this way. The pattern in itself is unlikely and probably not present in any application out there.

                                                                                  So there’s no general path to exploiting actix-based applications in general.

                                                                                  It’s basically equivalent to the openssl side of heartbleed: if you use openssl wrong, it is exploitable, you are still using it wrong. Given that the actix-web API didn’t seem to be intended for library clients use, it’s even less likely.

                                                                                  Not arguing that it shouldn’t be fixed, but let’s be realistic about the impact.

                                                                                  1. 4

                                                                                    It was proven that there’s a usage pattern that would trigger UB if the user used it this way. The pattern in itself is unlikely and probably not present in any application out there

                                                                                    From a security engineer’s point of view, doesn’t that constitute “completely broken”?

                                                                                    1. 6

                                                                                      No. You are literally flying planes or driving cars based on systems with potentials for such bugs. There’s usually expensive tooling around the “don’t do this, then” (linters and such).

                                                                                      1. 4

                                                                                        User here refers to the developer using the library, not the user interfacing with the resulting web service. Under normal circumstances (that is, not a contrived example code snippet) it’s not broken, but it requires some amount of care that the problematic pattern wasn’t used somewhere in spite of it being unusual.

                                                                                        strcpy isn’t “completely broken” and for a given use of it it’s possible to reason that it is safe (if it is). Still, a security engineer would recommend (and at some point: demand) not to use it to reduce the amount of reasoning required (and remove the code smell). The issue at hand in actix is much less worrisome than strcpy in terms of being able to shoot yourself in the foot.

                                                                                        AIUI the author was interested in handling this, but on their own terms. Apparently that wasn’t enough for some and so they cranked up the internet rage engines.

                                                                                        1. 3

                                                                                          from a security engineer’s perspective we would talk about Likelihood or Difficulty (both of Discovery and of Exploitation) as well as Impact; you may see other metrics thrown in like Confidentiality, Integrity, and Availability.

                                                                                          If the user must use a specific library in a problematic way, that usually constitutes a Likelihood of Very Low/Low or a Difficulty of High; basically, yes, this could be bad (Impact of High), but the Likelihood of a user doing that may be very low. Security people who speak in absolutes are usually trying to sell you something or not very effective at their jobs.

                                                                                1. 2

                                                                                  Some folks would’ve just requested a Forth or a Lisp. Then, they add whatever they want. Full environment. :)

                                                                                  1. 4

                                                                                    I wrote a Forth for CTFs once; it was meant to be able to easily compile statically, base64, and then upload to the host. Was a lot of fun.

                                                                                  1. 9

                                                                                    I can’t agree with this more. While doing olin/wasmcloud stuff I have resorted to this slightly insane method of debugging:

                                                                                    • debug printf the pointer of a value you want to look for
                                                                                    • immediately crash the program so none of the heap gets “contaminated”
                                                                                    • pray the value didn’t get overwritten somehow (in my subjective and wholly personal experience Rust does this sometimes, it’s kind of annoying)
                                                                                    • dump the webassembly linear memory/heap to a binary file
                                                                                    • search for that pointer using xxd(1) and less(1)
                                                                                    • grab my little-endian cheat sheet
                                                                                    • hopefully derive meaning from it

                                                                                    WebAssembly debuggers really need to happen, but until they do we are basically stuck in the printf debugging, black magick and tarot cards stone age, but at least it’s slightly easier to make a core dump. After a few hours though you learn to read little endian integers without needing to break out graph paper.

                                                                                    1. 2

                                                                                      I’ve used instrumentation tools like Wasabi for dynamic testing during client assessments (yes, we have clients working in WASM already). Also, we’ve extended Manticore, our symbolic executor to support WASM, and we have some symbolic debugging-like scripts for it internally.

                                                                                      In general tho, I whole-heartedly agree; we’re seeing more and more of this sort of thing, and the tooling definitely needs to catch up.

                                                                                      1. 2

                                                                                        To be fair, the changes needed on the DWARF/LLVM side are super recent; I believe I saw commits related to this in LLVM as recently as December. The debuggers will catch up, but it takes time for some of this stuff to percolate outwards, I haven’t done any testing myself in the last few months, but I suspect the browsers have experimental support on the way shortly, if not already available. It will take a bit longer for this stuff to make it into say, lldb, but not that much longer.

                                                                                        The current situation does suck for those of us needing to build stuff with WebAssembly right now, but I keep myself sane by knowing that its just growing pains - we’ll get the necessary tooling sooner rather than later.

                                                                                        1. 1

                                                                                          Call me a pessimist, but existing debugging infrastructure might not be good enough for this. We might wanna start with something like AVR debuggers or other things made for Harvard architectures.

                                                                                          1. 2

                                                                                            Presumably there are different needs. The first need is for front end / in browser use cases. For that, foo.wasm.debug could effectively include all the debug symbols, and source map, or whatever and you use the in browser debugger. That seems fine.

                                                                                            The server side is more problematic, I think, but that’s where your Harvard arch comes in. Provide a manhole cover that can be opened for the purposes of attaching a debugger to, in your execution environment, and poke and prod with step debuggers, perhaps? Still need the debug symbols and such… of course.

                                                                                            1. 1

                                                                                              Jtag into a wasm env, really this just needs to a handful of exported fn from the env itself. One could run wasm in wasm and implement it yourself in user space.

                                                                                        2. 1

                                                                                          What if your wasm env could visualize memory in real-time and set breakpoints in any fn call or memory write?

                                                                                          1. 1

                                                                                            Debuggers have two sides, symbol side and target side. Breaking/resuming and reading/writing memory is done in target side, and you can implement them in your WebAssembly environments. But that only lets you to do break 123 or print 456, not break function or print variable. The article is mostly about symbol side.

                                                                                            1. 1

                                                                                              That would solve a lot of problems, but it would also be a lot of work :)

                                                                                              I’m gonna see what I can do though.

                                                                                              1. 1

                                                                                                Here is a veritable zoo of binary visualizers

                                                                                                https://reverseengineering.stackexchange.com/questions/6003/visualizing-elf-binaries

                                                                                                I think the past of least friction is run wasm3 within the browser so that you can have 100% over execution, it will be slow ~25x, but provide first class access to all facets of execution of the code under debug.

                                                                                          1. 5

                                                                                            Besides what @nickpsecurity said, I think the other thing that has changed are that formal methods & fancier types are slowly creeping in from the edges.

                                                                                            • 20 years ago, linear & affine types were neat academic exercises, now we have at least one major programming language with them
                                                                                            • languages like ML, Haskell, &c were side curiosities, now you can often find major projects written in them
                                                                                            • tools like symbolic executors, abstract interpreters, “design by contract,” and so on are relatively normal now, not fancy wares of academic high towers
                                                                                            • property testing, fuzzing, and other types of random mutation testing aren’t seen as black arts, but rather mundane things that most people can use