1. 4

    I think agda has really nice syntax. And indeed, the language itself is quite nice. However, the standard library is littered with tons of unicode symbols. These are effectively un-typable in most editors. The dev’s position is that you should use their emacs plugin which converts ascii digraphs and trigraphs into symbols (as if they were ligatures). However, those of us who don’t use emacs (all 95% of us) are SOL. Given that most of these symbols are typed as digraphs anyway, this whole thing is just an exercise in aesthetics.

    1. 1

      :digraph

      ;-)

    1. 2

      The last time I checked OpenBSD couldn’t mount RAID1 if 1 drive was missing. Did this change in the meantime?

      1. 4

        Can anybody recommend a good drive & cartridges for SOHO use?

        1. 3

          15/25 handouts and 7/19 lectures are spent on parsing, which seems – after a cursory glance – like a very unbalanced approach. I know parsing isn’t “solved”, but it seems like much of the current research and many of the interesting problems are in the later stages of the compiler. More advanced type systems, better code generation, etc.

          This is the Spring 2018 version, which doesn’t seem substantially different from the above linked 2012 version.

          1. 2

            Can you recommend a course or text that covers these other topics better?

            1. 3

              Yeah, see, I don’t know. I’m still Joe Schmoe newbie in this area. Things I’ve referenced in the past include:

              Also, a bit of a shameless plug for my own (admittedly very simple) resource on bytecode compilers. With any luck I’ll have a series following the Ghuloum paper soon…

              1. 2

                Appel’s Tiger book is in SML (not OCaml).

                I didn’t like it very much because it’s using ml-yacc.

                1. 1

                  Ahhhh yes, you’re right. Well, you could substitute OCaml and Menhir!

                  1. 1

                    Or a recursive descent parser.

              2. 3

                Ooh, and how could I forget munificent’s book?

                1. 2

                  This is also another good resource, but more about making fast runtimes.

                  1. 2

                    Thanks a lot for this link! This provides a lot of useful knowledge I was looking for a while!

                  2. 2

                    The slides seem to be insufficient to understand the topics – especially the parsing part. I’d hope there is a course script available for actual students.

                    In general these kind of courses seem to spend

                    • 80% of their time telling people “how to do X”
                    • maybe 15% of the time on “why do X?”
                    • rarely more than 5% on “should you even do X?”

                    It’s kinda sad, but reflects the issues of this profession very well.

                    Even parsing itself, which the least interesting and important piece of a compiler course, doesn’t ask the important question of “should we build languages that would give us this much trouble parsing?”.

                    Popular example is “let’s assume we use <> for generics, what could possibly go wrong (ignoring the 20 years of examples of it going wrong)?”.

                  1. 9

                    I personally prefer to stick with nginx after having seen how the Caddy dev handled this security issue. But that’s just me.

                    1. 8

                      His responses when a LE outage meant that no-one could restart (or start, if stopped) Caddy despite having valid certs convinced me it’s not a viable tool.

                      “Automagic TLS Certificates” which really means “an ACME client you don’t have direct control over” is not really a feature I’d see as valuable for anyone with the slightest bit of operator/admin experience.

                      1. 7

                        That’s barely a “security vulnerability”, and a lot of accusatory debate, including on unrelated matters like the EULA, the kind that give us security people a bad rep with open source maintainers.

                        1. 4

                          I really only skimmed it, but what didn’t you like about it? They seemed pretty open to discussion and it seems to have been resolved?

                          1. 1

                            It was more of the developer’s attitude towards the security issue itself—they straight up dismissed it.

                          2. 2

                            If I understand this correctly it’s rather a leakage of public certificates (e.g. for other subdomains) available on the same server.

                          1. 4

                            I am always a bit disappointed when these tiny gadgets don’t include a battery. I guess it makes sense, but I really wanted a thing kinda like the Pebble Core (https://www.techradar.com/news/wearables/pebble-core-everything-you-need-to-know-1322135), without it being “tape a RasPi and a battery together”.

                            That being said, the … tininess of this product is really amazing. I really love things like micropython being available for this too, where you’re not having to choose between “code in C” and “run all of V8”.

                            1. 1

                              Had the same thoughts and did a little research. MeowBit and Odroid Go look interesting if the handheld form factor is ok. Haven’t seen something like a programmable fitness band though.

                              1. 2

                                If 7-segment LCDs are OK for you this watch[0] might be an option.

                                I use it with a custom firmware (which you can flash using CLI tools). Without RF and sensors I had a battery life of >1 year.

                                [0] https://processors.wiki.ti.com/index.php/EZ430-Chronos

                            1. 1

                              How do people find these bugs?

                              I don’t think it’s just code review. Maybe fuzzing?

                              1. 4

                                Maybe they have a specialized fuzzer for finding issues with sub process execution (see the other two vulnerabilities they’ve found at the end of the comment.), but I think its more likely that they are doing targeted reviews into this kind of bug class.

                                From my experience, I would start by searching for code paths that execute programs with user input and then work backwards to see if the user input is validated at all, if not then you already have your bug, otherwise you might have to do a full review on how the input is validated (which they probably did in this case and that’s how they found the logic error).

                                https://www.qualys.com/2019/12/11/cve-2019-19726/local-privilege-escalation-openbsd-dynamic-loader.txt https://www.qualys.com/2019/12/04/cve-2019-19521/authentication-vulnerabilities-openbsd.txt

                                1. 3

                                  Agree. For my own code, I’d just grep for system and exec* calls and review ‘em all. If I were in the business of reviewing other code for this, I think I’d probably write a taint checker to help me look. It feels like llvm could get you really close to that, these days. It looks like there might be at least the scaffolding for that there already. Last time I had to do it as a one-off on someone else’s code, I modified the codebase I was working with to taint certain variables then highlight whether any of a group of functions acted on them.

                              1. 7

                                This is so pointless. Here is wc in awk with 52 characters without a single branch. Took about 10 seconds to write.

                                {b+=length($0)+1;w+=NF;++l}END{print l,w,b,FILENAME}
                                
                                1. 1

                                  Here’s one in Perl (it doesn’t print the filename, though):

                                  $l++;$w+=@F;$b+=length}{print "$l $w $b\n"
                                  

                                  Explanation:

                                  • @F in scalar context is the number of fields (NF)
                                  • length without arguments is the length of current line
                                  • }{ is the END

                                  Example:

                                  echo "hello world" | perl -ane '$l++;$w+=@F;$b+=length}{print "$l $w $b\n"'
                                  
                                1. 8

                                  I still find it hard to get my head around how licensing works and I bet that this is the main reason why MIT is used. MIT really is the only license that is easy to understand. There are many blog articles on the web, but to this day I didn’t find a single overview that describes each of the major licenses in an understandable way. And I feel I could really use an overview. Day to day language, built around the most important questions: copyright, attribution, what can both non-commercial and commercial users do and what they can’t do. And also what are my rights as the owner. If you found any resources on this matter please let me know.

                                  1. 4

                                    This site lists the permissions, conditions and limitations for a few licenses
                                    https://choosealicense.com/licenses/

                                    This is a license chooser done by @qznc (needs JavaScript)
                                    http://beza1e1.tuxen.de/licences/

                                    Here are the pros and cons of several licenses from @arp242
                                    https://www.arp242.net/license.html

                                    1. 3

                                      https://tldrlegal.com/ exists and is useful.

                                      1. 1

                                        that is easy to understand

                                        There’s still a lot of implied baggage that you may want to be aware of. And the article I has some U.S.-specific parts, so YMMV even more across jurisdictions.

                                      1. 3

                                        Is TLA+ useful for people writing single-threaded (and safety critical) software?

                                        I have the impression most use cases are in distributed/parallel computing.

                                        1. 4

                                          I’ve used it to verify terminating algorithms, like graph sorting, but you’re correct it’s more oriented to concurrency problems. If you’re writing something deterministic and threaded, the problem domain is a lot simpler and tools like Dafny and Frama-C become more feasible.

                                          1. 3

                                            It’s useful for code where order matters. I’ve used it for checking front end state machines (like Redux).

                                          1. 32

                                            To me the big deal is that Rust can credibly replace C, and offers enough benefits to make it worthwhile.

                                            There are many natively-compiled languages with garbage collection. They’re safer than C and easier to use than Rust, but by adding GC they’ve exited the C niche. 99% of programs may work just fine with a GC, but for the rest the only practical options were C and C++ until Rust showed up.

                                            There were a few esoteric systems languages or C extensions that fixed some warts of C, but leaving the C ecosystem has real costs, and I could never justify use of a “weird” language just for a small improvement. Rust offered major safety, usability and productivity improvements, and managed to break out of obscurity.

                                            1. 38

                                              Ada provided everything except ADTs and linear types, including seamless interoperability with C, 20 years before Rust. Cyclone was Rust before Rust, and it was abandoned in a similar state as Rust was when it took off. Cyclone is dead, but Ada got a built-in formal verification toolkit in its latest revision—for some that stuff alone can be a reason to pick instead of anything else for a new project.

                                              I have nothing against Rust, but the reason it’s popular is that it came at a right time, in the right place, from a sufficiently big name organization. It’s one of the many languages based on those ideas that, fortunately, happened to succeed. And no, when it first got popular it wasn’t really practical. None of these points makes Rust bad. One just should always see a bigger picture especially when it comes to heavily hyped things. You need to know the other options to decide for yourself.

                                              Other statically-typed languages allow whole-program type inference. While convenient during initial development, this reduces the ability of the compiler to provide useful error information when types no longer match.

                                              Only in languages that cannot umabiguously infer the principal type. Whether to make a tradeoff between that and support for ad hoc polymorphism or not is subjective.

                                              1. 15

                                                I’ve seen Cyclone when it came out, but at that time I dismissed it as “it’s C, but weird”. It had the same basic syntax as C, but added lots of pointer sigils. It still had the same C preprocessor and the same stdlib.

                                                Now I see it has a feature set much closer to Rust’s (tagged unions, patterns, generics), but Rust “sold” them better. Rust used these features for Result which is a simple yet powerful construct. Cyclone could do that, but didn’t. It kept nullable pointers and added Null_Exception.

                                                1. 12

                                                  Ada provided everything except ADTs and linear types

                                                  Unfortunately for this argument, ADTs, substructural types and lifetimes are more exciting than that “everything except”. Finally the stuff that is supposed to be easy in theory is actually easy in practice, like not using resources you have already cleaned up.

                                                  Ada got a built-in formal verification toolkit in its latest revision

                                                  How much of a usability improvement is using these tools compared to verifying things manually? What makes types attractive to many programmers is not that they are logically very powerful (they are usually not!), but rather that they give a super gigantic bang for the buck in terms of reduction of verification effort.

                                                  1. 17

                                                    I would personally not compare Ada and Rust directly as they don’t even remotely fulfill the same use-cases.

                                                    Sure, there have been languages that have done X, Y, Z before Rust (the project itself does not lay false claim to inventing those parts of the language which may have been found elsewhere in the past), but the actual distinguishing factor for Rust that places it into an entirely different category from Ada is how accessible and enjoyable it is to interact with while providing those features.

                                                    If you’re in health or aeronautics, you should probably be reaching for the serious, deep toolkit provided by Ada, and I’d probably be siding with you in saying those people probably should have been doing that for the last decade. But Ada is really not for the average engineer. It’s an amazing albeit complex language, that not only represents a long history of incredible engineering but a very real barrier of entry that’s simply incomparable to that of Rust’s.

                                                    If, for example, I wanted today to start writing from scratch a consumer operating system, a web browser, or a video game as a business venture, I would guarantee you Ada would not even be mentioned as an option to solve any of those problems, unless I wanted to sink my own ship by limiting myself to pick from ex-government contractors as engineers, whose salaries I’d likely be incapable of matching. Rust on the other hand actually provides a real contender to C/C++/D for people in these problem spaces, who don’t always need (or in some cases, even want) formal verification, but just a nice practical language with a systematic safety net from the memory footguns of C/C++/D. On top of that, it opens up these features, projects, and their problem spaces to many new engineers with a clear, enjoyable language free of confusing historical baggage.

                                                    1. 6

                                                      Have you ever used Ada? Which implementation?

                                                      1. 15

                                                        I’ve never published production Ada of any sort and am definitely not an Ada regular (let alone pro) but I studied and had a fondness for Spark around the time I was reading “Type-Driven Development with Idris” and started getting interested in software proofs.

                                                        In my honest opinion the way the base Ada language is written (simple, and plain operator heavy) ends up lending really well to extension languages, but it also can make difficult for beginners to distinguish the class of concept used at times, whereas Rust’s syntax has a clear and immediate distinction between blocks (the land of namespaces), types (the land of names), and values (the land of data). In terms of cognitive load then, it feels as though these two languages are communicating at different levels. Like Rust is communicating in the mode of raw values and their manipulation through borrows, while the lineage of Ada languages communicate at a level that, in my amateur Ada-er view, center on the expression of properties of your program (and I don’t just mean the Spark stuff, obviously). I wasn’t even born when Ada was created, and so I can’t say for sure without becoming an Ada historian (not a bad idea…), but this sort of seems like a product of Ada’s heritage (just as Rust’s so obviously written to look like C++).

                                                        To try and clarify this ramble of mine, in my schooling experience, many similarly young programmers of my age are almost exclusively taught to program at an elementary level of abstract instructions with the details of those instructions removed, and then after being taught a couple type-level incantations get a series of algorithms and their explanations thrown at their face. Learning to consider their programs specifically in terms of expressing properties of that program’s operations becomes a huge step out of that starting box (that some don’t leave long after graduation). I think something that Rust’s syntax does well (if possibly by mistake) is fool the amateur user into expressing properties of their programs on accident while that expression becomes part of what seems like just a routine to get to the meat of a program’s procedures. It feels to me that expressing those properties are intrinsic to the language of speaking Ada, and thus present a barrier intrinsic to the programmer’s understanding of their work, which given a different popular curriculum could probably just be rendered as weak as paper to break through.

                                                        Excuse me if these thoughts are messy (and edited many times to improve that), but beyond the more popular issue of familiarity, they’re sort of how I view my own honest experience of feeling more quickly “at home” in moving from writing Rust to understanding Rust, compared to moving from just writing some form of Ada, and understanding the program I get.

                                                    2. 5

                                                      Other statically-typed languages allow whole-program type inference. While convenient during initial development, this reduces the ability of the compiler to provide useful error information when types no longer match.

                                                      Only in languages that cannot unabiguously infer the principal type. Whether to make a tradeoff between that and support for ad hoc polymorphism or not is subjective.

                                                      OCaml can unambiguously infer the principal type, and I still find myself writing the type of top level functions explicitly quite often. More than once have I been guided by a type error that only happened because I wrote the type of the function I was writing in advance.

                                                      At the very least, I check that the type of my functions match my expectations, by running the type inference in the REPL. More than once have I been surprised. More than once that surprise was caused by a bug in my code. Had I not checked the type of my function, I would catch the bug only later, when using the function, and the error message would have made less sense to me.

                                                      1. 2

                                                        At the very least, I check that the type of my functions match my expectations, by running the type inference in the REPL

                                                        Why not use Merlin instead? Saves quite a bit of time.

                                                        That’s a tooling issue too of course. Tracking down typing surprises in OCaml is easy because the compiler outputs type annotations in a machine-readable format and there’s a tool and editor integrations that allow me to see the type of every expression in a keystroke.

                                                        1. 2

                                                          Why not use Merlin instead? Saves quite a bit of time.

                                                          I’m a dinosaur, that didn’t take the time to learn even the existence of Merlin. I’m kinda stucks in Emacs’ Tuareg mode. Works for me for small projects (all my Ocaml projects are small).

                                                          That said, my recent experience with C++ and QtCreator showed me that having warnings at edit time is even more powerful than a REPL (at least as long as I don’t have to check actual values). That makes Merlin look very attractive all of a sudden. I’ll take a look, thanks.

                                                    3. 5

                                                      Rust can definitely credibly replace C++. I don’t really see how it can credibly replace C. It’s just such a fundamentally different way of approaching programming that it doesn’t appeal to C programmers. Why would a C programmer switch to Rust if they hadn’t already switched to C++?

                                                      1. 43

                                                        I’ve been a C programmer for over a decade. I’ve tried switching to C++ a couple of times, and couldn’t stand it. I’ve switched to Rust and love it.

                                                        My reasons are:

                                                        • Robust, automatic memory management. I have the same amount of control over memory, but I don’t need goto cleanup.
                                                        • Fearless multi-core support: if it compiles, it’s thread-safe! rayon is much nicer than OpenMP.
                                                        • Slices are awesome: no array to pointer decay. Work great with substrings.
                                                        • Safety is not just about CVEs. I don’t need to investigate memory murder mysteries in GDB or Valgrind.
                                                        • Dependencies aren’t painful.
                                                        • Everything builds without fuss, even when supporting Windows and cross-compiling to iOS.
                                                        • I can add two signed numbers without UB, and checking if they overflow isn’t a party trick.
                                                        • I get some good parts of C++ such as type-optimized sort and hash maps, but without the baggage C++ is infamous for.
                                                        • Rust is much easier than C++. Iterators are so much cleaner (just a next() method). I/O is a Read/Write trait, not a hierarchy of iostream classes.
                                                        1. 6

                                                          I also like Rust and I agree with most of your points, but this one bit seems not entirely accurate:

                                                          Fearless multi-core support: if it compiles, it’s thread-safe! rayon is much nicer than OpenMP.

                                                          AFAIK Rust:

                                                          • doesn’t guarantee thread-safety — it guarantees the lack of data races, but doesn’t guarantee the lack of e.g. deadlocks;
                                                          • guarantees the lack of data races, but only if you didn’t write any unsafe code.
                                                          1. 20

                                                            That is correct, but this is still an incredible improvement. If I get a deadlock I’ll definitely notice it, and can dissect it in a debugger. That’s easy-peasy compared to data races.

                                                            Even unsafe code is subject to thread-safety checks, because “breaking” of Send/Sync guarantees needs separate opt-in. In practice I can reuse well-tested concurrency primitives (e.g. WebKit’s parking_lot) so I don’t need to write that unsafe code myself.

                                                            Here’s an anecdote: I wrote some single threaded batch-processing spaghetti code. Since it each item was processed separately, I decided to parallelize it. I’ve changed iter() for par_iter() and the compiler immediately warned me that in one of my functions I’ve used a 3rd party library which used an HTTP client library which used an event loop library which stored some event loop data in a struct without synchronization. It pointed exactly where and why the code was unsafe, and after fixing it I had an assurance the fix worked.

                                                            1. 6

                                                              I share your enthusiasm. Just wanted to prevent a common misconception from spreading.

                                                              Here’s an anecdote: I wrote some single threaded batch-processing spaghetti code. Since it each item was processed separately, I decided to parallelize it. I’ve changed iter() for par_iter() and the compiler immediately warned me that in one of my functions I’ve used a 3rd party library which used an HTTP client library which used an event loop library which stored some event loop data in a struct without synchronization. It pointed exactly where and why the code was unsafe, and after fixing it I had an assurance the fix worked.

                                                              I did not know it could do that. That’s fantastic.

                                                            2. 9

                                                              Data races in multi-threaded code are about 100x harder to debug than deadlocks in my experience, so I am happy to have an imperfect guarantee.

                                                              guarantees the lack of data races, but only if you didn’t write any unsafe code.

                                                              Rust application code generally avoids unsafe.

                                                              1. 4

                                                                Data races in multi-threaded code are about 100x harder to debug than deadlocks in my experience, so I am happy to have an imperfect guarantee.

                                                                My comment was not a criticism of Rust. Just wanted to prevent a common misconception from spreading.

                                                                Rust application code generally avoids unsafe.

                                                                That depends on who wrote the code. And unsafe blocks can cause problems that show in places far from the unsafe code. Meanwhile, “written in Rust” is treated as a badge of quality.

                                                                Mind that I am a Rust enthusiast as well. I just think we shouldn’t oversell it.

                                                              2. 7

                                                                guarantees the lack of data races, but only if you didn’t write any unsafe code.

                                                                As long as your unsafe code is sound it still provides the guarantee. That’s the whole point, to limit the amount of code that needs to be carefully audited for correctness.

                                                                1. 2

                                                                  I know what the point is. But proving things about code is generally not something that programmers are used to or good at. I’m not saying that the language is bad, only that we should understand its limitations.

                                                                2. 1

                                                                  I find it funny that any critique of Rust needs to be prefixed with a disclaimer like “I also like Rust”, to fend off the Rust mob.

                                                              3. 11

                                                                This doesn’t really match what we see and our experience: a lot of organisations are investigating their replacement of C and Rust is on the table.

                                                                One advantage that Rust has is that it actually lands between C and C++. It’s pretty easy to move towards a more C-like programming style without having to ignore half of the language (this comes from the lack of classes, etc.).

                                                                Rust is much more “C with Generics” than C++ is.

                                                                We currently see a high interest in the embedded world, even in places that skipped adopting C++.

                                                                I don’t think the fundamental difference in approach is as large as you make it (sorry for the weak rebuttal, but that’s hard to quantify). But also: approaches are changing, so that’s less of a problem for us, as long as we are effective at arguing for our approach.

                                                                1. 2

                                                                  It’s just such a fundamentally different way of approaching programming that it doesn’t appeal to C programmers. Why would a C programmer switch to Rust if they hadn’t already switched to C++?

                                                                  Human minds are sometimes less flexible than rocks.

                                                                  That’s why we still have that stupid Qwerty layout: popular once for mechanical (and historical) reasons, used forever since. As soon as the mechanical problems were fixed, Sholes imself devised a better layout, which went unused. Much later, Dvorak devised another better layout, and it is barely used today. People thinking in Qwerty simply can’t bring themselves to take the time to learn the superior layout. (I know: I’m in a similar situation, though my current layout is not Qwerty).

                                                                  I mean, you make a good point here. And that’s precisely what’s make me sad. I just hope this lack of flexibility won’t prevent C programmers from learning superior tools.

                                                                  (By the way, I would chose C over C++ in many cases, I think C++ is crazy. But I also know ML (OCaml), a bit of Haskell, a bit of Lua… and that gives me perspective. Rust as I see it is a blend of C and ML, and though I have yet to write Rust code, the code I have read so far was very easy to understand. I believe I can pick up the language pretty much instantly. In my opinion, C programmers that only know C, awk and Bash are unreasonably specialised.)

                                                                  1. 1

                                                                    I tried to switch to DVORAK twice. Both times I started to get pretty quick after a couple of days but I cheated: if I needed to type something I’d switch back to QWERTY, so it never stuck.

                                                                    The same is true of Rust, incidentally. Tried it out a few times, was fun, but then if I want to get anything useful done quickly it’s just been too much of a hassle for me personally. YMMV of course. I fully intend to try to build something that’s kind of ‘C with lifetimes’, a much simpler Rust (which I think of as ‘C++ with lifetimes’ analogously), in the future. Just have to, y’know, design it. :D

                                                                    1. 3

                                                                      I too was tempted at some point to design a “better C”. I need:

                                                                      • Generics
                                                                      • Algebraic data types
                                                                      • Type classes
                                                                      • coroutines, (for I/O and network code, I need a way out of raw poll(2))
                                                                      • Memory safety

                                                                      With the possible exception of lifetimes, I’d end up designing Rust, mostly.

                                                                      1. 2

                                                                        I agree that you need some way of handling async code, but I don’t think coroutines are it, at least not in the async/await form. I still feel like the ‘what colour is your function?’ stuff hasn’t been solved properly. Any function with a callback (sort with a key/cmp function, filter, map, etc.) needs an async_ version that takes a callback and calls it with await. Writing twice as much code that’s trivially different by adding await in some places sucks, but I do not have any clue what the solution is. Maybe it’s syntactic. Maybe everything should be async implicitly and you let the compiler figure out when it can optimise things down to ‘raw’ calls.

                                                                        shrug

                                                                        Worth thinking about at least.

                                                                        1. 4

                                                                          Function colors are effects. There are two ways to solve this problem:

                                                                          1. To use polymorphism over effects. This is what Haskell does, but IMO it is too complex.
                                                                          2. To split large async functions into smaller non-async ones, and dispatch them using an event loop.

                                                                          The second approach got a bad reputation due to its association with “callback hell”, but IMO this reputation is undeserved. You do not need to represent the continuation as a callback. Instead, you can

                                                                          1. Define a gigantic sum type of all possible intermediate states of asynchronous processes.
                                                                          2. Implement each non-async step as an ordinary small function that maps intermediate states (not necessarily just one) to intermediate states (not necessarily just one).
                                                                          3. Implement the event loop as a function that, iteratively,
                                                                            • Takes states from an event queue.
                                                                            • Dispatches an appropriate non-async step.
                                                                            • Pushes the results, which are again states, back into the event queue.

                                                                          Forking can be implemented by returning multiple states from a single non-async step. Joining can be implemented by taking multiple states as inputs in a single non-async step. You are not restricted to joining processes that were forked from a common parent.

                                                                          In this approach, you must write the event loop yourself, rather than delegate it to a framework. For starters, no framework can anticipate your data type of intermediate states, let alone the data type of the whole event queue. But, most importantly, the logic for dispatching the next non-async step is very specific to your application.

                                                                          Benefits:

                                                                          1. Because the data type of intermediate states is fixed, and the event loop is implemented in a single centralized place, it is easier to verify that your code works “in all cases”, either manually or using tools that explicitly model concurrent processes using state machines (e.g., TLA+).

                                                                          2. Because intermediate states are first-order values, rather than first-class functions, the program is much easier to debug. Just stop the event loop at an early time and pretty-print the event queue. (ML can automatically pretty-print first-order values in full detail. Haskell requires you to define a Show instance first, but this definition can be generated automatically.)

                                                                          Drawbacks:

                                                                          1. If your implementation language does not provide sum types and/or pattern matching, you will have a hard time checking that every case has been covered, simply because there are so many cases.

                                                                          2. The resulting code is very much non-extensible. To add new asynchronous processes, you need to add constructors to the sum type of intermediate states. This will make the event loop fail to type check until you modify it accordingly. (IMO, this is not completely a drawback, because it forces you to think about how the new asynchronous processes interact with the old ones. This is something that you eventually have to do anyway, but some people might prefer to postpone it.)

                                                                          1. 3

                                                                            I agree that you need some way of handling async code, but I don’t think coroutines are it

                                                                            Possibly. I actually don’t know. I’d take whatever let me write code that looks like I’m dispatching an unlimited number of threads, but dispatches the computation over a reasonable number of threads, possibly just one. Hell, my ideal world is green threads, actually. Perhaps I should have lead with that…

                                                                            Then again, I don’t know the details of the tradeoffs involved. Whatever let me solve the 1M connections cleanly and efficiently works for me.

                                                                  2. 5

                                                                    I agree with @milesrout. I don’t think Rust is a good replacement for C. This article goes into some of the details of why - https://drewdevault.com/2019/03/25/Rust-is-not-a-good-C-replacement.html

                                                                    1. 17

                                                                      Drew has some very good points. Its a shame he ruins them with all the other ones.

                                                                      1. 25

                                                                        Drew has a rusty axe to grind: “Concurrency is generally a bad thing” (come on!), “Yes, Rust is more safe. I don’t really care.”

                                                                        Here’s a rebuttal of that awful article: https://telegra.ph/Replacing-of-C-with-Rust-has-been-a-great-success-03-27 (edit: it’s a tongue-in-cheek response. Please don’t take it too seriously: the original exaggerated negatives, so the response exaggerates positives).

                                                                        1. -3

                                                                          Drew is right and this article you link to is just blatant fanboyism. It’s the classic example of fanboyism because it tries to respond to every point, yet some of them are patently true. Like, really? You can’t argue that Rust is more portable than C on the basis that there’s a little bit of leaky abstraction over Windows-specific stuff in its standard library. C is just demonstrably more portable.

                                                                          It criticises C for not changing enough, but change is bad and C89 is all C ever needed in terms of standardisation for the most part. About the only useful thing added since then was stdint.h. -ftrapv exists and thus wanky nonsense about signed overflow being undefined is invalid.

                                                                          I love this bit in particular:

                                                                          In C I could use make, gnu make, cmake, gyp, autotools, bazel, ninja, meson, and lots more. The problem is, C programmers have conflicting opinions on which of these is the obvious right choice, and which tools are total garbage they’ll never touch.

                                                                          In Rust I can use Cargo. It’s always there, and I won’t get funny looks for using it.

                                                                          In C you can use whatever you like. In Rust, if you don’t like Cargo, you just don’t use Rust. That’s the position I’m in. This isn’t better.

                                                                          1. 11

                                                                            I didn’t read that post as blatant fanboyism, but if someone’s positive and successful experience with Rust is fanboyism, let’s agree to disagree for now.

                                                                            It criticises C for not changing enough, but change is bad and C89 is all C ever needed in terms of standardisation for the most part.

                                                                            Change isn’t necessarily bad! With a few exceptions for libraries/applications opting into unstable features, you can compile and use the same Rust code that was originally authored in 2015. However, some of the papercuts that people faced in the elapsed time period were addressed in a backwards-compatible way.

                                                                            About the only useful thing added since then was stdint.h. -ftrapv exists and thus wanky nonsense about signed overflow being undefined is invalid.

                                                                            Defaults matter a great deal. People have spent a heroic amount of work removing causes of exploitable behavior in “in-tree” (as much as “in-tree” exists in C…) with LLVM/ASAN, and even more work out-of-tree with toolkits like CBMC, but C is still not a safe language. There’s a massive amount of upfront (and continuous!) effort needed to keep a C-based project safe, whereas Rust works for me out of the box.

                                                                            In C you can use whatever you like. In Rust, if you don’t like Cargo, you just don’t use Rust. That’s the position I’m in. This isn’t better.

                                                                            My employer has a useful phrase that I’ll borrow: “undifferentiated heavy lifting”. I view deciding which build system I should use for a project as “undifferentiated heavy lifting”, as Cargo covers 90-95% of the use cases I need. The remainder is either patched over using ad-hoc scripts or there is an upcoming RFC addressing that. This allows me to focus on my project instead spinning cycles wrangling build systems! That being said, I’ll be the first to admit that Cargo isn’t the perfect build system for every use case, but for my work (and increasingly, for several organizations at my employer), Cargo and Rust are an excellent replacement for C.

                                                                            1. 9

                                                                              let’s imagine I download some C from github. How do I build it?

                                                                              hopefully it’s ./configure && make && make install, but maybe not! Hopefully I have the dependencies, but maybe not! Hopefully if I don’t have the dependencies they are packaged for my distro, but maybe not!

                                                                              let’s imagine I download some rust from github. How do I build it?

                                                                              cargo build --release

                                                                              done

                                                                              I know which one of those I prefer, personally

                                                                              1. -1

                                                                                You read the README. It says what you need to do.

                                                                                cargo build –release

                                                                                This ease-of-use encourages stuff like needing to compile 200+ Rust dependencies just to install the spotifyd AUR package. It’s a good thing for there to be a bit of friction adding new dependencies, in my opinion.

                                                                                1. 13

                                                                                  So the alternative that you propose is to:

                                                                                  1. Try to figure out which file(s) (if any) specify the dependencies to install
                                                                                  2. Figure out what those dependencies are called on your platform, or even exist.
                                                                                  3. Figure out what to do when they don’t exist, if you can compile them from source, how, etc
                                                                                  4. Figure out which versions you need, because the software may not work with the latest version available on your platform
                                                                                  5. Figure out how to install that older version without breaking whatever your system may have installed, making sure all your linker flags and what not are right, etc
                                                                                  6. Figure out how to actually configure/install the darn thing, which at this point is something you have probably lost interest in.

                                                                                  Honestly your argument that ease of use leads to 200+ dependencies is a weak argument. Even if all projects suffered from this, from the user’s perspective it’s still easier to just run cargo build --release and be done with it. Even if it takes 10 minutes to build, that’s probably far less time than having to do all the above steps manually.

                                                                                  1. 7

                                                                                    Dude everyone here has had to install C software in some environment at some point. Like we all know it’s not “just read the docs”, and you know we know. What’s the point of pretending it’s not a nightmare as a rule?

                                                                                2. 7

                                                                                  Sorry you got downvoted to oblivion. You make some good points, but you also tend to present trade-offs and opinions as black-and-white facts. You don’t like fanboyism, but you also speak uncritically about C89 and C build systems.

                                                                                  For example, -ftrapv exists and indeed catches overflows at run time, but it also doesn’t override the C spec that defines signed overflow is UB. Optimizers take advantage of that, and will remove naive checks such as if (a>0 && b>0 && a+b<0), because C allows treating it as impossible. It’s not “wanky nonsense”. It’s a real C gotcha that has lead to exploitable buffer overflows.

                                                                                  1. 6

                                                                                    -ftrapv exists and thus wanky nonsense about signed overflow being undefined is invalid.

                                                                                    Nope, the existence of this opt in flag doesn’t make the complaints about signed overflow nonsensical. When I write a C library, I don’t control how it will be compiled and used, so if I want any decent amount of portability, I cannot assume -ftrapv will be used. For instance, someone else might be using -fwrapv instead, so they can check overflows more easily in their application code.

                                                                                    In C you can use whatever you like.

                                                                                    So can I. So can they. Now good luck integrating 5 external libraries, that uses, say CMake, the autotools, and ninja. When there’s one true way to do it, we can afford lots of simplifying assumption that make even a non-ideal one true way much simpler than something like CMake.

                                                                                    (By the way, it seems that in the C and C++ worlds, CMake is mostly winning, as did the autotools before, and people will look at you funny for choosing something else.)

                                                                                    1. -2

                                                                                      I think I’m done discussing anything remotely controversial on this website. I’m going to get banned or something because people keep flagging my comments as ‘incorrect’ when they’re literally objective fact just because they can’t handle that some people don’t like Rust. It’s just sad. I thought this site was meant to be one where people could maturely discuss technical issues without fanboyism but it seems like while that’s true of most topics, when it comes to Rust it doesn’t matter where you are on the internet: the RESF is out to get you.

                                                                                      It’s not like I’m saying ‘RUST BAD C GOOD’ or some simplistic nonsense. I’ve said elsewhere in the thread I think it’s a great alternative to C++, but it’s just so fundamentally different from C in so many ways that it doesn’t make sense to think of it as a C replacement. I’d love to see a language that’s more like ‘C with lifetimes’ than Rust which is ‘C++ with lifetimes’. Something easier to implement, more portable, but with those memory safety guarantees.

                                                                                      1. 12

                                                                                        I thought this site was meant to be one where people could maturely discuss technical issues

                                                                                        It is. Maturity implies civility, in which almost every comment I read of yours is lacking, regardless of topic. Like, here, there are plenty of less abrasive ways of wording what you tried to say (“wanky nonsense” indeed). Then you assume that you are being downvoted because you hurt feelings with “objective facts” and everyone who disagreed with you is a fanboy, without considering that you could simply be wrong.

                                                                                        Lobste.rs has plenty of mature technical discussion. This ain’t it.

                                                                                        1. 5

                                                                                          Drew is right and this article you link to is just blatant fanboyism.

                                                                                          Is not at all objective. You are leaning far out of the window and people didn’t appreciate.

                                                                                          It’s fine to be subjective, but if you move the discussion to that field, be prepared for the response to be subjective.

                                                                                          1. 3

                                                                                            I’d like you to stay.

                                                                                            Before clicking “Post” I usually click “Preview” and read what I wrote. If you think this is a good idea, feel free to copy it :)

                                                                                            1. 2

                                                                                              A lot of the design of Rust seems to be adding features to help with inherent ergonomics issues with the lifetimes systems; out of interest, what are some of things Rust does (or doesn’t do) that you would change to make it more minimalistic?

                                                                                              I think it’s right not to view Rust as a C replacement in the general case. I kind of view it as an alternative to C++ for programmers who wanted something ‘more’ than C can provide but bounced of C++ for various reasons (complexity, pitfalls, etc).

                                                                                        2. 11

                                                                                          So many bad points from this post.

                                                                                          • We can safely ignore the “features per year”, since the documentation they are based on don’t follow the same conventions. I’ll also note that, while a Rust program written last year may look outdated (I personally don’t know Rust enough to make such an assessment), it will still work (I’ve been told breaking changes are extremely rare).

                                                                                          • C is not really the most portable language. Yes, C and C++ compilers, thanks to having decades of work behind them, target more devices than everything else put together. But no, those platforms do not share the same flavour of C and C++. There are simply too many implementation defined behaviours, starting with integer sizes. Did you know that some platforms had 32-bit chars? I worked with someone who worked on one.

                                                                                            I wrote a C crypto library, and went out of my way to ensure the code was very portable. and it is. Embedded developers love it. There was no way however to ensure my code was fully portable. I right-shift negative integers (implementation defined behaviour), and I use fixed width integers like uint8_t (not supported on the DSP I mentioned above).

                                                                                          • C does have a spec, but it’s an incomplete one. In addition to implementation defined behaviour, C and C++ also have a staggering amount of undefined and unspecified behaviour. Rust has no spec, but it still tries to minimise undefined behaviour. I expect this point will go away when Rust stabilises and we get an actual spec. I’m sure formal verification folks will want to have a verified compiler for Rust, like we currently have for C.

                                                                                          • *C have many implementations… and that’s actually a good point.

                                                                                          • C has a consistent & stable ABI… and so does Rust, somewhat? OK, it’s opt-in, and it’s contrived. My point is, Rust does have an FFI which allows it to talk to the outside world. It doesn’t have to be at the top level of a program. On the other hand, I’m not sure what would be the point of a stable ABI between Rust modules. C++ at least seems to be doing fine without that.

                                                                                          • Rust compiler flags aren’t sable… and that’s a good point. They should probably stabilise at some point. On the other hand, having one true way to manage builds and dependencies is a god send. Whatever we’d use stable compile flags for, we probably don’t want to depart from that.

                                                                                          • Parallelism and Concurrency are unavoidable. They’re not a bad thing, they’re the only thing that can help us cheat the speed of light, and with it single threaded performance. The ideal modern computer is more likely a high number of in-order cores, each with a small amount of memory, and an explicit (exposed to the programmer) cache hierarchy. Assuming performance and energy consumption trumps existing C (and C++) programs. Never forget that current computers are optimised to run C and C++ programs.

                                                                                          • Not caring about safety is stupid. Or selfish. Security vulnerabilities are often mere externalities, which you can ignore if it doesn’t damage your reputation to the point of affecting your bottom line. Yay Capitalism. More seriously, safety is a subset of correctness, and correctness is the main point of Rust’s strong type system and borrow checker. C doesn’t just make it difficult to write safe programs, it makes it difficult to write correct programs. You wouldn’t believe how hard that is. My crypto library had to resort to Valgrind, sanitisers, and the freaking TIS interpreter to eke out undefined behaviour. And I’m talking about “constant time” code, that has fixed memory access patterns. It’s pathologically easy to test, yet writing tests took as long as writing the code, possibly longer. Part of the difficulty comes from C, not just the problem domain.

                                                                                          Also, Drew DeVault mentions Go as a possible replacement for C? For some domains, sure. But the thing has a garbage collector, making it instantly unsuitable for some constrained environments (either because the machine is small, or because you need crazy performance). Such constrained environment are basically the remaining niche for C (and C++). For the rest, the only thing that keeps people hooked on C (and C++) are existing code and existing skills.

                                                                                          1. 4

                                                                                            Rust compiler flags aren’t sable… and that’s a good point. They should probably stabilise at some point. On the other hand, having one true way to manage builds and dependencies is a god send. Whatever we’d use stable compile flags for, we probably don’t want to depart from that.

                                                                                            This is wrong, though. rustc compiler flags are stable, except flags behind the -Z flag, which intentionally separates the interface between stable and unstable flags.

                                                                                            1. 2

                                                                                              Okay, I stand corrected, thanks.

                                                                                            2. 0

                                                                                              But the thing has a garbage collector, making it instantly unsuitable for some constrained environments (either because the machine is small, or because you need crazy performance).

                                                                                              The Go garbage collector can be turned off with debug.SetGCPercent(-1) and triggered manually with runtime.GC(). It is also possible to allocate memory at the start of the program and use that.

                                                                                              Go has several compilers available. gc is the official Go compiler, GCC has built-in support for Go and there is also TinyGo, which targets microcontrollers and WASM: https://tinygo.org/

                                                                                              1. 5

                                                                                                Can you realistically control allocations? If we have ways to make sure all allocations are either explicit or on the stack, that could work. I wonder how contrived that would be, though. The GC is on by default, that’s got to affect idiomatic code in a major way. To the point where disabling it probably means you don’t have the same language any more.

                                                                                                Personally, to replace C, I’d rather have a language that disables GC by default. If I am allowed to have a GC, I strongly suspect there are better alternatives than Go. (My most major objection being “lol no generics”. And if the designers made that error, that kind of cast doubt over their ability to properly design the rest of the language, and I lose all interest instantly. Though if I were writing network code, I would also say “lol no coroutines” at anything designed after 2015 or so.)

                                                                                                1. 1

                                                                                                  I feel like GC by default vs no GC is one of the biggest decision points when designing a language. It affects so much of how the rest of a language has to be designed. GC makes writing code soooo much easier, but you can’t easily put non-GC’d things into a GC’d language. Or maybe you can? Rust was originally going to have syntax for GC’d pointers. People are building GC’d pointers into Rust now, as libraries - GC manages a particular region of memory. People are designing the same stuff for C++. So maybe we will finally be able to mix them in a few years.

                                                                                                  1. 1

                                                                                                    Go is unrealistic not only because of GC, but also segmented stacks, thick runtime that wants to talk to the kernel directly, implicit allocations, and dynamism of interface{}. They’re all fine if you’re replacing Java, but not C.

                                                                                                    D lang’s -betterC is much closer, but D’s experience shows that once you have a GC, it influences the standard library, programming patterns, 3rd party dependencies, and it’s really hard to avoid it later.

                                                                                                    1. 1

                                                                                                      Can you realistically control allocations? If we have ways to make sure all allocations are either explicit or on the stack, that could work.

                                                                                                      IIRC you can programmatically identify all heap allocations in a given go compilation, so you can wrap the build in a shim that checks for them and fails.

                                                                                                      The GC is on by default, that’s got to affect idiomatic code in a major way.

                                                                                                      Somewhat, yes, but the stdlib is written by people who have always cared about wasted allocations and many of the idioms were copied from that, so not quite as much as you might imagine.

                                                                                                      That said - if I needed to care about allocations that much, I don’t think it’d be the best choice. The language was designed and optimized to let large groups (including many clever-but-inexperienced programmers) to write reliable network services.

                                                                                              2. 1

                                                                                                I don’t think replacing C is a good usecase for Rust though. C is relatively easy to learn, read, and write to the level where you can write something simple. In Rust this is decidedly not the case. Rust is much more like a safe C++ in this respect.

                                                                                                I’d really like to see a safe C some day.

                                                                                                1. 6

                                                                                                  Have a look at Cyclone mentioned earlier. It is very much a “safe C”. It has ownership and regions which look very much like Rust’s lifetimes. It has fat pointers like Rust slices. It has generics, because you can’t realistically build safe collections without them. It looks like this complexity is inherent to the problem of memory safety without a GC.

                                                                                                  As for learning C, it’s easy to get a compiler accept a program, but I don’t think it’s easier to learn to write good C programs. The language may seem small, but the actual language you need to master includes lots of practices for safe memory management and playing 3D chess with the optimizer exploiting undefined behavior.

                                                                                              1. 0

                                                                                                GCC’s configure script wouldn’t even finish before Go was already built.

                                                                                                1. 1

                                                                                                  names.sr.ht needs to be finished

                                                                                                  Yes :)

                                                                                                  1. 1

                                                                                                    Why not Oberon, the successor of Pascal?

                                                                                                    1. 5

                                                                                                      Pascal didn’t stood still when the modulas and Oberons were created. Delphi uses Objective Pascal is a pretty nice language. I haven’t used that in more than 10 years and these days, every time I have to solve some dependency hell problem in a nodejs program or ship an electron gizmo, I remember Delphi fondly.

                                                                                                    1. 15

                                                                                                      You may consider learning perl instead:

                                                                                                      • the programs are as terse as awk
                                                                                                      • much more flexible
                                                                                                      • wide set of libraries
                                                                                                      • as widely available

                                                                                                      remember the power of perls ‘while(<>) func’ which hides command line arg parsing, stdio handling, and per-line loop. i highly recommend perl cookbook, you will be amazed how practical it is. dont allow to grow your perl(or awk) programs more than 10 lines long - they become a pain to maintain as they grow.

                                                                                                      1. 7

                                                                                                        I know perl, but found that awk was much more likely to be available and it’s much faster for quick scripts because there is less overhead in running the binary. i’ve moved most of my muscle memory to reach for awk instead.

                                                                                                        1. 5

                                                                                                          Awk does have the advantage of being a smaller language. You can understand awk enough to do useful things with it in an afternoon. Perl is Byzantine in it’s complexity as a language, which always scared me off using it for one-liners

                                                                                                          1. 3

                                                                                                            This is true especially when reading Perl scripts written by someone else. On the other hand, learn Perl enough to be able to write useful oneliners and process text streams, is quite easy.

                                                                                                            1. 2

                                                                                                              You can understand awk enough to do useful things with it in an afternoon.

                                                                                                              I did that with Perl and a bunch of other languages at various times. The trick is you learn just the subset you need for structured programming, basic I/O, and whatever data format you deal with. Much tinier. Just cuz it’s there doesn’t mean you have to use it.

                                                                                                              For Perl, I also had to learn regular expressions. They kept paying off outside of Perl, though.

                                                                                                              1. 1

                                                                                                                Just cuz it’s there doesn’t mean you have to use it.

                                                                                                                Yeah, but finding a useful subset of Perl means I have to learn enough Perl to know what a useful subset would be, where awk is already that useful subset.

                                                                                                                Doesn’t mean you can’t approach it like that, for sure, but I was pleasantly surprised how easy awk was to pick up when I decided to try to learn it a while back.

                                                                                                              2. 1

                                                                                                                Perl’s most ardent users are the language’s worst enemy ;) The downside of TMTOTDI[1] is that experienced Perl hackers settle into a set of personal idioms that they are comfortable with, but that others may not be.

                                                                                                                Bondage and discipline languages with a much stricter focus on what’s “officially” idiomatic, like Python, don’t have this problem, and neither do small, focused languages like AWK.

                                                                                                                [1] “There’s More Than One Way To Do It!”

                                                                                                              3. 4

                                                                                                                One of the reasons I prefer Perl is it’s portability between BSD and Linux. Sadly, this isn’t the case with AWK due to different implementations.

                                                                                                                1. 2

                                                                                                                  scripts written for the One True awk (which most BSDs use) should work with gawk.

                                                                                                                  1. 1

                                                                                                                    There is also GNU awk as package.

                                                                                                                  2. 3

                                                                                                                    I used to think this, but after stuff like this I exited.

                                                                                                                    1. 1

                                                                                                                      Update: That diff is not very clear, but here’s the issue from another repo I own that triggered the patch: https://github.com/akkartik/wart/issues/5

                                                                                                                      1. 1

                                                                                                                        Sad that enabling warnings caused this, it’s usually a given when writing scripts.

                                                                                                                        1. 2

                                                                                                                          It was a warning for a few minor versions, and then an error at some minor version.

                                                                                                                          1. 3

                                                                                                                            Wow, a lot of sotware was affected by this, based on this google search. Looks like an Autotools artifact? Someone wanted to avoid using / as delimeters?

                                                                                                                            Edit for this specific use case, I think sed and AWK are a better fit…

                                                                                                                    1. 2

                                                                                                                      On my Quad G5 I usually ran it in Reduced performance, and Highest when I was doing a job I needed done fast. I realize CPU power management has changed greatly since 2005, but there’s definitely precedent for this in the macOS, and there’s good use cases for throttling when you know you won’t need the CPU cycles.

                                                                                                                      1. 1

                                                                                                                        I wonder what the power consumptions in reduced/highest modes on a Quad G5 are. Do you have any numbers?

                                                                                                                        1. 2

                                                                                                                          I seem to recall it was a difference on the order of 50W or so, depending on CPU load. Definitely not trivial. ISTR the Quad in Reduced drew somewhere between 200-250W per the UPS readout (it’s no longer hooked up to it, so I can’t easily check just now).

                                                                                                                      1. 1

                                                                                                                        On OpenBSD, apmd -A automatically adjusts the performance.

                                                                                                                        1. 5

                                                                                                                          macOS (obviously) also automatically performs frequency scaling, etc., based on use. Marco is asking for official support to disable Turbo Boost. This can be done with an extension, but since Apple is tightening up extensions and moving to user-space third-party extensions, this extension may not work anymore in future macOS versions.

                                                                                                                        1. 3

                                                                                                                          No strict price constraint, but aiming for somewhere in the mid 4 figures. Let’s call it a ceiling of $5000.

                                                                                                                          Ouch.

                                                                                                                          1. 1

                                                                                                                            Does anybody know why none of the German car makers are MISRA members? Do the they use another coding standard?

                                                                                                                            1. 5

                                                                                                                              I worked for one German auto industry company, and we did work according to MISRA-C:2003 if i recall the specific revision well. We had static analysis enforcing MISRA, and in the extremely rare cases when a violation was deemed necessary extensive safety evaluation was carried out, and the violation had its justification, and its tradeoffs, risks documented, and approved, which was also audited by independent experts.

                                                                                                                              I found MISRA a pretty good collection helping to create more robust software.

                                                                                                                              1. 3

                                                                                                                                In this MISRA document appendix A is an example for “in the extremely rare cases when a violation was deemed necessary extensive safety evaluation was carried out, and the violation had its justification, and its tradeoffs, risks documented”.

                                                                                                                                1. 2

                                                                                                                                  Wow. Thanks a lot for this real world use case.

                                                                                                                                2. 3

                                                                                                                                  The talk mentions the AUTOSAR Standard and that it tries to modernize MISRA. AUTOSAR is big in Germany. So maybe it is indirect. I asked around at work but got no answers.

                                                                                                                                  MISRA compliance is important for us (and our customers). However, we do have an internal standard based on MISRA and a few other ones. I guess as a big organization, the Not-Invented-Here syndrome is stronger.

                                                                                                                                  1. 1

                                                                                                                                    Thanks a lot. Somehow I managed to ignore AUTOSAR while being interested in secure software. (Maybe that’s caused by spending to much time on English forums - where, it seems, MISRA dominates).

                                                                                                                                1. 3

                                                                                                                                  My main experience of MISRA is they are trying to force people to write in a language that isn’t C. It might be compiled by a C compiler, but it’s an arcane (mostly) subset of C.

                                                                                                                                  Admittedly C is a shitty language for safety critical software, but the fix isn’t coding standards.

                                                                                                                                  The fix is to use a language that doesn’t have all those footguns.

                                                                                                                                  eg. D or Rust.

                                                                                                                                  1. 7

                                                                                                                                    Which D or Rust compilers possessing certifications necessary for safety-critical applications do you suggest for this? Do they support the target architecture used?

                                                                                                                                    I don’t have fond memories of the tools we worked with when I worked on a car component’s firmware, yet I understand why accountability is important in safety critical systems, and very few vendors are willing to pay the costs of verifying their toolchain, and selling that. Open Source simply does not cut it there, with the famous “NO WARRANTY WHATSOEVER” phrase in each and every license. Being responsible for your own code is just enough of a burden, when you may even end up in jail if your product malfunctions in some disastrous way.

                                                                                                                                    I also think that C is not a very good language for writing robust code, as it has too many traps, yet currently that is one of the few languages mature enough (with regards to tooling and accumulated knowledge of its pitfalls) available for such work (Ada being another one I know of). On the longer run I’m sure Rust will make its way to the embedded and safety critical scene, but is not there yet. It is not matured enough for other vendors to build a compiler and have it audited and certified. Also it does not have enough developers yet, so the customer side is also not ready yet for adopting it. Maybe a decade later.

                                                                                                                                    Until then coding standards are a tool to ensure better quality code we already posses, and it is known to work, albeit admittedly with great overhead.

                                                                                                                                    1. 2

                                                                                                                                      certifications necessary for safety-critical applications. Open Source simply does not cut it there, with the famous “NO WARRANTY WHATSOEVER” phrase in each and every license

                                                                                                                                      I always wondered about that.

                                                                                                                                      One of the reasons I switched wholly to open source was I entirely disinterested in warranties. I wanted my code to work, not someone to blame when it didn’t.

                                                                                                                                      I found over a decade ago no warranty, open source code, was usually less buggy than paid for code.

                                                                                                                                      Especially compilers.

                                                                                                                                      Why? More users, more people creating compelling test cases, more people looking at the code.

                                                                                                                                      I have received patches for everything from libraries I use, down to the kernel, in my mailbox, the next day… simply by writing a compelling test case for a bug that was bothering me… and submitting that to an email list.

                                                                                                                                      In other cases I fixed it myself.

                                                                                                                                      In the close source projects where massive amounts of monies were paid… the usual response is “it will be fixed in the next release”….

                                                                                                                                      As promised by a customer service rep who actually hadn’t a clue what I was talking about.

                                                                                                                                      But then it’s almost irrelevant. Yes, I have been hit by compile bugs in my life. About three orders of magnitude less than common or garden homegrown bugs.

                                                                                                                                      What finds and fixes bugs before they even get to release builds? Compiler warnings. These days they are bloody marvellous. Every new release of gcc they come up with more. Every release I run around our code base fix up the new warnings. Every time I find a couple of real live bugs.

                                                                                                                                      Alas, the C language design disables a lot of what the compiler could warn about.

                                                                                                                                      Unit tests, especially with valgrind. Valgrind is worth way more than any certified compiler (or coding standard).

                                                                                                                                      The GNU D is a front end for the gcc compiler, so whichever targets the backend supports (ie. more than any other compiler) in principle D supports. There is some fine print around the library runtime support though for odd targets though. Support for arm does exist.

                                                                                                                                      1. 3

                                                                                                                                        Well we have also met some bugs in the Green Hills compiler we used in a project, and actually the code was compiled with multiple compilers and large parts tested on multiple architectures. C also had this advantage, that you could cross check compilers. (In-system tests were single-compiler tested, but 95% of the code was tested on multiple compilers on the project I worked on)

                                                                                                                                        Still I can only recall Ada a mature and established embedded language with multiple compilers available… (Also avoiding vendor lock in at least as an option is important at a large project)

                                                                                                                                        edit: BTW oftentimes regulators may also demand you use certified tooling. (I’m not totally sure, but lots of regulations applied to the automotive industry, AFAIK developing according to MISRA was a regulatory requirement)

                                                                                                                                        1. 1

                                                                                                                                          C also had this advantage, that you could cross check compilers.

                                                                                                                                          D has two compilers the DMD and the gnu D compilers so you have that advantages two open source compilers.

                                                                                                                                          Certainly if you’re multithreading (heaven forfend, wash your mouth out if you do “safety critical” and “multithreaded” together), running your code under every damn target, os, compiler and number of CPU’s you can find does wondrously at shaking bugs out of the tree.

                                                                                                                                          BTW oftentimes regulators may also demand you use certified tooling

                                                                                                                                          BTW oftentimes regulators may also demand you use certified tooling suffer from regulatory capture

                                                                                                                                          1. 3

                                                                                                                                            I see you really want to believe, yet still that is not how the industry works. Safety critical embedded development is quite different from other areas of the software industry. It is more conservative, independently of regulatory capture’s involvment or not. Projects move slowly and lot of stuff has to be decided up front.

                                                                                                                                            I don’t know why you mention multi-threaded programming, it was not considered anywhere in the discussion. Using multiple architectures and compilers can help catching other subtle bugs around memory alignment, endiannes and such things, which are common in low level code. Or in the compilers themselves.

                                                                                                                                            D might have two open source compilers, but that is only one aspect of the problem. D has not proven in that field, there are no expert and seniors in D, and the industry is not backing it. However better it may be (I believe it is not revolutionary enough to justify investment, Rust is more promising), nobody will risk its billion euro project for this, when C or ADA is already known and proven with all its pros and cons.

                                                                                                                                            A new generation of programmers needs to come and they must push safer languages, and their tooling must meet the regulatory requirements. If you don’t like regulators, and government, so be it, yet those are the rules, and if you want to sell stuff as a automotive/aerospace component, you must play by those rules.

                                                                                                                                            1. 2

                                                                                                                                              I been on this hamster wheel for a few turns of the cycle.

                                                                                                                                              The way it works is the bureaucracy ramps up and up and the costs ramp up and up…..

                                                                                                                                              ( Here’s a very very old joke….

                                                                                                                                              What do you call a MilSpec mouse?

                                                                                                                                              An Elephant )

                                                                                                                                              Until the users note that commodity devices are way way way cheaper, way way more functional, and actually way less buggy.

                                                                                                                                              The regulations and red tape remain in place, but the users aren’t actually using those devices.

                                                                                                                                              Eventually someone renames that class of devices the users are using so the old regs don’t apply…. and the slow the cycle starts up on the new class of devices.

                                                                                                                                              Sigh.

                                                                                                                                              Gone round that wheel a couple of times. On the “users are starting to ignore us and all the regs” phase of the current one.

                                                                                                                                      2. 1

                                                                                                                                        Which D or Rust compilers possessing certifications necessary for safety-critical applications do you suggest for this? Do they support the target architecture used?

                                                                                                                                        Ada/SPARK has you covered. I think they also have compilers to C for event the target isn’t supported.

                                                                                                                                        1. 1

                                                                                                                                          Sure, but it may be more dificult to find experienced workforce for Ada. (Note that I also mentioned Ada as an available mature tool)

                                                                                                                                          1. 1

                                                                                                                                            The bootstrap problem is best solved by just teaching people how to use the better tools. Like Jane St does. They can do mock or just non-critical stuff for the learning projects.

                                                                                                                                            1. 4

                                                                                                                                              If you have people expert in those better tools, and confidence in those tools. Being revolutionary on the internet is easy, but then Euro billions are at stake you tend to be more conservative. Especially if you are German. :)

                                                                                                                                              For non-critical stuff it was free for all, we used Python, Java, Excel/VBA, even PHP. For safety critical stuff only approved tools were used.

                                                                                                                                              Lots of Simulink was used BTW, as most of the complexity was in control system. Simple stuff like hardware drivers were fine in C, and I understand the decision, as it was easy to kickstart the project back in the past with C..

                                                                                                                                            2. 1

                                                                                                                                              Ada is not used in automotive as far as I know. Apparently, Toyota made a push a few years ago but it did not catch on.

                                                                                                                                        2. 3

                                                                                                                                          Michael Wong called it “MISRAable code”. 😉

                                                                                                                                          1. 1

                                                                                                                                            A common anti-pattern I have seen in “MISRAable code” is that it’s correct in the small and wrong in the large. ie. much thought and care had been invested in ticking every rule for that particular function, but when you look at all the invocations of it, you know things aren’t going to quite work, and certainly not reliably.

                                                                                                                                            1. 4

                                                                                                                                              I find MISRA warnings often have a quick fix which makes the code worse. However, if you think further it may also reveal a reasonable design issue. Of course, a design (or architecture) issues are more work to fix, so the quick fix is tempting.

                                                                                                                                              Example: We use object-oriented state machines a lot. This means every state is a member variable. At some point someone wanted to iterate over the states and thus put them in an array. Our checker then came up with some lifetime issue. The quick fix was some casting trickery. The real problem is about the design. We have ownership confusion: Is a state object owned by the array or by the object?

                                                                                                                                              (Btw learning some Rust has helped me for C++. Concepts like ownership and borrowing are useful terms.)

                                                                                                                                              1. 2

                                                                                                                                                I know the situation you are talking about. This also depends on the review culture of the org. We have thrown out lots of “clever” code (MISRA passing) at the review phase.

                                                                                                                                                MISRA is just one element in a puzzle, and while some of its rules are totally straightforward (eg. don’t use if(a() == b()) because order of evaluation is undefined, which can cause problems if they have side-effects), but usually they should make you stop and consider what are you about to do and why? They are indicators of code smells. Just as you said: the real problem usually is the design.

                                                                                                                                          2. 3

                                                                                                                                            Just by following the Directive 4.12 (Dynamic memory allocation shall not be used) you can eliminate quite a few footguns, I think.

                                                                                                                                            As the article says, MISRA “wrote out any and all ways to make a mistake”, which is a treasure trove by it’s own; whether you enforce MISRA-C in your project or not.

                                                                                                                                            1. 1

                                                                                                                                              Translation: Thou shalt effectively re-invent (poorly) your own schemes to solve the problem hand carved out of a raw emasculated C.

                                                                                                                                              Yes, malloc() is an atrocious design, I could rant on for hours about it’s flaws.

                                                                                                                                              What it needs to provide, is out of the box, a collection of better, safer, more effective tools, not merely don’t use the sharp ones..

                                                                                                                                              C++ is heading in the right direction here…

                                                                                                                                              https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md

                                                                                                                                              ..they are aiming for machine checkable standards complete with supporting libraries of better alternatives.

                                                                                                                                              1. 2

                                                                                                                                                I have never seen the need for dynamic allocation in safety critical code. Everything had fixed buffer sized, fixed sampling rates, fixed response times.

                                                                                                                                                Could you provide an example where dynamic allocation is necessary in safety critical context?

                                                                                                                                                1. 2

                                                                                                                                                  Typically where you have a resource limited device that has different modes and can only be in one at a time.

                                                                                                                                                  Yup, the joys of embedded, we will always be pushed to deliver mechanically smaller, lower power, cheaper unit cost devices.

                                                                                                                                                  Always.

                                                                                                                                                  No matter what Moore’s law gives us.

                                                                                                                                                  Another common place is resource pools. eg. in an IP connected device, you’ll have a fixed size pool of ram for packets. If you get flooded with more packets than you have in your pool, you just drop them. So note…

                                                                                                                                                  • Malloc is a shitty solution as you can make a trivial DoS attack the rest of your system by flooding with packets, so MISRA is right so far, don’t do that.
                                                                                                                                                  • The packets are typically allocated when/where they arrive in the stack and deallocated when/where they leave (or acked). ie. It’s dynamic allocation.

                                                                                                                                                  Yup, I’m most familiar with the insides of an IP stack, but I bet other packet based protocols will have something similar.

                                                                                                                                                  1. 2

                                                                                                                                                    Well, when I was working on auto parts (5+ years ago) our component did not need an IP stack. (Thank God!) I understand that nowadays everything is connected, but are those devices also safety critical? Anyways, where I worked MISRA was a fine aid in our work, gave a sane baseline for automatically checkable quality, so the testers could focus on tests for larger scale logic problems (which was mentioned in a different thread).

                                                                                                                                                    1. 1

                                                                                                                                                      I’m not saying MISRA doesn’t make things safer, it certainly does make things safer than using C without anything else.

                                                                                                                                                      I’m saying it’s a bad tool for the job. It’s a bandaid, it’s security theater.

                                                                                                                                                      There are a lot better ways of making things safer.

                                                                                                                                                      Alas, where MISRA makes things less safe, it allows butt coverers to say, “Job Done, the safety risks with using C have been addressed.”

                                                                                                                                                      Which is simply not true, not even close.

                                                                                                                                                      I use IP stack as an example,, as I have the code in front of me. But I bet any packet based protocol based on top of say CAN bus will have the same.

                                                                                                                                                      1. 3

                                                                                                                                                        MISRA should be a part of a safety net, not the sole item providing safety. Safety critical code must be designed and implemented in such manner. The regulations usually also require development practices, independent audits, tests, requirements traceability, and safety experts who are liable for the safety of the system.

                                                                                                                                                        If at your project safety criticality is claimed after passing a set of static tests for coding style (MISRA), than that sure is security theater, but also that is a serious deviation from the norms experienced in the European automotive industry. With such attitude a component cannot qualify for roadworthiness in the EU.

                                                                                                                                                        Our CAN stack used fixed sized buffers, the whole system was hard realtime, no dynamic allocation happened.