1. 26
  1.  

  2. 32

    This is just complete bullshit right from the beginning. All relevant operating systems are built with C and C++. Including practically every single cool new hipster language you have, and if you disagree, take a look at what LLVM is written in. I think maybe only Go is an exception. For this one simple reason, you cannot compare them to COBOL, which is well separated.

    Remember that we’re building layers on top of layers today. The legacy of C is here to stay for a few more decades at minimum, enjoying strong health.

    1. 9

      The author devotes a section to this very point. tl;dr new systems won’t be in C and C++, and the legacy base of all those layers can and will be rewritten.

      1. 10

        That’s not going to happen in 2017 though..

        1. 17

          More seriously, the author has written an awful lot about the ills of C language family. The author also co-created Cyclone, was a member of the team that built the original SML compiler, and built a cross-compiler from C to Go that also happens to enable secure multiparty computation.

          Completely dismissing the author’s arguments in the above knee-jerk fashion, in my opinion, says more about the commenter. It was difficult for me to not take off and downvote the comment from orbit. It’s the only way to be sure.

          1. 21

            Yeah, who could possibly summarily disown somebody who ends a post with:

            Wake up, sheeple: C/C++ must die.

            I get the whole C delenda est urge, but sanctimoniously acting like people aren’t writing new systems in C (they are) because alternative languages are near good enough (they aren’t) and because people genuinely care about security (not even close) is just plain wishful thinking, and I think @premysl is reasonable in saying as much.

            1. 10

              Damn @angersock, I didn’t expect you— of anyone here— to write off someone’s otherwise good discourse because of their irrational opinions. :-( Anyway…

              I think the author recognizes new systems are being built in C and is trying to convince readers against that decision. I think that because of the section entitled “How to accelerate the death of C/C++.”

              As for the alternatives being good enough, I think the author and you are in violent agreement:

              Everything I have said above argues that a safe systems language, if it can be built, will displace C and C++. And this is going to happen.

              Meaning one hasn’t been built yet. To me, that statement excludes the “cool new hipster languages” of the moment.

              Sadly, I’m with you on the people not giving a shit about security. But I don’t think @premysl brought that up…

              1. 12

                The article is just full of quotes that I get an allergic reaction to:

                Unsafe languages are too hard to use

                C, maybe, and yet there is tremendous amount of new stuff written in it. Is it a proof to the contrary?

                vast bulk of new code is not being written in C/C++

                Only most of the infrastructure everything else relies on… Google Fuchsia is a new OS… with a kernel written in C?!

                New programs are being written in safe languages, because they prevent many of the bugs of C/C++

                Purely wishful thinking, that’s rarely the reason.

                Programmers love safe languages.

                Programmers don’t care, really. Has that person ever met any regular nine to fivers that keep the world spinning?

                And legacy programs do get rewritten. Programmers love to reimplement programs!

                Most business applications are arguably even impossible to rewrite because of the costs implied.

                And the biggest BS is the clickbait title to cover it all.

                And yes, I know how terrible C/C++ projects are at security. That’s besides the point.

                1. 2

                  C, maybe, and yet there is tremendous amount of new stuff written in it. Is it a proof to the contrary?

                  Are there more new projects being started in C now as compared to last year? Ten years ago?

                  Google Fuchsia is a new OS… with a kernel written in C?!

                  I was going to mention Fuchsia in a sibling thread. The kernel is C, but they’ve heavily invested in ensuring the vast bulk of the OS is language agnostic.

                  All your other counterpoints seem on to me. This article is clickbait.

                  I wish more programmers valued quality over perceived ease. But, hey, deadline is looming and gotta have something for the boss.

                  1. 5

                    The kernel is C, but they’ve heavily invested in ensuring the vast bulk of the OS is language agnostic.

                    As opposed to Linux, which enforces… uh.. nothing on the rest of the OS. It would be a huge amount of work, but there’s no theoretical issue with removing every line of C in Linux’s userspace.

                    1. 4

                      I suppose it depends on what you mean by using C. Even most software written in non-C languages ultimately goes via C for the syscall interface, because the interface is so C-oriented that it’s difficult (though not impossible) to do otherwise. Most languages will implement syscalls either via a small shim written in C in an otherwise self-hosting compiler, or via autogenerated C bindings through an FFI system.

                      Two reasons this is difficult to avoid: 1) The Linux API is entirely based around C data structures, including structs, various kinds of pointers, etc. So you either have to go via C, or have an embedded implementation of these C structures in your language, which it seems few people want to do. 2) The official formal documentation of the API is specified in C headers, so the easiest way to make sure you’re correctly interfacing with, say, the inotify call, is to load the sys/inotify.h header, at which point you’re doing C.

                      Examples of doing it this way include the LuaJIT syscall API (literal C source code embedded in Lua strings!), the various Haskell syscall bindings (FFI-based), the SBCL syscall bindings (C shim), etc. The only example I know of that goes 100% directly with no C involved was an old implementation of syscalls for the CMUCL common lisp system.

                      1. 2

                        I have written a language (Myrddin) that bypasses libc. Go does the same. And there are probably others out there.

                        1. 2

                          Adding on to your excellent response:

                          Fuchsia is built on a C microkernel. Its division of labour between kernelspace and userspace is very different than with Linux.

                  2. 8

                    Damn @angersock, I didn’t expect you— of anyone here— to write off someone’s otherwise good discourse because of their irrational opinions. :-(

                    Fair point, I’m usually much more charitable. Mostly, it’s because his tone in the entire piece is predicated on the assumptions:

                    • Security matters more than shipping software/supporting legacy developers, which is empirically false
                    • COBOL is well and truly dead, as opposed to just really dug in in places where it’ll probably never be displaced
                    • People are not actively writing more and more potentially insecure things every day or are willing to switch, which is patently false if you look at the commit activity on Chrome, Linux, or ‘BSD, or most modern game development (go ask Mike Acton about migrating Insomniac to Rust, and what that might look like)

                    And frankly, even if he’s right–which he probably is–the simple fact is that he’s living in a world where people will learn a new language just to improve security and somehow apply that to new projects and somehow waste cycles reimplementing all the boring shit we’ve come to depend on.

                    How long, despite knowing better, did it take for people to give enough of a shit to start fixing OpenSSL? How long, despite having the bloody source code, has it taken people to fix the problems of <insert your favorite whipping post of Linux kernel and driver dev here>?

                    That’s probably the most galling bit: the idea that the magic pixie dust of open source, combined with the light vapors of safe languages, will just naturally erode away the impacted garbage that all of modern computing is built on.

                    Not enough people care, and really, never will.

                    1. 3

                      Security matters more than shipping software/supporting legacy developers, which is empirically false

                      It’s empirically false to date. Every day attack technologies improve and the public Internet becomes more hostile. In the age of Mirai it’s very visibly unsafe to put devices with default passwords on the Internet - and better still we’re starting to see some rumblings of consequences in terms of lawsuits, ISPs disconnecting subscribers and so on. My guess - and it is little more than a guess and perhaps a hope - is that such attacks will move up the stack, and we’ll reach a point where putting old versions of Linux or OpenSSL on the Internet is unsafe-with-consequences, and then where putting software with latent vulnerabilities on the Internet is unsafe-with-consequences.

                      That’s probably the most galling bit: the idea that the magic pixie dust of open source, combined with the light vapors of safe languages, will just naturally erode away the impacted garbage that all of modern computing is built on.

                      Safe languages are happening and are working, in the commercial sector as in open source. I don’t see rewriting all the C code happening so much as sandboxing it where it can’t do any harm and/or bypassing it.

                      Yes, there are tragedies of the commons around Linux and OpenSSL. Yes, it will take longer than any of us would like to fix any of this stuff. But it will happen. In a way I’m glad there’s a functioning market in DDoS attacks, as that means we’re starting to see people with financial incentives to do something about software security. Money is much more effective at getting people to care than principles.

                      1. 2

                        Not enough people care, and really, never will.

                        :-(

                        I often wonder what Java’s superfund is going to look like.

                    2. 8

                      Yeah, who could possibly summarily disown somebody

                      Trevor Jim is the real deal when it comes to languages & security, you should take him seriously. Although, I get that this is rarely the case in this age of every programmer having a blog and writing authoritatively about any subject that crosses their mind.

                      Wake up, sheeple: C/C++ must die.

                      This, but completely unironically. If your first response to “stop using C” is knee-jerk defensiveness then I don’t think you’re taking memory safety as seriously as you should be. Whether or not the industry is still using it is irrelevant, and if developers don’t care about security then it is up to the community to call them out on their shit.

                      1. 2

                        His work is good (esp Cyclone) but angersock’s points on his presentation are spot on. He mostly ignores the real drivers and constraints of most real-world projects. He also sounds a bit like a snob. This usually leads to dismissal by much of the intended audience.

                        1. 2

                          Perhaps the situation with unsafe languages wouldn’t be so bad if they had an axiomatic semantics. In this setting, the meaning of a program is what you can prove from the axioms and nothing more, so it isn’t necessary to explicitly say “such and such has undefined behavior” in the language’s definition. (The last bit is mostly for the benefit of other readers, not you, of course.) I’m not saying this to dismiss mechanically enforceable notions of safety (e.g., pattern matching exhaustiveness), but rather to allow a single language to have both machine-enforced and human-enforced notions of safety.

                          But I’m afraid that the kind of programmer who advocates C and/or C++ would be even more violently against this than against Cyclone and Rust..

                    3. 10

                      Well, not with that attitude! ;-)

                  3. 13

                    I’ve spoken to a lot of people who build high-reliability systems. I’m a major fan of functional programming so I expected to hear that they use OCaml and Haskell. Nope. They use C for production code (they might use Haskell to analyze it). Code review, static analysis, and tight style guides ensure a very high degree of reliability. C is horrible and insecure under a typical software organization’s time budget but there’s nothing inherently evil about it. People exist who know how to write reliable C code… they’re just not found in the typical startup environment.

                    I think that a higher-level language is a better choice for the use case of a web programmer on a limited time budget– while highly-reliable C is possible, it’s not written quickly, and C is a terrible choice if you’re constantly adapting to new business requirements– but virtually no one uses C in that use case anymore.

                    1. 4

                      I write C++, python and Go on a daily basis. The rate of errors in each language is honestly not that dissimilar. Sure, when C++ crashes it gives a segfault and when python crashes it gives a backtrace and then stops… Which doesn’t change the fact that it’s crashed either way.

                      1. 4

                        Well it often makes the difference between leaking your secret keys (and your users' credit card numbers) versus just not running.

                      2. 4

                        They use C for production code (they might use Haskell to analyze it). Code review, static analysis, and tight style guides ensure a very high degree of reliability.

                        This is absolutely my experience. At a startup you probably don’t find yourself needing to write critical stuff in C so much because (a) you can use out-of-the-box components or (b) you just don’t have enough traffic to demand it (HN frontpage is nowhere near what I’m talking about), but when you’re Web Scale™, you get damn good people who can write damn good C, and they’re the people who make your systems run reliably and at an acceptable speed.

                        1. 3

                          Pieter Hintjens et al at iMatix did something in between that you might like: a series of high-level DSL’s for expressing many types of problems that auto-generate high-performance C. They did this for everything from web servers to messaging systems to database schemas to web apps. The main tool is in the Hacker News link below with my comment at the top having links to more information.

                          https://news.ycombinator.com/item?id=11558007

                          The GitHub page also has an excellent write-up of how they do things, pro’s, cons, fake cons (myths), etc. Only thing I never liked about it was that it was XML-based. I doubt it would have gotten even its small amount of adoption if it was another LISP variant, though. ;)

                          EDIT: Saw it wasn’t submitted to Lobsters. Added it below in case anyone wants to discuss it.

                          https://lobste.rs/s/xabvfq/gsl_universal_code_generator

                          1. [Comment removed by author]

                            1. 5

                              Depends on how far removed from the actual computer your system is. Functional languages are a useful complexity management abstraction which doesn’t map very well on the underlying hardware. C on the other hand, is mostly a notation for pointer arithmetic. If you need to set up TLBs, arbitrage a bus on hardware interrupt, do DMA or memory-mapped I/O, you’d have much easier time with something like C.

                              1. 2

                                C definitely has an advantage there but functional isn’t totally out. You do that stuff like they do in House or COGENT. In House, they made the H-Layer where one set of unsafe, low-level code handles stuff like the hardware with everything else being high-level code interacting with it through interface hopefully checking those interactions. They did an operating system. In COGENT, they feed it a combo of specs and functional language that gets transformed into assembly with proof of correctness. They did an ext2 filesystem in it.

                                http://programatica.cs.pdx.edu/House/

                                https://ts.data61.csiro.au/projects/TS/cogent.pml

                          2. 3

                            It is a rather exaggerated article.

                            Maybe C is dead like COBOL was in 1980, which is to say, displaying just the first signs of being on its way out.

                          3. 8

                            My view is that the author drastically underestimates how much momentum remains behind C and C++, especially in existing projects. There are a lot of extremely valuable C and C++ code bases under active development that would be far too expensive to redevelop or rewrite in a new language, things like kernels, compilers (eg LLVM), interpreted languages, display servers, runtime environments, and browsers (and also plenty of commercial products). The existence and continued development of these code bases will feed demand for C and C++ programmers, probably for decades.

                            As far as new projects go, there are significant areas where nothing today is challenging C/C++ as far as I know (for example embedded systems and firmware) as well as a great deal of momentum behind writing new libraries, runtime systems, and so on in C/C++. I suspect that we are several years away from a major success story for Rust (or D, or your favorite alternative) being used instead of C/C++ in its home ground of efficient, non-GC’d, compiled programming.

                            (For people who are prepared to tolerate garbage collection (and large binaries), Go is clearly already starting to supplant C and C++, as the article mentions.)

                            1. 2

                              Most of the software I write has resource and speed requirements which mean that C/C++ and rust are the only realistic options. Rust is definitely interesting but I’ve found it much more expensive to develop than C++. Rust is very significantly slower to code than the alternatives due to its very complex approach to memory management. I like that it provides some nice guarantees but in the end the additional time/expense is a killer for us. That’s the big reason why I don’t think rust is going to replace C/C++ any time soon.

                            2. 14

                              The author has subscribed fully into the ‘post truth’ doctrine, where opinions shouted loudly and frequently enough can win out over facts. This guy has cranked out at least 20 other posts like this one over the last two years, all spouting the same rhetoric. Some examples:

                              Once more unto the breach: C/C++ must die

                              C is bad and you should feel bad if you don’t say it is bad

                              A public health campaign to stop C/C++

                              What we’ve got here is a failure to communicate

                              Constructing a narrative against the use of C

                              Let’s sunset C/C++

                              … so here’s some free career advice for the younger lobsters who haven’t learned it the hard way yet.

                              Beware of Politicians in Engineer’s clothing. You should sometimes suffer their opinions because you might learn something, but never argue with them. You’ll get dirty, they’ll enjoy it, and their position won’t change.

                              1. 5

                                I agree in a way with author, but not that they are dead, right now, like COBOL. Author seems to agree with me in the end as he wrote:

                                These efforts will help, but even without them, the future of C/C++ is set in stone: death like COBOL.

                                However I believe that C and C++ had more years of active use than COBOL. Also many of those years were in times of more widespread computer use. There are more than billion Android devices out there that still use kernel written in C (and not only kernel). I dare to say that C and C++ programs have far more deployments now that COBOL ever had. Network effects are strong. I think that that C and C++ will eventually share COBOL fate, but it will take more time.

                                There are no new unsafe languages in C sense because tools that help C get around being an unsafe language are quite tied to the language. There are debuggers, profilers and valgrind. But the most important thing is an accumulated knowledge to deal with one unsafe language also in terms of tools usage.

                                Also I don’t consider many dynamic languages as usefully safe as they require more complete tests, which often are not there. At least they are usually safe against overflows and few other memory related issues.

                                1. 6

                                  However I believe that C and C++ had more years of active use than COBOL.

                                  Maybe in terms of man-years, but not calendar years. Some systems written in COBOL have been running for nearly 50+ years. These are systems that were deployed before C was even invented.

                                2. 5

                                  Working at IBM on z/OS (aka mainframes) I hear a lot about COBOL. It’s still used to run many critical systems, but there is one serious problem with it: no one can take over running these systems because no one is learning COBOL (or z/OS, for that matter). It really is the case the COBOL programmers are retiring and/or dying and there is almost no one to take the reins.

                                  I don’t think C/C++ is going to suffer from this kind of problem in the near future, but the author’s point that legacy systems get rewritten is kinda, sorta true. Some of these critical systems that are languishing with a total lack of skills are multiple decades old and a rewrite is a huge financial risk. The opportunity to rewrite them has existed for a long time and yet, for whatever reason, it hasn’t happened. Sure, programmers would love to rewrite them, but management sure doesn’t.

                                  If C/C++ are dead like COBOL, then it will be a very long time before they no longer matter.

                                  1. 6

                                    Time for me to learn COBOL and earn my stay in the country! On a related note, for my personal needs, the best C alternative is C + all the fantastic tools we now have at our disposal, that help us avoid many of these pitfalls.

                                    1. 5

                                      I’ve seriously considered learning more COBOL to stay employed as I become part of the “grey generation” of computer programmers. It really isn’t a bad idea. It’s not sexy at all, but it would pay the bills and keep me employed for some time.

                                      1. 1

                                        I considered doing it, too. However, I have to do something mentally interesting. So, my take on it was to write DSL’s in Racket/Haskell to generate COBOL from better code. Alternatively, to write reverse engineering aids or transformation tools a la Semantic Designs. Or even static analysis tools to catch the common errors I see. Do this as I’m doing the COBOL day to day. Then, the job wouldn’t be as boring. One of these might even turn into a business on the side that’s definitely not boring and also not going way since it’s tied to COBOL code that’s not going away. ;)

                                    2. 4

                                      no one can take over running these systems because no one is learning COBOL (or z/OS, for that matter)

                                      Surely this problem can be alleviated by throwing money at it?

                                      1. 3

                                        There’s plenty of money in COBOL development/maintenance. In fact, you can make a fair bit doing it. It’s just that, well, it’s not particularly fun or flashy, so no one wants to do it. Startups are still the big draw in terms of employment. No one wants to work on a mainframe, regardless of how technically interesting it actually is.

                                        1. 1

                                          They are. It’s taught at lots of community colleges & such. There’s just little interest for it vs the number of people that will be retiring.

                                      2. 7

                                        There are some good posts on this blog about parsing, but this one is gravely mistaken, probably because the author hasn’t been exposed to what’s going on in industry. C++ is alive as ever, and GROWING, due to C++11,14,17, and LLVM/Clang. It’s being used all over industry and all over open source.

                                        I’ve been watching CppCon videos and they are a testament to how much work is going into this ecosystem. It’s at least an order of magnitude more work than is going into Rust. I’m kind of sick of hearing about Rust honestly. The ratio of social media posts to actual work getting done is a little out of hand…

                                        I think Rust’s strategy of “making systems programming accessible to everyone” is questionable. How about “convincing people who ACTUALLY WRITE all the stuff we use to switch from C/C++ to Rust”. That is a much harder problem, and I’m not seeing that much success there.

                                        1. 4

                                          C with static analysis tools is still safer than most alternatives. And if we can get compiler writers to stop producing surprising “optimizations” we will plug the biggest new source of errors.

                                          1. 1

                                            C with static analysis tools is still safer than most alternatives.

                                            Citation needed. Remember to hold programmer effort/cost (and tool cost) constant when comparing. My anecdotal impression is that it is cheaper/quicker to fulfil a given set of requirements at a given level of safety using a higher-level language (e.g. OCaml), perhaps plus its own static analysis tools, than using C with static analysis tools.

                                            And if we can get compiler writers to stop producing surprising “optimizations” we will plug the biggest new source of errors.

                                            Agreed, but I think if this were possible someone would have done it by now.

                                            1. 1

                                              “Citation needed.”

                                              Why? A cursory Google on the topic for static analysis plus specific, severe errors in C language will lead to tools to spot them. Either commercial or academic. It’s a network effect that comes from how widespread and important C is. R&D steps in to solve the problems of such things.

                                              “Remember to hold programmer effort/cost (and tool cost) constant when comparing.”

                                              This is a much better point. :) Is that whole set of necessary tools free or cheap? Are the methods of coding for maximal analysis as easy as writing in (insert alternative)? Is the performance difference that important? If it is, can (insert alternative) be used with the fast path coded in C or assembly hidden behind an interface with safety checks? C looks pretty bad the more questions like this you ask unless you have a nice budget for tools and talent.

                                              1. 1

                                                A cursory Google on the topic for static analysis plus specific, severe errors in C language will lead to tools to spot them. Either commercial or academic.

                                                That demonstrates that static analysis tools exists. It doesn’t demonstrate that C with static analysis tools is “safer than most alternatives”.

                                                1. 1

                                                  The lack of safety comes from the specific defects C allows that the alternative prevent or catch at runtime. The static analysis knocks those out. Hence, C with static analysis is safer than most alternatives in its category. The safest is still SPARK where they designed it for verification in the first place then integrated powerful analysis and provers. Rust is safest on handling dynamic memory. However, its techniques might be added to C with static analysis or something like Frama-C.

                                                  1. 1

                                                    The lack of safety comes from the specific defects C allows that the alternative prevent or catch at runtime. The static analysis knocks those out. Hence, C with static analysis is safer than most alternatives in its category.

                                                    Again you’re just talking about mitigating C’s disadvantages, and leaping from “at a similar safety level” to “safer than” without any justification. If someone is claiming C-with-static-analysis-tools is “safer than most alternatives” the onus is on them to demonstrate not just that they can mitigate C’s well-known defects but also that “most alternatives” have defects that C-with-static-analysis-tools does not.

                                                    1. 2

                                                      Ok, here’s some of its current, safety benefits from the various tooling people have made:

                                                      1. Pretty much all defects that could lead to code injection are spotted with the analysis tools. The competition usually lacks some of these, uses runtime checks at too high cost, or has C in its standard libraries or VM.

                                                      2. The formal specifications of correct behavior are more easily proved thanks to highly-integrated tools like Frama-C.

                                                      3. Others can automate everything from turning it into specs (eg AutoCorres) to generating tests for it.

                                                      4. Common issues such as how stack space or worst-case timing can be easily analyzed with embedded tools. Important in real-time market.

                                                      5. A verified compiler exists to ensure translation to machine code doesn’t give it errors.

                                                      6. Multiple compilers, including SAFEcode and Softbound+CETS, exist that can make arbitrary code memory safe for security purposes.

                                                      Now, list the systems languages that make code as low-level & fast as C without a GC that has all the above attributes in its language design, compiler, and/or tooling ecosystem. I can think of one combo: Ada +SPARK. Although even it doesn’t have a verified compiler. The assembly is verified by eye against the code in the IDE.

                                                      EDIT: Maybe vyodaiken should’ve said C + safety tools/methods instead of C + static analysis. Comment would be more accurate while preserving likely original intent.

                                          2. 6

                                            I think he underestimates the new lifeblood that was injected into C++ with the new standards. C++11, C++14, C++17 all have transformed the language into something completely modern and fairly safe. As a developer of programs that take safety seriously, I believe there is more to software than safety. Safety is a necessary demon the same way insurance companies are. Nobody like insurance, nobody likes safety.

                                            1. 9

                                              No, modern C++ isn’t safe. Dangling references are still possible. Using an object after you have std::move()d it is still possible. Mutating a shared object without coordinating with other threads is still possible. Using uninitialized memory is still possible.

                                              1. 3

                                                I did say ‘fairly safe’. While you are right that all those are possible, in modern practice it’s not really a concern because with move semantics you can write more code with value semantics vs references.

                                                My point is it’s fairly safe for its expressive power.

                                                1. 5

                                                  I did say ‘fairly safe’.

                                                  Unfortunately memory safety is more like pregnancy. Either you’re safe full-stop, or you just temporarily don’t know what your CVEs are.

                                                  1. 2

                                                    That’s a great point if you’re a mathematician but not so compelling if you’re writing software for a living. The reality of modern C++ is that you don’t lose a lot of time dealing with problems which are due to a lack of memory safety. But rust costs you a lot of time in dealing with its tricky memory management. You may not ever have null pointer segfaults in rust but you’re still programming slower.

                                                  2. 2

                                                    Move semantics is a good thing. The problem is that C++ doesn’t get it right. For move semantics to work correctly, the language must statically forbid reusing objects that have been moved elsewhere, otherwise double cleanup of the same resource could still happen.

                                                    1. 3

                                                      This is a common misunderstanding of move semantics. Move from an object should leave the object in a valid but unspecified state. Meaning after you move, you can use the object after safely, but obviously won’t have the same value as before the move.

                                                      1. 2

                                                        What’s the meaning of a std::move()d thread?

                                                        1. 2

                                                          A thread no longer points to unit of execution. Empty thread

                                                          1. 1

                                                            IMO, the way you have phrased it requires too much mental gymnastics, and a far simpler way to view this is that a std::thread doesn’t always correspond to an actual thread.

                                                            1. 1

                                                              Couldn’t have said it better.

                                                              1. 1

                                                                The problem is that if a std::thread isn’t always an actual thread, then the member functions std::thread::join and std::thread::detach are unsafe to call. Pick your poison: either moves are unsafe, or all member functions of any movable-but-not-copyable class are unsafe.

                                                                1. 3

                                                                  What does “unsafe” mean in this context? If std::thread::join on a moved-from thread throws an exception, why does that make it unsafe? When I access an array in Rust with an invalid index, it panics. Does that make it unsafe to index arrays?

                                                  3. 2

                                                    Do you realize Gödel’s incompleteness theorems prove that you either have an unsafe language or you’ll suffer from the lack of expressiveness. For whatever definition of “safe”, if you can prove that your program has that feature, your proof language (i.e. your type system) is either inconsistent or incomplete.

                                                    1. 5

                                                      Yes, and I prefer incomplete.


                                                      Edit: Also see Gödel’s completeness theorem, which states that, given a first-order theory, a statement is provable iff it holds in all models. Contrapositively, a statement isn’t provable iff there is a model in which it doesn’t hold.

                                                      Now, see:

                                                      • The language specification as a theory
                                                      • Language implementations as models of the theory
                                                      • Programs that provably have defined behavior as theorems (statements that have a proof)

                                                      For every program you can’t prove correct, I can come up with a conforming language implementation in which your program breaks.

                                                      1. 4

                                                        The most wondrously remarkable result I have seen in this domain is….

                                                        The 8000th Busy Beaver number eludes ZF set theory

                                                        Utterly fascinating.

                                                        Conversely, and example of an explicitly incomplete, but very useful language is eBPF

                                                        https://en.wikipedia.org/wiki/Berkeley_Packet_Filter

                                                      2. 1

                                                        Suffer how? Idris won’t let me write code that maybe works but I don’t actually know why it works and under what conditions it would stop working (or rather, it will force me to be explicit about the fact that I don’t know why it works and under what conditions it would stop working). I don’t see that as a downside.

                                                        1. 1

                                                          But then you also can’t write an Idris compiler in Idris! We sure hope the current compiler we have works.

                                                          1. 1

                                                            But then you also can’t write an Idris compiler in Idris!

                                                            Huh? Why not?

                                                            1. 1

                                                              Because you can’t prove the correctness of a consistent type system using the type system itself. Of course, if you take all the features of Idris into account, you can implement an Idris compiler in Idris, but that’s because Idris, with all its features has an inconsistent type system (f x = f x is a proof of anything). If that’s your point, then I’d say going into an infinite loop isn’t any better than crashing with a segfault, so your choice of using Idris is a subjective trade-off.

                                                              Besides, can you really write performant code in a total dependently-typed programming language? If you have a function that accepts two vectors of the same length, it’s very hard to avoid constructing the proof of the equality of their lengths at runtime! You can’t justify spending time on constructing proofs in the world of high performance computing.

                                                              1. 1

                                                                Because you can’t prove the correctness of a consistent type system using the type system itself.

                                                                Doesn’t that only apply to systems that contain PA? Or you could use the traditional large cardinal hack.

                                                                f x = f x is a proof of anything

                                                                Yes but it’s explicitly non-total.

                                                                I’d say going into an infinite loop isn’t any better than crashing with a segfault

                                                                It’s less likely to lead to security vulnerabilities, though that’s not my main argument.

                                                                Besides, can you really write performant code in a total dependently-typed programming language? If you have a function that accepts two vectors of the same length, it’s very hard to avoid constructing the proof of the equality of their lengths at runtime! You can’t justify spending time on constructing proofs in the world of high performance computing.

                                                                Eh maybe. Of course if you define high performance computing to be computing where you need the maximum possible performance then your statement is tautologically true. But I think the niche where you need absolutely as much performance as possible shrinks every day. I’ve done heavy number crunching work (machine-learning-like stuff) where we needed to use a cluster but we were still always happy to sacrifice a little performance for a stronger correctness guarantee. And I find there is rarely a significant runtime performance impact in the first place. Why do you believe those two vectors are the same length? Maybe they’ve come from the same source, in that case you can probably pass them around in a single datatype (using a value type, so zero overhead compared to storing both) that retains the fact that they’re the same length.

                                                                1. 2

                                                                  Doesn’t that only apply to systems that contain PA? Or you could use the traditional large cardinal hack.

                                                                  Well, this branch of the discussion is happily leaving the boundaries of my knowledge, as I’m not a mathematician, I borrowed the explanation about why Idris’s type system couldn’t prove the correctness of an Idris compiler from the Idris documentation, so I really can’t reconstruct the argument from scratch. Now this is very embarassing since I can’t find a link to where it says that in the Idris documentation. I embrace my status as the town idiot in this instance :/

                                                                  It’s less likely to lead to security vulnerabilities, though that’s not my main argument.

                                                                  Well, a null pointer access is also unlikely to lead to security vulnerabilities, but a dangling pointer is a disaster of course.

                                                                  But I think the niche where you need absolutely as much performance as possible shrinks every day.

                                                                  Can you really approach anywhere near “as much performance as possible” with Idris. I don’t mean within 10%-20%, I mean 1000%. Even without having to prove myself in C++, I find it very hard to express high-level concepts in a way that avoids heap allocation. Is that really possible with Idris? “Here’s some result along with its proof of correctness, and by the way, both the result and the proof is allocated on the stack!”

                                                                  Maybe they’ve come from the same source

                                                                  I understand the argument here, but I’m really not sure if this will scale to real problems. If the proof is that simple, you don’t need as sophisticated a type system as that of Idris anyway, and if it’s more complicated, can you still avoid proof construction at runtime? Or even if the proof disappears at runtime, you might still have to pessimize your code in order to prove it. An example here would be how in C you’d use a T * and an integer of correct size to store a vector of values, but in idris you’d have to define it inductively (like how Vec is defined).

                                                                  1. 1

                                                                    Can you really approach anywhere near “as much performance as possible” with Idris. I don’t mean within 10%-20%, I mean 1000%. Even without having to prove myself in C++, I find it very hard to express high-level concepts in a way that avoids heap allocation. Is that really possible with Idris? “Here’s some result along with its proof of correctness, and by the way, both the result and the proof is allocated on the stack!”

                                                                    I don’t know. I would expect performance to be Haskell-like (i.e. 10%-20% behind C, not orders of magnitude) and most of the types to go away at runtime. Sometimes you as the programmer have to fiddle a little - code rarely needs to do lots of different things at top performance, there is always some structure to what you’re doing (if only because no human could express enough unique operations to occupy a processor for more than a few seconds), e.g. a loop, and you hoist the proof of correctness out of the loop. If you’re doing HPC you’re already in the business of hoisting constants out of hot loops and doing it with your proof witnesses is not really any different from doing it with any other value.

                                                                    I understand the argument here, but I’m really not sure if this will scale to real problems. If the proof is that simple, you don’t need as sophisticated a type system as that of Idris anyway, and if it’s more complicated, can you still avoid proof construction at runtime?

                                                                    In my experience complex problems are made up of simple problems, and having the computer do the bookkeeping for you in the simple cases lets you scale up to the complex cases more easily, to the point where it’s no bother to encode the bookkeeping in a way the computer can understand at that level as well.

                                                                    Or even if the proof disappears at runtime, you might still have to pessimize your code in order to prove it. An example here would be how in C you’d use a T * and an integer of correct size to store a vector of values, but in idris you’d have to define it inductively (like how Vec is defined).

                                                                    Maybe. I’m sure there are cases like that. At the same time in the cases where the compiler can figure out the optimized representation you get it “for free” everywhere, even in places you might not have thought of it when writing. It’s the classic developer time / CPU time tradeoff, and every day the latter gets cheaper - I was quite surprised to be doing serious matrix algebra on a cluster in Scala (even if delegating to Fortran for the actual optimized implementation), but in business terms it made complete sense - the cluster was expensive but manageable, whereas getting the case study that would show the business actually worked was a matter of the business’s survival, and we could buy more cluster capacity much more quickly than onboarding more developers.

                                                    2. 3

                                                      I share your enthusiasm for C++ but Rust’s promises WRT performant memory safety are next-level. That said, I will be waiting for indisputable signs of maturity from Rust before I even look at it. I am thinking another 5 years at least. It takes much more than memory safety to make a language usable in practice. Just imagine how much work needs to be done for a 7-year-old language (which has been stable (v1.0) for only a little more than 1.5 years!) to catch up to one which is 34 years old and has been in heavy use for most of that time.

                                                      I also wonder how long it will be before the various sanitizer tools (address, memory, undefined behaviour, integer, etc.) in clang and gcc are able to catch the same errors that the Rust compiler can. Are they even that far off today? Surely there’s no reason they won’t get there? If that happens in the near future it could hurt Rust badly.

                                                      Edit: I guess the sanitizers will never be able to do what Rust can because safety is designed into the language.

                                                      Edit2: But then again, many of the sanitizer checks are done at runtime which probably opens many language-independent possibilities, although it’s obviously inferior to having it all done at compile time and requires excellent test coverage to be effective.

                                                      1. 3

                                                        Where you see “language-independent”, I see “runtime-environment-dependent”. Which is easier to give a formal semantics (so that somebody can establish, once and for all, the way in which bugs can ruled out): a programming language or a runtime environment?

                                                        1. 2

                                                          I wasn’t trying to say that the “language-independent possibilities” were better than Rust’s built-in checks, just that it allows you to do more than you can in static analysis of C++. Probably not the best choice of words.

                                                          1. 1

                                                            I would think that proving that a language runs correctly on a runtime with particular semantics, and that a specific runtime has those semantics, wolud generally be easier than proving that the language runs correctly on a specific runtime “directly” - indeed if I had to do the latter I would probably approach it by doing the former. (And I would probably approach both halves by doing a similar “matryoshka” strategy - prove that high-level semantics hold provided that slightly lower level semantics hold, then that those hold provided some slightly lower level semantics hold, and so on down to the machine semantics).

                                                      2. 4

                                                        It’s hard to make a civilized comment towards such an ignorant rant. In the world of high-performance computing, C++ is indispensable. And no! Rust is not an alternative, because its type system is too weak to express many abstractions. And none of the GC'ed languages are alternatives either, because when you’re trying to keep 32 cores busy at the same time, you should simply stay on the stack for as long as possible (otherwise you’ll have to reach out to the main memory and it’ll become a bottleneck)!

                                                        Enough with these “I’m too ignorant to know what others do in their day jobs, so I’ll just assume they need a piece of my brain” posts.

                                                        1. 1

                                                          If you’re in HPC, what do you think about ParaSail and Chapel? ParaSail would be closer to the Rust of HPC given it’s the parallel-by-default answer to Ada, a safe-by-default language for systems programming. Chapel was Cray’s attempt at a language to replace MPI, etc. with something more high-level.

                                                          http://www.embedded.com/design/other/4375616/ParaSail--Less-is-more-with-multicore?page=0

                                                          http://chapel.cray.com/

                                                          1. [Comment removed by author]

                                                            1. 4

                                                              Garbage collection per se isn’t incompatible with stack allocation, but languages with garbage collection tend to be designed with heap allocation as the default. This is because stack-allocated values live only for the duration of the stack frame they’re allocated in (e.g. in a C-style function, until the function returns), while in garbage collected languages you usually expect to be able to return and store objects in arbitrary ways without them disappearing. They should only be deallocated when no longer reachable through any live reference, which may be after the stack frame they were first allocated in has long since returned.

                                                              Good compilers of GC’d languages can sometimes statically determine that references to a given value will never “escape” the stack frame’s lifetime, and are therefore safe to stack-allocate, but that’s difficult to do in general even when true, and in GC’d languages common style often means it’s not even true.

                                                              1. 2

                                                                I’m going to make rough generalizations in the following, but it’s in response to a rant to the tune of “I may be a kid, but I’ve learned to write a blog post!?!?!” anyway, so I believe I’m off the hook.

                                                                Garbage collection has no inherent connection to being forced out of the stack, but languages built around garbage collectors usually (usually as in virtually all the ones I know of) don’t let you make a distinction between stack and heap allocated values. The idea is either “Who needs performance anyway, CPUs are cheap!” (f.x. Ruby) , “We can sacrifice that much performance” (f.x. Haskell) or “The compiler can analyze your code and figure out what can be allocated on the stack.” (f.x. Java).

                                                                The compiler can indeed figure out what should go where under many circumstances, and that makes this mentality quite applicable to a wide class of problems, but some people really need to be very careful about memory allocation. It might, for instance, be mission critical software so you have to be sure that certain parts of the program have absolutely deterministic runtime behavior. Or maybe your code will run for years, and you don’t want to risk memory fragmentation. Or, as I said, you’re writing software to be run in a massively parallel setting, so you have to be absolutely sure about how you’re using the various CPU caches. If what you expect from the compiler is, “I’ll let it optimize and if it isn’t enough, I’ll just buy more CPUs”, then sure, stop using C++.

                                                                As for Haskell, I really really love the language, and reach for it whenever performance is not critical. Its type system is very powerful and it lets me express very abstract architectural concepts without relying on type-level hacks. You have to keep in mind, however, that when you start caring about squeezing out the last bit of performance, and when you still want to keep writing declarative and high level code, your requirements from the type system explode exponentially. Expressing business logic is hard, so you need a type system like that of Haskell, but expressing logic AND the way it should be run is a much much harder problem, so any “safe” type system will fail you (at least ALL the ones we have today).

                                                                You say “what’s special about the type system of C++ then?”: The fact that C++ is nothing more that a scripting language attached to a C compiler! It really doesn’t have a type system beyond C (I’m ignoring the OOP features). Yes, the template language is a powerful enough domain specific scripting language to generate C code. And if C++ didn’t exist, that’s how I’d be writing performant programs: Using LISP to generate C code! But I don’t do that today, because the huge pile of mess that’s C++ templates gets the job done!

                                                                1. 2

                                                                  languages built around garbage collectors usually … don’t let you make a distinction between stack and heap allocated values

                                                                  One I can think of that sort-of does is Lisp, though it’s just an annotation on variables that otherwise work identically, and you lose guaranteed memory safety if you use it. You can declare a variable as dynamic-extent, which is a promise to the compiler that it won’t escape the enclosing context. The compiler is allowed to rely on this information if it wants, even if it can’t otherwise prove it safe; SBCL, for example, does so.

                                                                  1. [Comment removed by author]

                                                                    1. 1

                                                                      I didn’t intend that sentence to refer to the character of the author. The sentence targets his post (which unfortunately, inevitably reflects on his character).

                                                                2. 1

                                                                  In the world of high-performance computing, C++ is indispensable. And no! Rust is not an alternative, because its type system is too weak to express many abstractions.

                                                                  This I would like to know more about.

                                                                  1. 2

                                                                    The main mechanism for expressing polymorphism in Rust is traits. That means you’ll be using traits a lot while writing Rust code, but the language has no features for abstracting over traits themselves. Don’t get me wrong, I like Rust and the people behind it (aside from the aggressive marketing campaign they’re following), but the type system isn’t powerful enough to replace C++ (it’s probably powerful enough for most problems, though).

                                                                    I wish I could give an example you couldn’t possibly express in Rust, but for that, I’d have to be an expert of the language, since its type system is fairly powerful. To give a rough idea though, Haskell’s type system is pretty much a superset of that of Rust’s in those things that it takes into account (f.x. Haskell doesn’t take borrow semantics into account, so it’s not a part of that statement), so one can simply check why Haskell needs those extra features to see where Rust will fall short. On top of the previous sentence, you usually need even more abstraction power when you want to express memory management in addition to high-level concepts. That’s why, I believe, we’re nowhere near a “safe” language (one whose type system guarantees the absence of race conditions and undefined behavior) when we also want control over memory management.

                                                                    1. 1

                                                                      So the specific case you are thinking of is being able to pass an ABC as a template parameter?

                                                                      1. 2

                                                                        Yes, in essence. I guess the rule of thumb is that whatever feature a language has that you can’t abstract over, you’ll bleed expressiveness. The logical extreme here is probably LISP, where you can abstract over anything, but formal verification is impossible. I like where C++ stands. You use templates to generate C code and the template mechanics are in essence an untyped scripting language. The type system of C never intended to make programming safe, it was always to enable the compiler to emit performant assembly. The philosophy of C is pure in that respect.

                                                                        There is no language today that tries to do what C++ does better. Some people look at C++ and see a type system that’s broken for the purpose of proving program correctness. I see two languages, one object language which has a very simple type system with the sole purpose of generating assembly and a meta language which doesn’t have a type system. I’m sure if some folks adopted this view and tried to modernize it, they’d come up with something much better than C/C++, but nobody is trying that as far as I can see.

                                                                        1. 2

                                                                          The “meta language which doesn’t have a type system” would seem to be something of a problem for verification (and readable error messages) but I see where you’re going: separation of concerns.

                                                                          There actually has been a lot of interesting work done on typed meta-languages for generating C, for example ImProve and Atom, but the area of application seems to be entirely embedded systems.

                                                                          Are templates guaranteed to have static dispatch? I should think so – but then you might need two systems of polymorphism, similar to what happens in Rust and Swift now.

                                                                          1. 2

                                                                            would seem to be something of a problem for verification (and readable error messages)

                                                                            Well, there are two remedies to these problems from two ends:

                                                                            1) You get some verification from the object language. From the meta-language perspective, this is similar to writing unit-tests in a dynamically typed conventional language. You write templates, and when they’re instantiated, it sort of becomes a test-case.

                                                                            2) You can embed typed languages into the untyped meta-language, and perform verification during the execution of the meta-language. This sounds complicated but it’s not that bad in C++. You define composable pieces with traits, and when composed, these pieces try to match-up those traits.

                                                                            The error messages are obviously world-famous, but you learn to read them like stack traces and carry on with your life.

                                                                            Are templates guaranteed to have static dispatch? I should think so – but then you might need two systems of polymorphism

                                                                            Yes, templates have to be dispatched statically. This is more a necessity than a feature, since you’d need a C++ compiler at runtime otherwise. On the other hand, C++’s templates being the abusable mess they are, it is in fact possible to define dynamically dispatched polymorphism on top of them! And the extent of polymorphism you can express this way is well beyond what Rusts traits can do. I’ve recently been experimenting with a library for expressing the following: You write some templates that can be instantiated with a type and generate efficient specialized assembly for that particular type, but you can also reuse the very same code to run with a type-erased value along with a runtime-representation of its type. Like, the same code accepts int as well as a data structure that hods (void *, "int").

                                                                              1. 1

                                                                                With regards to the library you describe, it seems like the kind of “shape polymorphism” in Rust that would be handled with macros. I suppose C++ templates are a kind of hygienic macros for C.

                                                                                With regards to type checking in the object language: this seems fishy since much of the fancy type information has been compiled away by the time that happens. Maybe you push down typedefs so things have nice names but do those result in actual errors? As when having a money type represented by an int32. I assume C++ actually handles this kind of special stuff by being not exactly C.

                                                                                1. 1

                                                                                  I suppose C++ templates are a kind of hygienic macros for C.

                                                                                  That’s probably quite accurate, and in fact, the early motivation for the introduction of templates was exactly that. There’s a bit more to templates though, they’re better integrated with the object language than macros usually are. For instance if you have a templated function that receives a variably-typed argument with type T, that T is deduced from the call site of that function.

                                                                                  As for Money vs int32, you’d defer type-safety to the object language by defining money as struct Money { int32 val; }; so that a plain int32 can’t be passed. As I said though, you can also mimic a custom type system during template instantion. Here’s a related article from the Racket world: http://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf

                                                                            1. 2

                                                                              the template mechanics are in essence an untyped scripting language.

                                                                              It’s important to recognize that considered as a general-purpose programming language, C++ template metaprogramming is a really bad one. It’s neither safe (you can only typecheck the complete template expansion, so essentially any invocation of a template where you haven’t tested those exact arguments might fail, and if you want to debug why a particular expansion failed there’s no way to determine whether an intermediate expansion is correct or not) nor particularly expressive (there’s a huge amount of syntax overhead to doing even the simplest things like if/else). It performs terribly too (a friend wrote a parser generator and tested it with a parser that would recognize whether a string was “K” or not; it took two hours and 512mb to run).

                                                                              There is no language today that tries to do what C++ does better. Some people look at C++ and see a type system that’s broken for the purpose of proving program correctness. I see two languages, one object language which has a very simple type system with the sole purpose of generating assembly and a meta language which doesn’t have a type system. I’m sure if some folks adopted this view and tried to modernize it, they’d come up with something much better than C/C++, but nobody is trying that as far as I can see.

                                                                              There are workflows around generating C with e.g. Haskell.

                                                                              (I do think the language we need doesn’t exist with - I want a Rust-like language with HKT in which borrows are not special but just an ordinary library implemented in the language. When I get time I want to try to implement Noether in a Rust-like style; something like that plus an explicit cost semantics a la http://www.cs.cmu.edu/~rwh/papers/iolambda/short.pdf would, I think, give a safe language that could be used for practical programming)

                                                                              1. 1

                                                                                Well it shouldn’t be considered a general-purpose programming language, it’s a special purpose programming language for generating C code. I also think your impression of of template metaprogramming in C++ is severely outdated (does you impression include C++11/14/17?). With the newer metaprogramming utilities, it’s quite easy to write complex meta-programs that are reasonably fast to compile. For example your parser example could be written with constexpr functions today to be run at compile time, and it would be blazing fast.

                                                                                Considering everything from IDEs, debuggers, build tools, existing libraries, existing programmers to C++’s handy features like constexpr (which let you share code between the meta and object languages), variadic macros etc. building a metalanguage to generate C code from scratch would be a big undertaking. You should also realize that even if you built such a library in Haskell, writing code in it won’t be as pretty as writing plain Haskell. It’s also quite unlikely to run faster than a decades old compiler like GCC.

                                                                                My point is, there’s a large class of programming tasks for which C++ is a readily available and mature tool without any existing alternatives.

                                                                    2. 0

                                                                      What a stupid click bait article. I’d hope we are above falling for this crap.