1. 4

    I spent several weeks recently wrapping a godawful cryptographic RPC service with a pythonic interface, only to discover later that vendor for said service was not forthright about their pricing or licensing restrictions. So this week I get to finish rejiggering my nice pythonic interface to work with a different, far less ugly cryptographic service.

    1. 1

      Link to the good and bad one please. No need to say which is which.

      1. 3

        One or the other: https://www.gemalto.com And another: https://aws.amazon.com/kms/

    1. 2

      Working a job during the day and saving the internet during the night. The work never ends.

      1. 5
        • nor will we defend our lack of knowledge of their hyped and volatile technologies

        Well, I guess ignorance is bliss. My suggestion is to rename this the NIHS manifesto.

        1. 4

          I disagree. I don’t need more than a superficial understanding of Blockchain to know that I don’t need/want to build my next webapp on it.

          1. 1

            Sure, but I wouldn’t either.

            This is because people have got excited that they finally understand what a block is and want to play with encryption. I would argue that blockchain isn’t anything new, it just has received a ton of popularity and a fancy buzzword now.

            It’s not like people didn’t link things together with CFB before blockchain existed. Everyone has just jumped on it because they think it’s fancy now. 🤷‍♀️

            Personally, what I see from you here is an extremeist response and a lack of accurately and fairly representing the points being discussed.

          2. 2

            I would agree that this is a weak point of the manifesto. Learning and understanding the benefits and drawbacks of a new technology should be considered a good thing and should be something to strive for, even if they’re hyped to hell and back. Every bit of software that people are using today was at some point unknown by everyone except the creator(s) themselves.

            How else will you know when something dramatically better comes along and should rightfully obsolete whatever you’re currently using? There was some guy on the internet that said 90% of all advice is useless and that we should hear as much as we can and take what is useful. While the numbers aren’t going to be the same I think the idea fits the state of software tools fairly well. Maybe one day people will realize that SOAP apis were the best tool/access method by far. I doubt it but it doesn’t mean that give the state of things way into the future it couldn’t happen.

            Don’t close off, just choose.

            1. 1

              I totally agree that understanding the trade-offs for your decisions is an essential skill, but I still feel like this article’s intention feels more like they want to persuade us all into NIHS than it even remotely represents a fair discussion about accepting poor trade-offs.

          1. 7

            I always laugh when people come up with convoluted defenses for C and the effort that goes into that (even writing papers). Their attachment to this language has caused billions if not trillions worth of damages to society.

            All of the defenses that I’ve seen, including this one, boil down to nonsense. Like others, the author calls for “improved C implementations”. Well, we have those already, and they’re called Rust, Swift, and, for the things C is not needed for, yes, even JavaScript is better than C (if you’re not doing systems-programming).

            1. 31

              Their attachment to this language has caused billions if not trillions worth of damages to society.

              Their attachment to a language with known but manageable defects has created trillions if not more in value for society. Don’t be absurd.

              1. 4

                [citation needed] on the defects of memory unsafety being manageable. To a first approximation every large C/C++ codebase overfloweth with exploitable vulnerabilities, even after decades of attempting to resolve them (Windows, Linux, Firefox, Chrome, Edge, to take a few examples.)

                1. 2

                  Compared to the widely used large codebase in which language for which application that accepts and parses external data and yet has no exploitable vulnerabilities? BTW: http://cr.yp.to/qmail/guarantee.html

                  1. 6

                    Your counter example is a smaller, low-featured, mail server written by a math and coding genius. I could cite Dean Karnazes doing ultramarathons on how far people can run. That doesn’t change that almost all runners would drop before 50 miles, esp before 300. Likewise with C code, citing the best of the secure coders doesn’t change what most will do or have done. I took author’s statement “to first approximation every” to mean “almost all” but not “every one.” It’s still true.

                    Whereas, Ada and Rust code have done a lot better on memory-safety even when non-experts are using them. Might be something to that.

                    1. 2

                      I’m still asking for the non C widely used large scale system with significant parsing that has no errors.

                      1. 3

                        That’s cheating saying “non-c” and “widely used.” Most of the no-error parsing systems I’ve seen use a formal grammar with autogeneration. They usually extract to Ocaml. Some also generate C just to plug into the ecosystem since it’s a C/C++-based ecosystem. It’s incidental in those cases: could be any language since the real programming is in the grammar and generator. An example of that is the parser in Mongrel server which was doing a solid job when I was following it. I’m not sure if they found vulnerabilities in it later.

                    2. 5

                      At the bottom of the page you linked:

                      I’ve mostly given up on the standard C library. Many of its facilities, particularly stdio, seem designed to encourage bugs.

                      Not great support for your claim.

                      1. 2

                        There was an integer overflow reported in qmail in 2005. Bernstein does not consider this a vulnerability.

                    3. 3

                      That’s not what I meant by attachment. Their interest in C certainly created much value.

                    4. 9

                      Their attachment to this language has caused billions if not trillions worth of damages to society.

                      Inflammatory much? I’m highly skeptical that the damages have reached trillions, especially when you consider what wouldn’t have been built without C.

                      1. 12

                        Tony Hoare, null’s creator, regrets its invention and says that just inserting the one idea has cost billions. He mentions it in talks. It’s interesting to think that language creators even think of the mistakes they’ve made have caused billions in damages.

                        “I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years.

                        If the billion dollar mistake was the null pointer, the C gets function is a multi-billion dollar mistake that created the opportunity for malware and viruses to thrive.

                        1. 2

                          He’s deluded. You want a billion dollar mistake: try CSP/Occam plus Hoare Logic. Null is a necessary byproduct of implementing total functions that approximate partial ones. See, for example, McCarthy in 1958 defining a LISP search function with a null return on failure. http://www.softwarepreservation.org/projects/LISP/MIT/AIM-001.pdf

                          1. 3

                            “ try CSP/Occam plus Hoare Logic”

                            I think you meant formal verification, which is arguable. They could’ve wasted a hundred million easily on the useless stuff. Two out of three are bad examples, though.

                            Spin has had a ton of industrial success easily knocking out problems in protocols and hardware that are hard to find via other methods. With hardware, the defects could’ve caused recalls like the Pentium bug. Likewise, Hoare-style logic has been doing its job in Design-by-Contract which knocks time off debugging and maintenance phases. The most expensive. If anything, not using tech like this can add up to a billion dollar mistake over time.

                            Occam looks like it was a large waste of money, esp in the Transputer.

                            1. 1

                              No. I meant what I wrote. I like spin.

                          2. 1

                            Note what he does not claim is that the net result of C’s continued existence is negative. Something can have massive defects and still be an improvement over the alternatives.

                          3. 7

                            “especially when you consider what wouldn’t have been built without C.”

                            I just countered that. The language didn’t have to be built the way it was or persist that way. We could be building new stuff in a C-compatible language with many benefits of HLL’s like Smalltalk, LISP, Ada, or Rust with the legacy C getting gradually rewritten over time. If that started in the 90’s, we could have equivalent of a LISP machine for C code, OS, and browser by now.

                            1. 1

                              It didn’t have to, but it was, and it was then used to create tremendous value. Although I concur with the numerous shortcomings of C, and it’s past time to move on, I also prefer the concrete over the hypothetical.

                              The world is a messy place, and what actually happens is more interesting (and more realistic, obviously) than what people think could have happened. There are plenty of examples of this inside and outside of engineering.

                              1. 3

                                The major problem I see with this “concrete” winners-take-all mindset is that it encourages whig history which can’t distinguish the merely victorious from the inevitable. In order to learn from the past, we need to understand what alternatives were present before we can hope to discern what may have caused some to succeed and others to fail.

                                1. 2

                                  Imagine if someone created Car2 which crashed 10% of the time that Car did, but Car just happened to win. Sure, Car created tremendous value. Do you really think people you’re arguing with think that most systems software, which is written in C, is not extremely valuable?

                                  It would be valuable even if C was twice as bad. Because no one is arguing about absolute value, that’s a silly thing to impute. This is about opportunity cost.

                                  Now we can debate whether this opportunity cost is an issue. Whether C is really comparatively bad. But that’s a different discussion, one where it doesn’t matter that C created value absolutely.

                            2. 8

                              C is still much more widely used than those safer alternatives, I don’t see how laughing off a fact is better than researching its causes.

                              1. 10

                                Billions of lines of COBOL run mission-critical services of the top 500 companies in America. Better to research the causes of this than laughing it off. Are you ready to give up C for COBOL on mainframes or you think both of them’s popularity were caused by historical events/contexts with inertia taking over? Im in latter camp.

                                1. 7

                                  Are you ready to give up C for COBOL on mainframes or you think both of them’s popularity were caused by historical events/contexts with inertia taking over? Im in latter camp.

                                  Researching the causes of something doesn’t imply taking a stance on it, if anything, taking a stance on something should hopefully imply you’ve researched it. Even with your comment I still don’t see how laughing off a fact is better than researching its causes.

                                  You might be interested in laughing about all the cobol still in use, or in research that looks into the causes of that. I’m in the latter camp.

                                  1. 5

                                    I think you might be confused at what I’m laughing at. If someone wrote up a paper about how we should continue to use COBOL for reasons X, Y, Z, I would laugh at that too.

                                    1. 3

                                      Cobol has some interesting features(!) that make it very “safe”. Referring to the 85 standard:

                                      X. No runtime stack, no stack overflow vulnerabilities
                                      Y. No dynamic memory allocation, impossible to consume heap
                                      Z. All memory statically allocated (see Y); no buffer overflows
                                      
                                      1. 3

                                        We should use COBOL with contracts for transactions on the blockchains. The reasons are:

                                        X. It’s already got compilers big businesses are willing to bet their future on.

                                        Y. It supports decimal math instead of floating point. No real-world to fake, computer-math conversions needed.

                                        Z. It’s been used in transaction-processing systems that have run for decades with no major downtime or financial losses disclosed to investors.

                                        λ. It can be mathematically verified by some people who understand the letter on the left.

                                        You can laugh. You’d still be missing out on a potentially $25+ million opportunity for IBM. Your call.

                                        1. 1

                                          Your call.

                                          I believe you just made it your call, Nick. $25+ million opportunity, according to you. What are you waiting for?

                                          1. 4

                                            You’re right! I’ll pitch IBM’s senior executives on it the first chance I get. I’ll even put on a $600 suit so they know I have more business acumen than most coin pitchers. I’ll use phrases like vertical integration of the coin stack. Haha.

                                      2. 4

                                        That makes sense. I did do the C research. Ill be posting about that in a reply later tonight.

                                        1. 10

                                          Ill be posting about that in a reply later tonight.

                                          Good god man, get a blog already.

                                          Like, seriously, do we need to pass a hat around or something? :P

                                          1. 5

                                            Haha. Someone actually built me a prototype a while back. Makes me feel guilty that I dont have one instead of the usual lazy or overloaded.

                                              1. 2

                                                That’s cool. Setting one up isn’t the hard part. The hard part is doing a presentable design, organizing the complex activities I do, moving my write-ups into it adding metadata, and so on. I’m still not sure how much I should worry about the design. One’s site can be considered a marketing tool for people that might offer jobs and such. I’d go into more detail but you’d tell me “that might be a better fit for Barnacles.” :P

                                                1. 3

                                                  Skip the presentable design. Dan Luu’s blog does pretty well it’s not working hard to be easy on the eyes. The rest of that stuff you can add as you go - remember, perfect is the enemy of good.

                                                  1. 0

                                                    This.

                                                    Hell, Charles Bloom’s blog is basically an append-only textfile.

                                                  2. 1

                                                    ugh okay next Christmas I’ll add all the metadata, how does that sound

                                                    1. 1

                                                      Making me feel guilty again. Nah, I’ll build it myself likely on a VPS.

                                                      And damn time has been flying. Doesnt feel like several months have passed on my end.

                                            1. 1

                                              looking forward to read it:)

                                      3. 4

                                        Well, we have those already, and they’re called Rust, Swift, ….

                                        And D maybe too. D’s “better-c” is pretty interesting, in my mind.

                                        1. 3

                                          Last i checked, D’s “better-c” was a prototype.

                                        2. 5

                                          If you had actually made a serious effort at understanding the article, you might have come away with an understanding of what Rust, Swift, etc. are lacking to be a better C. By laughing at it, you learned nothing.

                                          1. 2

                                            the author calls for “improved C implementations”. Well, we have those already, and they’re called Rust, Swift

                                            Those (and Ada, and others) don’t translate to assembly well. And they’re harder to implement than, say, C90.

                                            1. 3

                                              Is there a reason why you believe that other languages don’t translate to assembly well?

                                              It’s true those other languages are harder to implement, but it seems to be a moot point to me when compilers for them already exist.

                                              1. 1

                                                Some users of C need an assembly-level understanding of what their code does. With most other languages that isn’t really achievable. It is also increasingly less possible with modern C compilers, and said users aren’t very happy about it (see various rants by Torvalds about braindamaged compilers etc.)

                                                1. 4

                                                  “Some users of C need an assembly-level understanding of what their code does.”

                                                  Which C doesnt give them due to compiler differences and effects of optimization. Aside from spotting errors, it’s why folks in safety- critical are required to check the assembly against the code. The C language is certainly closer to assembly behavior but doesnt by itself gives assembly-level understanding.

                                            2. 2

                                              So true. Every time I use the internet, the solid engineering of the Java/Jscript components just blows me away.

                                              1. 1

                                                Everyone prefers the smell of their own … software stack. I can only judge by what I can use now based on the merits I can measure. I don’t write new services in C, but the best operating systems are still written in it.

                                                1. 5

                                                  “but the best operating systems are still written in it.”

                                                  That’s an incidental part of history, though. People who are writing, say, a new x86 OS with a language balancing safety, maintenance, performance, and so on might not choose C. At least three chose Rust, one Ada, one SPARK, several Java, several C#, one LISP, one Haskell, one Go, and many C++. Plenty of choices being explored including languages C coders might say arent good for OS’s.

                                                  Additionally, many choosing C or C++ say it’s for existing tooling, tutorials, talent, or libraries. Those are also incidental to its history rather than advantages of its language design. Definitely worthwhile reasons to choose a language for a project but they shift the language argument itself implying they had better things in mind that werent usable yet for that project.

                                                  1. 4

                                                    I think you misinterpreted what I meant. I don’t think the best operating systems are written in C because of C. I am just stating that the best current operating system I can run a website from is written in C, I’ll switch as soon as it is practical and beneficial to switch.

                                                    1. 2

                                                      Oh OK. My bad. That’s a reasonable position.

                                                      1. 3

                                                        I worded it poorly, I won’t edit though for context.

                                              1. 4

                                                I’m currently working on my automatic manga scanlator, warning NSFW. users upload their own content for scanlating. https://robotscanlations.com

                                                Progress is slow due to all of the different manga styles but by running each page through the system multiple times with different parameters and with a growing set of rules it’s becoming more reliable and general.

                                                1. 2

                                                  This is a rare one, you don’t hear much about C++ web frameworks much less C ones. Has anyone else used it before? I assumed that writing nginx modules or using H2o as a library and then building from there would be the way to do it in C. C++ has more options like Cutelyst or fudging about with uWebsockets + nginx. What’s more they’re low level enough that there’s minimal magic while still being able to get things done. I’ve gone and tried to use more of the options from the Tech Empower benchmarks leader board just to see what’s possible but many frameworks get fairly difficult to use or understand vs just rolling out something with python & flask pretty easily due to lack of docs.

                                                  1. 4

                                                    I use Kore for anything I do.

                                                    It’s my TLS termination point, runs my few web services, ran my html5 game, we use a slimmed down version of it at my dayjob and more.

                                                    Funny you mention Python, because Kore since a few months ago also supports handlers written in pure Python. Including await support for asynchronous pgsql queries with the built in pgsql support Kore has. It’ll be part of the next release.

                                                  1. 3

                                                    This is a nice blog post and reinforces the feeling I have that a properly configured Postgres can handle most of the search and lookup problems people have, especially if the important indexes can fit in memory.

                                                    I’ve been looking into the options for full text search lately, especially for handling both alphabetical text and asian text. So far I’ve found:

                                                    • Elastic Search (easy)
                                                    • Solr (kinda easy)
                                                    • Vanilla Lucene
                                                    • CLucene (probably dead)
                                                    • Sphinx (Stagnant, asian text is difficult)
                                                    • Manticore Search (Sphinx fork, Active, asian text is still difficult)
                                                    • Trinity (C++ multiple codecs are supported, pretty young)
                                                    • Tantivy (Rust, inspired by Lucene, not as full featured yet)
                                                    • Bleve (Go based text search indexing, young)
                                                    • Riot (Go based full text search engine, asian languages supported, young)
                                                    • Postgres (Easiest if Postgres is already being used.) (zhparser extension is possible)
                                                    • Groonga (Usable in Postgres with Pgroonga or Mysql with Mgroonga)
                                                    • Mysql (supports fts of asian languages with mecab tokenizer)

                                                    Of these, ElasticSearch (Lucene, really) is the most powerful out of the box and with plugins supports asian languages with text analysis. Postgres is most easily runnable on a single box with everything else from a site and with pgroonga can handle asian languages deftly, especially Postgres 10 with the ability to search jsonb. For what it’s worth, I’ve been using postgres because my search requirements will remain relatively small and I’m having trouble imagining that replication can’t solve whatever scaling issues I’d have.

                                                    What I’m wondering is has anyone used any of these in production besides ElasticSearch and come up with benchmarks for various queries? Relevancy is difficult to benchmark but comparisons can be drawn. I find this especially important for minimizing the costs associated with running a full featured service without being bankrolled.

                                                    1. 2

                                                      I’ve also had good results from picky for search in ruby apps; fast and good relevance when doing the obvious thing (vs bleve where the obvious thing gave me bad relevance slowly - I’m sure I was doing it wrong but perf on naive usage is one metric to evaluate.