1. 5

    I must be the only person ever who would want the reverse: C for C++ users. I honestly find C a lot weirder than C++.

    Due to a curious historical accident, in 1998 the AP Board (a US high school educational thing) decided to use C++ for a single year to teach AP Computer Science. Normally they go with Pascal, Java and I think right now it’s Python. But for one glorious year, they decided to go with C++. So that was my first “real” programming language, i.e. the first I learned in a classroom setting. This means I really did learn C++ without knowing C.

    This makes me a feel a bit different from nearly everyone else who considers C to be the foundation and is baffled by C++.

    1. 5

      Really to get to C from C++ you mostly just have to remove features. I’m curious what specific confusions you’ve run into that such a guide could cover?

      1. 1

        I don’t understand memcpy, malloc, free, strcpy, strncpy.These are not functions you use in C++. Why do I have to do struct foo x; if foo is a struct type instead of just doing foo x;?.The general syntax for structs is really weird in C. You have to typedef, you can’t just declare new structs? And I can’t declare looping variables, I have to instead declare it outside the loop. Ah, and no single-line coments, // is not a valid comment starter. What’s with all the pointers? We don’t use nearly that many pointers in C++. Arrays are confusing as shit. I always forget when do arrays become pointers and when do they not. Why is there no actual array type?

        I think newer versions of C have made some of these things more like C++, but you still run into a lot of C that has to be written in an older way. C is just very foreign if you use C++.

        1. [Comment removed by author]

          1. 1

            I think I already understand most of these things, but what I was trying to convey was that C is foreign, not so much looking for an explanation.

            It is entirely possible to know C++ and find C to be weird and foreign.

          2. 1

            I’ll try to summarize these.

            memcpy, malloc, free, strcpy – the best source for these is their manpages, which are generally well-written

            You do not use typedef to define structs in C (though some people confusingly choose to add a typedef as well). The name of the type is struct foo so that’s why you say struct foo x to declare variable.

            The reason C89 doesn’t allow variable declaration in for loop is for consistency (new things can only be added to the stack after an open brace) – there was no one “right way” semantic for the special case – however C99 dispensed with this so it’s the same as C++ now.

            Single line comments weren’t in C89 probably because no one thought having two comment syntaxes would be easier than just having one. C99 added the second one if you prefer it.

            All the C++ I’ve ever seen is still full of pointers – do you routinely pass full object instances on the stack? Anytime you say new in C++ you have created a pointer.

            Arrays in C are always “converted” to pointers when used in a value context. Declared arrays have their own semantics for sizeof, etc (since they are of “array type”). In arguments, int x[] is just sugar for int *x

            Hopefully those are helpful.

            1.  

              Arrays are confusing as shit. I always forget when do arrays become pointers and when do they not. Why is there no actual array type?

              http://www.torek.net/torek/c/pa.html

              1.  

                That still confuses me. I always need to refer to this document which I always forget.

                http://c-faq.com/aryptr/index.html

                Btw, I wasn’t looking for an explanation. I was trying to impress upon you how weird and foreign C is for a C++ person. I can look up these explanations, but C will always be weird and foreign to me because I was raised on C++.

          3. 2

            I went from Pascal straight to C++. My parents happened to buy me Stroustrup’s C++ book. I only had to learn to read C later as I started encountering C libraries.

          1. 7

            I always laugh when people come up with convoluted defenses for C and the effort that goes into that (even writing papers). Their attachment to this language has caused billions if not trillions worth of damages to society.

            All of the defenses that I’ve seen, including this one, boil down to nonsense. Like others, the author calls for “improved C implementations”. Well, we have those already, and they’re called Rust, Swift, and, for the things C is not needed for, yes, even JavaScript is better than C (if you’re not doing systems-programming).

            1. 31

              Their attachment to this language has caused billions if not trillions worth of damages to society.

              Their attachment to a language with known but manageable defects has created trillions if not more in value for society. Don’t be absurd.

              1. 4

                [citation needed] on the defects of memory unsafety being manageable. To a first approximation every large C/C++ codebase overfloweth with exploitable vulnerabilities, even after decades of attempting to resolve them (Windows, Linux, Firefox, Chrome, Edge, to take a few examples.)

                1. 2

                  Compared to the widely used large codebase in which language for which application that accepts and parses external data and yet has no exploitable vulnerabilities? BTW: http://cr.yp.to/qmail/guarantee.html

                  1. 6

                    Your counter example is a smaller, low-featured, mail server written by a math and coding genius. I could cite Dean Karnazes doing ultramarathons on how far people can run. That doesn’t change that almost all runners would drop before 50 miles, esp before 300. Likewise with C code, citing the best of the secure coders doesn’t change what most will do or have done. I took author’s statement “to first approximation every” to mean “almost all” but not “every one.” It’s still true.

                    Whereas, Ada and Rust code have done a lot better on memory-safety even when non-experts are using them. Might be something to that.

                    1. 2

                      I’m still asking for the non C widely used large scale system with significant parsing that has no errors.

                      1. 3

                        That’s cheating saying “non-c” and “widely used.” Most of the no-error parsing systems I’ve seen use a formal grammar with autogeneration. They usually extract to Ocaml. Some also generate C just to plug into the ecosystem since it’s a C/C++-based ecosystem. It’s incidental in those cases: could be any language since the real programming is in the grammar and generator. An example of that is the parser in Mongrel server which was doing a solid job when I was following it. I’m not sure if they found vulnerabilities in it later.

                    2. 5

                      At the bottom of the page you linked:

                      I’ve mostly given up on the standard C library. Many of its facilities, particularly stdio, seem designed to encourage bugs.

                      Not great support for your claim.

                      1. 2

                        There was an integer overflow reported in qmail in 2005. Bernstein does not consider this a vulnerability.

                    3. 3

                      That’s not what I meant by attachment. Their interest in C certainly created much value.

                    4. 9

                      Their attachment to this language has caused billions if not trillions worth of damages to society.

                      Inflammatory much? I’m highly skeptical that the damages have reached trillions, especially when you consider what wouldn’t have been built without C.

                      1. 12

                        Tony Hoare, null’s creator, regrets its invention and says that just inserting the one idea has cost billions. He mentions it in talks. It’s interesting to think that language creators even think of the mistakes they’ve made have caused billions in damages.

                        “I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years.

                        If the billion dollar mistake was the null pointer, the C gets function is a multi-billion dollar mistake that created the opportunity for malware and viruses to thrive.

                        1. 2

                          He’s deluded. You want a billion dollar mistake: try CSP/Occam plus Hoare Logic. Null is a necessary byproduct of implementing total functions that approximate partial ones. See, for example, McCarthy in 1958 defining a LISP search function with a null return on failure. http://www.softwarepreservation.org/projects/LISP/MIT/AIM-001.pdf

                          1. 3

                            “ try CSP/Occam plus Hoare Logic”

                            I think you meant formal verification, which is arguable. They could’ve wasted a hundred million easily on the useless stuff. Two out of three are bad examples, though.

                            Spin has had a ton of industrial success easily knocking out problems in protocols and hardware that are hard to find via other methods. With hardware, the defects could’ve caused recalls like the Pentium bug. Likewise, Hoare-style logic has been doing its job in Design-by-Contract which knocks time off debugging and maintenance phases. The most expensive. If anything, not using tech like this can add up to a billion dollar mistake over time.

                            Occam looks like it was a large waste of money, esp in the Transputer.

                            1. 1

                              No. I meant what I wrote. I like spin.

                          2. 1

                            Note what he does not claim is that the net result of C’s continued existence is negative. Something can have massive defects and still be an improvement over the alternatives.

                          3. 7

                            “especially when you consider what wouldn’t have been built without C.”

                            I just countered that. The language didn’t have to be built the way it was or persist that way. We could be building new stuff in a C-compatible language with many benefits of HLL’s like Smalltalk, LISP, Ada, or Rust with the legacy C getting gradually rewritten over time. If that started in the 90’s, we could have equivalent of a LISP machine for C code, OS, and browser by now.

                            1. 1

                              It didn’t have to, but it was, and it was then used to create tremendous value. Although I concur with the numerous shortcomings of C, and it’s past time to move on, I also prefer the concrete over the hypothetical.

                              The world is a messy place, and what actually happens is more interesting (and more realistic, obviously) than what people think could have happened. There are plenty of examples of this inside and outside of engineering.

                              1. 3

                                The major problem I see with this “concrete” winners-take-all mindset is that it encourages whig history which can’t distinguish the merely victorious from the inevitable. In order to learn from the past, we need to understand what alternatives were present before we can hope to discern what may have caused some to succeed and others to fail.

                                1. 2

                                  Imagine if someone created Car2 which crashed 10% of the time that Car did, but Car just happened to win. Sure, Car created tremendous value. Do you really think people you’re arguing with think that most systems software, which is written in C, is not extremely valuable?

                                  It would be valuable even if C was twice as bad. Because no one is arguing about absolute value, that’s a silly thing to impute. This is about opportunity cost.

                                  Now we can debate whether this opportunity cost is an issue. Whether C is really comparatively bad. But that’s a different discussion, one where it doesn’t matter that C created value absolutely.

                            2. 8

                              C is still much more widely used than those safer alternatives, I don’t see how laughing off a fact is better than researching its causes.

                              1. 10

                                Billions of lines of COBOL run mission-critical services of the top 500 companies in America. Better to research the causes of this than laughing it off. Are you ready to give up C for COBOL on mainframes or you think both of them’s popularity were caused by historical events/contexts with inertia taking over? Im in latter camp.

                                1. 7

                                  Are you ready to give up C for COBOL on mainframes or you think both of them’s popularity were caused by historical events/contexts with inertia taking over? Im in latter camp.

                                  Researching the causes of something doesn’t imply taking a stance on it, if anything, taking a stance on something should hopefully imply you’ve researched it. Even with your comment I still don’t see how laughing off a fact is better than researching its causes.

                                  You might be interested in laughing about all the cobol still in use, or in research that looks into the causes of that. I’m in the latter camp.

                                  1. 5

                                    I think you might be confused at what I’m laughing at. If someone wrote up a paper about how we should continue to use COBOL for reasons X, Y, Z, I would laugh at that too.

                                    1. 3

                                      Cobol has some interesting features(!) that make it very “safe”. Referring to the 85 standard:

                                      X. No runtime stack, no stack overflow vulnerabilities
                                      Y. No dynamic memory allocation, impossible to consume heap
                                      Z. All memory statically allocated (see Y); no buffer overflows
                                      
                                      1. 3

                                        We should use COBOL with contracts for transactions on the blockchains. The reasons are:

                                        X. It’s already got compilers big businesses are willing to bet their future on.

                                        Y. It supports decimal math instead of floating point. No real-world to fake, computer-math conversions needed.

                                        Z. It’s been used in transaction-processing systems that have run for decades with no major downtime or financial losses disclosed to investors.

                                        λ. It can be mathematically verified by some people who understand the letter on the left.

                                        You can laugh. You’d still be missing out on a potentially $25+ million opportunity for IBM. Your call.

                                        1. 1

                                          Your call.

                                          I believe you just made it your call, Nick. $25+ million opportunity, according to you. What are you waiting for?

                                          1. 4

                                            You’re right! I’ll pitch IBM’s senior executives on it the first chance I get. I’ll even put on a $600 suit so they know I have more business acumen than most coin pitchers. I’ll use phrases like vertical integration of the coin stack. Haha.

                                      2. 4

                                        That makes sense. I did do the C research. Ill be posting about that in a reply later tonight.

                                        1. 10

                                          Ill be posting about that in a reply later tonight.

                                          Good god man, get a blog already.

                                          Like, seriously, do we need to pass a hat around or something? :P

                                          1. 5

                                            Haha. Someone actually built me a prototype a while back. Makes me feel guilty that I dont have one instead of the usual lazy or overloaded.

                                              1. 2

                                                That’s cool. Setting one up isn’t the hard part. The hard part is doing a presentable design, organizing the complex activities I do, moving my write-ups into it adding metadata, and so on. I’m still not sure how much I should worry about the design. One’s site can be considered a marketing tool for people that might offer jobs and such. I’d go into more detail but you’d tell me “that might be a better fit for Barnacles.” :P

                                                1. 3

                                                  Skip the presentable design. Dan Luu’s blog does pretty well it’s not working hard to be easy on the eyes. The rest of that stuff you can add as you go - remember, perfect is the enemy of good.

                                                  1. 0

                                                    This.

                                                    Hell, Charles Bloom’s blog is basically an append-only textfile.

                                                  2. 1

                                                    ugh okay next Christmas I’ll add all the metadata, how does that sound

                                                    1. 1

                                                      Making me feel guilty again. Nah, I’ll build it myself likely on a VPS.

                                                      And damn time has been flying. Doesnt feel like several months have passed on my end.

                                            1. 1

                                              looking forward to read it:)

                                      3. 4

                                        Well, we have those already, and they’re called Rust, Swift, ….

                                        And D maybe too. D’s “better-c” is pretty interesting, in my mind.

                                        1. 3

                                          Last i checked, D’s “better-c” was a prototype.

                                        2. 5

                                          If you had actually made a serious effort at understanding the article, you might have come away with an understanding of what Rust, Swift, etc. are lacking to be a better C. By laughing at it, you learned nothing.

                                          1. 2

                                            the author calls for “improved C implementations”. Well, we have those already, and they’re called Rust, Swift

                                            Those (and Ada, and others) don’t translate to assembly well. And they’re harder to implement than, say, C90.

                                            1. 3

                                              Is there a reason why you believe that other languages don’t translate to assembly well?

                                              It’s true those other languages are harder to implement, but it seems to be a moot point to me when compilers for them already exist.

                                              1. 1

                                                Some users of C need an assembly-level understanding of what their code does. With most other languages that isn’t really achievable. It is also increasingly less possible with modern C compilers, and said users aren’t very happy about it (see various rants by Torvalds about braindamaged compilers etc.)

                                                1. 4

                                                  “Some users of C need an assembly-level understanding of what their code does.”

                                                  Which C doesnt give them due to compiler differences and effects of optimization. Aside from spotting errors, it’s why folks in safety- critical are required to check the assembly against the code. The C language is certainly closer to assembly behavior but doesnt by itself gives assembly-level understanding.

                                            2. 2

                                              So true. Every time I use the internet, the solid engineering of the Java/Jscript components just blows me away.

                                              1. 1

                                                Everyone prefers the smell of their own … software stack. I can only judge by what I can use now based on the merits I can measure. I don’t write new services in C, but the best operating systems are still written in it.

                                                1. 5

                                                  “but the best operating systems are still written in it.”

                                                  That’s an incidental part of history, though. People who are writing, say, a new x86 OS with a language balancing safety, maintenance, performance, and so on might not choose C. At least three chose Rust, one Ada, one SPARK, several Java, several C#, one LISP, one Haskell, one Go, and many C++. Plenty of choices being explored including languages C coders might say arent good for OS’s.

                                                  Additionally, many choosing C or C++ say it’s for existing tooling, tutorials, talent, or libraries. Those are also incidental to its history rather than advantages of its language design. Definitely worthwhile reasons to choose a language for a project but they shift the language argument itself implying they had better things in mind that werent usable yet for that project.

                                                  1. 4

                                                    I think you misinterpreted what I meant. I don’t think the best operating systems are written in C because of C. I am just stating that the best current operating system I can run a website from is written in C, I’ll switch as soon as it is practical and beneficial to switch.

                                                    1. 2

                                                      Oh OK. My bad. That’s a reasonable position.

                                                      1. 3

                                                        I worded it poorly, I won’t edit though for context.

                                              1. 2

                                                Open positions currently lists a Linux job, should we add a linux tag?

                                                1. 2

                                                  The title has Linux but the actual job description has OpenBSD listed. I guess that Linux-only won’t be added to this job board.

                                                  1. 4

                                                    True. Thank you for the comment.

                                                    I should clarify that for job posters on the site. BSD should be in description or/and in the title of a job post.

                                                1. 18

                                                  I love postgres (I’m a postgres DBA), and really dislike mysql (due to a long story involving a patch-level release causing server crashes and data loss).

                                                  That said, there is still a technical reason to choose mysql over postgres. Mysql’s replication story is still significantly better than postgres’. Multi-master, in particular, is something that’s relatively straightforward in mysql, but which requires third-party extensions and much more fiddling in postgres.

                                                  Now, postgres has been catching up on this front. Notably, the addition of logical replication over the last couple major versions really expands the options available. There’s a possibility that this feature will even be part of postgres 11, coming out this year (it’s on a roadmap). But until it does, it’s a significant feature missing from postgres that other RDBMSes have.

                                                  1. 7

                                                    There’s a possibility that this feature will even be part of postgres 11

                                                    PG 11 is in feature freeze since April. I don’t think there was anything significant for multi-master committed before that.

                                                    1. 3

                                                      Good point. I’d seen the feature freeze deadline, but wasn’t sure if it had actually happened, and what had made it in (I haven’t followed the -hackers mailing list for a while). I was mostly speculating based on the fact that they’d announced a multi-master beta for last fall.

                                                      I’m not surprised it’s taking a long time – it’s a hard problem – but it means that “clustering” is going to be a weak point for postgres for a while longer.

                                                    2. 3

                                                      Once you take all the other potential issues and difficulties with MySQL into account though, surely Postgres is a better choice on balance, even with more difficult replication setup?

                                                      1. 5

                                                        It really depends. If you need horizontally-scalable write performance, and it’s important enough to sacrifice other features, then a mysql cluster is still going to do that better than postgres. It’s possible that a nosql solution might fit better than mysql, but overall that’s a decision that I can’t make for you.

                                                        I’ll add that there are bits of postgres administration that aren’t intuitive. Specifically, bloat of on-disk table size (and associated slowdowns) under certain loads can really confuse people. If you can’t afford to have a DBA, or at least a dev who’s a DB expert, mysql can be very attractive. I’m not saying that’s a good reason to choose it, but I understand why some people do.

                                                        1. 1

                                                          What are your thoughts on MySQL vs MariaDB, especially the newer versions?

                                                          1. 3

                                                            Honestly, I haven’t looked closely at MariaDB lately. The last time I did was just to compare json datatypes – at the time, both mysql and mariadb were just storing json as parsed/verified text blobs without notable additional functionality.

                                                            I have to assume it’s better than mysql at things like stability, data safety, and other boring-but-necessary features. That’s mostly because mysql sets such a low bar, though, that it would take effort to make it worse.

                                                          2. 1

                                                            You clearly know more about databases than me, but I would question idea that MySQL is a good choice when you lack a DB expert. If anything, it is then when you shouldn’t use it. I still carry scars from issues caused by such lack of expertise at one of my previous employers.

                                                      1. 5

                                                        So, it’s working really well but the complex stacks in Linux are breaking a lot. Especially on hardware stuff. I’m curious if anyone in NetBSD or OpenBSD camps plans to try to get those working on it. That might be a better experience esp for console users.

                                                        1. 4

                                                          There is a FreeBSD effort to port to POWER9, I think one developer has also received a TALOS II.

                                                          Hopefully the market will expand and we will see lower prices. I am all for diversity, but damn $7000 for any machine is a lot of money.

                                                          1. 4

                                                            That’s a good thing. A high-quality, high-performance UNIX on POWER9. Especially for high-security since a contender for most secure UNIX is CheriBSD which is FreeBSD on capability-secure, MIPS-based CPU. One might use TALOS II to develop for it. More importantly, in long term such security enhancements like CHERI could get merged into an OpenPOWER core that becomes a future TALOS workstation. A significant drop in performance from checks might not hurt if the starting point is what a POWER9 or up can do. ;)

                                                          2. 3

                                                            I think the price tag might put a damper on it for both devs and users. Not much demand, so why sink the development time?

                                                            1. 3

                                                              Some people (outside the “NetBSD or OpenBSD camps”, too…) spend their time on such things because they are able to and they value diversity (POWER9 is the most interesting part in this case, at least for me).

                                                              1. 1

                                                                I think that this was true for Apple’s PPC hardware supports your point. It cost a bit extra with small market share. A lot less software was developed for it. However, the groups that stayed on it after Apple ditched PPC made a lot of software work on it. So, it’s possible a niche effort could make these usable.

                                                                Im not sure what the overlap is between those kind of devs and people that can drop almost ten grand on a machine.

                                                            1. 2

                                                              I see that it was fixed in 2016 to support int const *x, but static struct foo { short bar; } * footab[16]; (Debian bug #431027) still isn’t supported.

                                                              I’m not sure about “especially useful for people learning C” as it requires basic knowledge of the language (which is the biggest problem for people learning C as they often learn it wrong from teachers, silly tutorials and incorrect books), but it has its uses.

                                                              1. 7

                                                                I think DHH has a lot of accurate points here, but I think he’s wrong about not needing to write SQL or have an understanding of the database technology supporting your application. With applications that store few data in their database, then I agree it may be possible to have a completely opaque perception of the database technology. However, I don’t see a way to forgo knowledge of the database layer and avoid writing SQL for things like running migrations on tables with millions of rows.

                                                                As a simple example, creating an index on a large Postgres table without the CONCURRENTLY keyword is a surefire way to block reads and writes on a table while the index is created and cause downtime. I don’t work with ActiveRecord, but it appears there is an abstraction for this keyword (algorithm: :concurrently). But how would you know to use this option if you don’t have an understanding of the database and its transactional isolation behavior?

                                                                As another example, adding a NOT NULL constraint on a large table will also block reads and writes in Postgres, while it validates that all current rows conform to the constraint. You’re better off creating an INVALID check constraint to ensure a column is non-null, and then VALIDATE’ing that constraint later to avoid downtime. These are the type of things where knowledge of just an abstraction layer and not of the underlying database will cause problems.

                                                                To be fair, DHH does only mention that Basecamp 3 has no raw SQL in “application logic”, and he never mentions migrations in the post, so maybe he is ignoring migration-type SQL commands in this context.

                                                                1. 3

                                                                  As another example, adding a NOT NULL constraint on a large table will also block reads and writes in Postgres, while it validates that all current rows conform to the constraint. You’re better off creating an INVALID check constraint to ensure a column is non-null, and then VALIDATE’ing that constraint later to avoid downtime.

                                                                  And this (among other things) is why I just can’t believe the claim that they could move from MySQL to Postgres and “not give a damn”.

                                                                  1. 1

                                                                    I interpreted that as he wouldn’t care what underlying technology he used not that the migration process would be trivial.

                                                                    1. 1

                                                                      But I’m not talking about the migration process either. He will care about the underlying technology when, for example, his team will have to tackle vacuum issues – long after the move has been done.

                                                                      1. 1

                                                                        But if your claim is that you do have to care about the technology then your problem is with the entire blog post, not just if he would give a damn about running postgresql.

                                                                1. 5

                                                                  Those are some pretty flaky arguments regarding OpenBSD. What is “theoretical” SMP? I’m running this from a 4-core OpenBSD laptop. You know, non-theoretically. Same language snark goes with vmm: they tried to implement a hypervisor? I’ll be sure to inform mlarkin of his failure to execute. It may not be what the author wants, but that’s a different story. Anyway, if there are good comparisons between the two systems security-wise, they look like they’re in that chart from https://hardenedbsd.org/content/easy-feature-comparison. Is it up to date with the recent anti-ROP efforts?

                                                                  1. 2

                                                                    It is. OpenBSD has an SROP mitigation, whereas HardenedBSD doesn’t. HardenedBSD has non-Cross-DSO CFI (Cross-DSO CFI is actively being worked on), whereas OpenBSD doesn’t. HardenedBSD also applies SafeStack to applications in base. CFI provides forward-edge safety while SafeStack provides backward-edge safety (at least, according to llvm’s own documentation.)

                                                                    HardenedBSD inherits MAP_STACK from FreeBSD. The one thing about OpenBSD’s MAP_STACK implementation that HardenedBSD may lack (I need to verify) is that the stack registers (rsp/rbp) is checked during syscall enter to ensure it points to a valid MAP_STACK region. If FreeBSD’s syscall implementation doesn’t do this already, doing so would be a good addition in HardenedBSD.

                                                                    So, there’s room for improvement by both BSDs, as should be expected. It looks like OpenBSD is starting the migration towards an llvm toolchain, which would allow OpenBSD to catch up to HardenedBSD with regards to CFI and SafeStack.

                                                                    Sorry for the excessive use of commas. I enjoy them perhaps a bit too much. ;)

                                                                    1. 1

                                                                      I haven’t read the whole article, because I’m not interested in HardenedBSD.

                                                                      What is “theoretical” SMP? I’m running this from a 4-core OpenBSD laptop. You know, non-theoretically.

                                                                      The article is indeed vague about it, but I think the author meant scalability issues. Too much time spent in the kernel space.

                                                                      Same language snark goes with vmm: they tried to implement a hypervisor? I’ll be sure to inform mlarkin of his failure to execute.

                                                                      I don’t have any experience with virtualization, but the point seems to be that you can only have OpenBSD and Linux guests under an OpenBSD host which compares less than something like bhyve.

                                                                      1. 1

                                                                        SMP

                                                                        From what I have read about SMP on OpenBSD its not that it would not detect 4 or 64 cores, its that its subsystems (like FreeBSD 5.0 for example) were not entirely rewritten to fully itilize all cores, that in many places still so called GIANT LOCK is used, may have changed recently, sorry if information is not up to latest date.

                                                                        vmm

                                                                        Now ints very limited, can You run Windows VM on it? … or Solaris VM? Last I read about it only OpenBSD and Linux VMs worked.

                                                                        Is it up to date with the recent anti-ROP efforts?

                                                                        I am not sure, You may ask here - https://www.twitter.com/HardenedBSD - or on the HardenedBSD forums - https://groups.google.com/a/hardenedbsd.org/forum/#!forum/users

                                                                        1. 3

                                                                          or Solaris VM? Last I read about it only OpenBSD and Linux VMs worked.

                                                                          It runs Illumos derivatives (eg. OpenIndiana). There’s a speicific feature missing that FreeBSD/NetBSD need which is being worked on. It doesn’t run Windows because Windows needs graphics.

                                                                          1. 2

                                                                            Thanks for clarification, I hope that graphics support/emulation will also came to vmm soon.

                                                                            I added that information to the post.

                                                                        2. 1

                                                                          I’m not sure, the article seems like it makes an honest enough comparison between hardenedBSD and OpenBSD that I make OpenBSD a priority to consider the next time I need truly secure OS.

                                                                          1. 3

                                                                            The “One may ask…” paragraph is so slanted toward HardenedBSD over OpenBSD that I’d have immediately assumed a HardenedBSD developer or fan was writing it.

                                                                            1. 1

                                                                              Tried my best, I thought that it was clean enough from the article that OpenBSD is secure for sure while HardenedBSD aspires to that target with FreeBSD codebase as start …

                                                                            2. 1

                                                                              Tried my best, I thought that it was clean enough from the article that OpenBSD is secure for sure while HardenedBSD aspires to that target with FreeBSD codebase as start …

                                                                          1. 7

                                                                            When was the last time you productively used an octal number-representation?

                                                                            Whenever I terminate a C string with a null character.

                                                                            1. 1

                                                                              I’m curious, why do you need octal for that? 0, 0x0, 0b0, and 00 all act the exact same, don’t they?

                                                                              1. 2

                                                                                It’s a joke, because 0 is an octal literal.

                                                                            1. 2

                                                                              It’s a shame it’s open-core.

                                                                              1. 7

                                                                                Spanner/F1 and FoundationDB were closed. CochroachDB was first (AFAIK) of those competing with Spanner to give us anything at all. Let’s give them credit, eh? ;)

                                                                                1. 3

                                                                                  FoundationDB was open (in some form) before it disappeared into Apple.

                                                                                  1. 4

                                                                                    I don’t believe any of the interesting tech was open source. It was sort of the opposite of open-core, with a proprietary core but some ancillary stuff like an SQL parser that was open source. That other stuff is what disappeared when Apple bought them (GitHub deleted, packages pulled from repos, etc.), which caused a bit of a stir as they disappeared with no warning and some people had been depending on the packages.

                                                                                    1. 2

                                                                                      I never heard that. Ill look into it further. Thanks.

                                                                                      1. 1

                                                                                        Looking into it, the core DB that was what was really valuable was closed with some peripheral stuff open. This write-up goes further to say it was kind of fake FOSS that lured people in. I don’t have any more data to go on since Apple pulled all the Github repos.

                                                                                      2. 1

                                                                                        It doesn’t seem to me that CockroachDB competes with Spanner. I’d thought of MongoDB before CockroachDB.

                                                                                        1. 8

                                                                                          It’s explicitly the origin for CockroachDB. Spanner less the cesium clocks.

                                                                                    1. 2

                                                                                      Great post; this is one of my pet peeves.

                                                                                      One minor point: ‘This interactivity is usually missing in “compiled” languages’ <- I would say it’s true that this is missing from several mainstream compiled languages, (to their great detriment) but the majority of compiled languages do support it.

                                                                                      Edit: I guess the scare quotes around “compiled” might indicate it’s not meant to apply to most compiled languages, just the ones that are typically thought of as compiled?

                                                                                      1. -1

                                                                                        Great post; this is one of my pet peeves.

                                                                                        Mine as well. Even the local safe languages guru tends to conflate programming languages with their implementations.

                                                                                        1. 13

                                                                                          That’s unlikely. What’s more likely is that humans don’t need to be 100% unambiguously precise in every form of communication, and instead can usually rely on other humans to know what they mean.

                                                                                          1. 4

                                                                                            In addition to what burntsushi said, while it’s true that languages and their implementations are separable, it’s often not an accident that languages have their particular implementations. Bytecode interpreters and/or JIT compilers fit with Java, Python, etc in a way they just don’t for C. You can write them for C, but it’s typically less valuable.

                                                                                        1. 2

                                                                                          Good article. I’ve been tentatively looking for a slim, performant and asynchronous web-service framework, considering various languages and this article repels me from using Rust due to its syntax.

                                                                                          One of the first scaling problems you’re likely to run into with Postgres is its modest limits around the maximum number of allowed simultaneous connections. Even the biggest instances on Heroku or GCP (Google Cloud Platform) max out at 500 connections, and the smaller instances have limits that are much lower (my small GCP database limits me to 25). Big applications with coarse connection management schemes (e.g., Rails, but also many others) tend to resort to solutions like PgBouncer to sidestep the problem.

                                                                                          pgBouncer is not really “a solution you resort to in order to sidestep the problem”, it’s something that should be built into core Postgres but isn’t.

                                                                                          At the end of the day, your database is going to be a bottleneck for parallelism, and the synchronous actor model supports about as much parallelism as we can expect to get from it, while also supporting maximum throughput for any actions that don’t need database access.

                                                                                          I don’t know why. Especially since all the Postgres parallelization work that’s been put into PG 9.6 and 10.

                                                                                          1. 18

                                                                                            I believe that the “single entry, single exit” coding style has not been helpful for a very long time, because other factors in how we design programs have changed.

                                                                                            Unless it happens that you still draw flowcharts, anyway. The first exhortation for “single entry, single exit” I can find is Dijkstra’s notes on structured programming, where he makes it clear that the reason for requiring that is so that the flowchart description of a subroutine has a single entry point and a single exit point, so its effect on the whole program state can be understood by understanding that subroutine’s single collection of preconditions and postconditions. Page 19 of the PDF linked above:

                                                                                            These flowcharts share the property that they have a single entry at the top and a single exit at the bottom: as indicated by the dotted block they can again be interpreted (by disregarding what is inside the dotted lines) as a single action in a sequential computation. To be a little bit more precise” we are dealing with a great number of possible computations, primarily decomposed into the same time-succession of subactions and it is only on closer inspection–i.e, by looking inside the dotted block–that it is revealed that over the collection of possible computations such a subaction may take one of an enumerated set of distinguished forms.

                                                                                            These days we do not try to understand the behaviour of a whole program by composing the behaviour of each expression. We break our program down into independent modules, objects and functions so that we only need to understand the “inside” of the one we’re working on and the “outside” of the rest of them (the “dotted lines” from Dijkstra’s discussion), and we have type systems, tests and contracts to support understanding and automated verification of the preconditions and postconditions of the parts of our code.

                                                                                            In other words, we’re getting the benefits Dijkstra wanted through other routes, and not getting them from having a single entry point and a single exit point.

                                                                                            1. 7

                                                                                              An interesting variant history is described in https://softwareengineering.stackexchange.com/a/118793/4025

                                                                                              The description there is that instead of “single exit” talking about where the function exits from, it actually talks about a function only having a single point it returns to, namely the place where it was called from. This makes a lot more sense, and is clearly a good practice. I’ve heard this description from other places to, but unfortunately I don’t have any better references.

                                                                                              1. 3

                                                                                                Yes, very interesting. It makes sense that “single exit” means “don’t jump to surprising places when you’re done” rather than “don’t leave the subroutine from arbitrary places in its flow”. From the Structured Programming perspective, both support the main goal: you can treat the subroutine as a box that behaves in a single, well-defined way, and that programs behave as a sequence of boxes that behave in single, well-defined ways.

                                                                                              2. 3

                                                                                                The first exhortation for “single entry, single exit” I can find is Dijkstra’s notes on structured programming

                                                                                                Also Tom Duff’s “Reading Code From Top to Bottom” says this:

                                                                                                During the Structured Programming Era, programmers were often taught a style very much like the old version. The language they were trained in was normally Pascal, which only allowed a single return from a procedure, more or less mandating that the return value appear in a variable. Furthermore, teachers, influenced by Bohm and Jacopini (Flow Diagrams, Turing Machines and Languages with only two formation rules, Comm. ACM 9#5, 1966, pp 366-371), often advocated reifying control structure into Boolean variables as a way of assuring that programs had reducible flowgraphs.

                                                                                                1. 1

                                                                                                  These days we do not try to understand the behaviour of a whole program by composing the behaviour of each expression. We break our program down into independent modules, objects and functions so that we only need to understand the “inside” of the one we’re working on and the “outside” of the rest of them (the “dotted lines” from Dijkstra’s discussion), and we have type systems, tests and contracts to support understanding and automated verification of the preconditions and postconditions of the parts of our code.

                                                                                                  Maybe I’m missing something, but it seems to me that Dijkstra’s methodology supports analyzing one program component at a time, treating all others as black boxes, so long as:

                                                                                                  • Program components are hierarchically organized, with lower-level components never calling into higher-level ones.
                                                                                                  • Relevant properties of lower-level components (not necessarily the whole analysis) are available when analyzing higher-level components.

                                                                                                  In particular, although Dijkstra’s predicate transformer semantics only supports sequencing, selection and repetition, it can be used to analyze programs with non-recursive subroutines. However, it cannot handle first-class subroutines, because such subroutines are always potentially recursive.

                                                                                                  In other words, we’re getting the benefits Dijkstra wanted through other routes, and not getting them from having a single entry point and a single exit point.

                                                                                                  Dijkstra’s ultimate goal was to prove things about programs, which few of us do. So it is not clear to me that we are “getting the benefits he wanted”. That being said, having a single entry point and a single exit point merely happens to be a requirement for using the mathematical tools he preferred. In particular, there is nothing intrinsically wrong about loops with two or more exit points, but they are awkward to express using ALGOL-style while loops.

                                                                                                1. 3

                                                                                                  That does seem rather clever, though it raises design questions in my mind: Why not define user defined order in some other way, such as a pg_array in a different table?

                                                                                                  1. 3

                                                                                                    If by pg_array you mean standard Postgres array type, then my answer would be: mainly because current Postgres lacks support for arrays referring to a foreign key. Also possibly because of increased lock granularity in case of an array.

                                                                                                  1. 7

                                                                                                    All of these advantages are really due to C’s popularity though, aren’t they?

                                                                                                    C was the best language around a long time ago, and its popularity due to that led to it being standard to make C compilers for hardware, and the tremendous amount of work that went into C’s compilers due to years and years of improvement, along with the language’s low level, made it quite fast.

                                                                                                    I’m not trying to argue that C wasn’t a good choice; clearly it was due to the reasons listed in the article, but I don’t think there’s a lot that is inherently “the best” in the language C itself; perhaps if the same colossal amount of work were put into other languages, they’d be thriving as much as (or more than) C. Its popularity makes itself a self-fulfilling prophecy.

                                                                                                    1. 9

                                                                                                      C was the best language

                                                                                                      It wasn’t. The early works show it was what they could get to compile on a PDP-11 based on a chopped down ALGOL that was what Richards et al could compile on an EDSAC. The only goals were (a) it compiles on terrible hardware and (b) it runs efficiently on terrible hardware. Terribly by today’s standards that is. The ALGOL-derived languages, including smaller ones like Wirth’s, tried to balance requirements of safety, maintainability, compile speed, and runtime speed. Most projects doing system programming today benefit from each of those as well. Hence, a design ignoring one or more is objectively worse in most projects.

                                                                                                      So, both BCPL and C were great languages for hacking on crap hardware when little to nothing else was available. From there, like you said, it’s popularity from being used in a killer app (UNIX) did much of the rest. The momentum behind it created more compilers, compiler optimizations, developer tools, 3rd party libraries, and people that know how to program it. That ecosystem makes it a good choice in a lot of systems programming projects. That became self-reinforcing, too.

                                                                                                      The language itself is either poorly or barely designed for the current use-cases despite doing original one really well. Anything that does current use-cases better is possibly a better choice. That’s exponentially more true if it’s C-compatible in data types, calling conventions, and/or ability to compile to C. I recommend last points for all C competitors given the ecosystem being an army of freight trains that aren’t stopping or turning around any time soon. There’s also still a niche of people using other system languages like Free Pascal, Ada/SPARK, and now Rust. Thanks to Rust work, that niche is getting revitalized. I stand by my recommendation of being C compatible, though, since both backward compatibility and incremental change are long-proven to be best ways to transition whether we like it or not. A transition that may take decades for legacy systems since the build outs took decades.

                                                                                                      1. 5

                                                                                                        [C] was what they could get to compile on a PDP-11 based on a chopped down ALGOL that was what Richards et al could compile on an EDSAC. The only goals were (a) it compiles on terrible hardware and (b) it runs efficiently on terrible hardware.

                                                                                                        That’s not the impression I get from dmr’s The Development of the C Language. It’s not even an impression I’d get from the presentation you’re referring to.

                                                                                                        The impression I get from both of those sources is that C is first and foremost a programming language that is relatively easy to implement and then relatively straightforward to use.

                                                                                                        That’s exponentially more true if it’s C-compatible in data types, calling conventions, and/or ability to compile to C. I recommend last points for all C competitors given the ecosystem being an army of freight trains that aren’t stopping or turning around any time soon.

                                                                                                        I think that the ability to compile from one language to C is many cases is either impractical or pointless. It only is potentially useful when an operating system comes with a C compiler but not Rust and you’re a user that needs to run a Rust program but cannot install a Rust implementation - but then it can be difficult to debug the program unless you’re willing to invest your time into analyzing and improving the generated C source code.

                                                                                                        1. 3

                                                                                                          It was relatively easy to implement at that time period if they didn’t know about LISP notation. It’s straightforward to use if, like in their use cases, safety and maintainability of large programs don’t matter much. If they do, it’s more straightforward to write in a language that knocks defects out by design. Wirth’s Modula-2 addressed both needs in a language that was even easier to implement. He iterated on from there baking in things like concurrency and OOP without C++’s mess.

                                                                                                          “I think that the ability to compile from one language to C is many cases is either impractical or pointless.”

                                                                                                          Here’s a few examples of where it could be beneficial. In each, we’d do what I stated above to seemlessly use available C libraries or to compile it with C’s many optimizing compilers.

                                                                                                          1. Coded in Modula-2 or other simple, safe-by-default programming language (i.e. Oberons) that extracts to C. Languages are straight-forward to use, will have less defects, and handle more stuff in programming in the large.

                                                                                                          2. The LISP machines and Smalltalk environment allowed coding a system incrementally against a running, live image. Every step can be immediately checked. Instead of crashes, mistakes take you where the problem is. The system can be changed while it’s running: redefining a class will redefine all instances in the system. In LISP’s, macros let one make it easy to express constructs that take more code in C. Studies on both Common LISP and Smalltalk vs C had the former running circles around the latter in productivity. Lower defects, too. A C embedded in CL environment would be better than C itself. PreScheme was a system, programming language that kept the compile-time-only features of LISP’s to get some language-level benefits. Compiled to C.

                                                                                                          3. Code in ML or a dialect to get its benefits. ML itself is strongly-typed with type inference that lends well to functional programming. Since language is clean, CompSci folks regularly extend it with dialects that do things like concurrency models or covert-channel analysis. Major provers can all extract verified code directly to ML. CakeML will compile that to assembly for verified, reference implementation. If the reference implementation is too slow, then a ML-to-C compiler like Tolmach et al built can possibly speed it up a lot. Exhaustive, equivalence checking on a per function or module basis between reference and C versions shows latter is at least as correct.

                                                                                                          4. Similar to 1, code in Ada 2012 + SPARK 2014. Ada is a complex derivative of ALGOL that’s systematically designed for about every feature to be safe by default. You can turn safety off in selective ways when necessary. SPARK lets you prove the absence of common errors such as divide by zero or integer overflows in sample code if you put some work into it. Mostly automated. Unlike C with Frama-C, SPARK language was specifically designed to make verification easy. It can compile to C for portability reasons. Throw Rust in 4 since it would be used for similar reasons to Ada with better model for temporal safety and concurrency.

                                                                                                          5. Haskell is a pure, functional language that gets those benefits. It also has many features in its type system for knocking out whole classes of problems while maintaining relatively-concise programs. It’s like a middle point between a safe language and something that would take lots of mathematical skills. It can do DSL’s. Both C-like and hardware languages have been embedded in Haskell for its benefits. They extract to the other language once program is finished.

                                                                                                          6. Any DSL-oriented language given they boost productivity and/or safety at certain things. They might reduce to C form for execution. They will also have things they can’t handle which necessitate a FFI.

                                                                                                          7. For distributed algorithms, tools such as Event-B or TLA+ can precisely specify them following checks or proofs of key properties. Users of such tools often prefer a code generator that reduces odds they’ll make a mistake on something tedious doing it by hand. The primitives themselves will usually be coded-by-hand, though. They can use something like 1-6, though, to reduce defects. Each tool does what it’s good at.

                                                                                                          So, we have a series of languages that are a mix of easier to compile, easier to transform, easier to be safer, easier to maintain, and so on. Everything about C has been beaten by better designs except for piles of work put into its compilers. Even they with the kinds of bugs we see show they’d have benefited from a gradual port to a better systems language at some point. The main reasons people keep with C for language rather than social reasons are to use the tools in its ecosystem. If includes or deploys have no penalties, then one can use a language better across the board in that ecosystem. There’s no reason to continue using C past familiarity if alternatives have all its benefits with few to none of its detriments. Even familiarity didn’t make it more productive in the fairest study comparing it to Ada.

                                                                                                      2. 4

                                                                                                        Yes, in other words, programming languages have strong network effects. To interact with software written in C, the path of least resistance is to also write your code in C.

                                                                                                        So if your goal is to reuse code, then every piece of software written in C makes the C ecosystem more valuable. And since sqlite is a library, it wants other people to reuse it, even if it doesn’t reuse anything besides libc itself.

                                                                                                        This effect is especially strong in the case of kernels and interpreters/VMs. C has pretty much a monopoly on kernels: Linux, BSD, and Windows. It also has a near monopoly on interpreters and VMs: Python, Ruby, Perl, PHP, R, bash, Lua, Tcl, the JVM, etc. (An exception would be v8 which has a somewhat weird C++ interface.)

                                                                                                      1. 3

                                                                                                        Two things:

                                                                                                        • consider using more const char * for functions that do not mutate their arguments
                                                                                                        • consider passing the extra field of the pgNotify object as an argument to the shell command (so you can send message) more details here

                                                                                                        Note that you probably want to write the payload to a temporary file and spool it to the shell command instead of inserting it directly, because of security concerns.

                                                                                                        EDIT:

                                                                                                        For the payload thing, maybe also consider fork and execlp for passing args.

                                                                                                        1. 3

                                                                                                          consider using more const char * for functions that do not mutate their arguments

                                                                                                          I don’t want to pick nits, but since people may learn from this comment, I’d argue for going full char const * const, like this:

                                                                                                          -void           listen_forever(PGconn *, const char *, const char *);
                                                                                                          +void           listen_forever(PGconn *, char const * const, char const * const);
                                                                                                          

                                                                                                          I also can’t find a good reason for BUFSZ to be a file scope int. Usually it’d be defined with a #define. Used like that it has a side effect of the char sql[7 + BUFSZ + 1]; becoming a VLA unless gcc decides otherwise.

                                                                                                          1. 2

                                                                                                            Neither of those are necessary. The * const is trivially inferable and no optimizing C compiler worth anything will generate a variable length array from a const int. BUFSZ may be completely eliminated by the link time optimizer as well, since this module will have an initial executable storage class.

                                                                                                            cc: @friendlysock

                                                                                                            1. 1

                                                                                                              The * const is trivially inferable

                                                                                                              In a simple file like this it probably is trivial, but in larger projects I find it easier to follow the code when both pointers and objects pointed to are const-qualified (assuming neither should be modified by the program).

                                                                                                              1. 1

                                                                                                                I agree it’s useful as documentation to the developer, and as a safeguard against developer mistakes. But it doesn’t help the optimizer. And the last const qualifier on a pointer is local to the function, so it’s only useful for larger / more convoluted functions. I think any time the developer can easily see whether the pointer could be labeled const, it’s unnecessary. If it’s not easy to see, and the function can’t be quickly refactored to make it easier to see, then yeah throw a const on it.

                                                                                                            2. 1

                                                                                                              Good point! I’m a little lazy on const stuff sometimes. :(

                                                                                                            3. 1

                                                                                                              Thanks for the tips. I changed the functions to using const char * for their arguments. For the second point, this program doesn’t actually pass the payload to the command it executes. That’s all the functionality I needed for my own use case but I can see how sending the payload to the command via stdin or something could be useful in certain cases.

                                                                                                            1. 5

                                                                                                              There are various ORMs/mappers, but most advice you’ll find (and what I’ll say, also) is to not use them. Wrap a database connection in a struct that has methods to do what you want. Something I’ve found conditionally useful are generators to write the function to populate a struct from a database row.

                                                                                                              The community will also say to not use web frameworks, and again I’d agree. The stdlib http package provides a stable foundation for what you want to do. You’ll have more luck looking for packages that do what specific thing you want, rather than thinking in terms of frameworks.

                                                                                                              All that said, some coworkers like echo but I can’t for the life of me understand why. Any web-oriented package shouldn’t need to give a shit if it’s hooked up to a tty or not.

                                                                                                              1. 3

                                                                                                                The problem is that when you do search and filtering on various conditions (like in a shop) you don’t want to resort to sql string stitching, I wasn’t able to find anything nice when I looked at the docs, for example in gorm:

                                                                                                                db.Where("name = ? AND age >= ?", "jinzhu", "22") - I expect that when you have 20 conditions with different operators and data types you will end up having a bad time.

                                                                                                                1. 4

                                                                                                                  I’m at a small scale but I just write the query entirely and or do string concatenation. I’m having a fine time though. I just use sqlx.

                                                                                                                  1. 2

                                                                                                                    sqlx

                                                                                                                    https://godoc.org/github.com/jmoiron/sqlx#In is pretty useful when you need it.

                                                                                                                    1. 3

                                                                                                                      I admit I don’t understand how to use that from the doc string. Could you show a simple example or elaborate?

                                                                                                                      1. 4

                                                                                                                        It’s useful for queries using the IN keyword, like this:

                                                                                                                        query, params, errIn := sqlx.In("SELECT column1 FROM table WHERE something IN (?);", some_slice)
                                                                                                                        // if errIn != nil...
                                                                                                                        rows, errQuery := db.Query(query, params...)
                                                                                                                        
                                                                                                                    2. 2

                                                                                                                      We built something in house that is very similar in spirit to sqlx but adds a bunch of helpers.

                                                                                                                      https://github.com/blendlabs/spiffy

                                                                                                                    3. 4

                                                                                                                      https://github.com/Masterminds/squirrel (which kallax uses) seems somewhat akin to the SQLAlchemy expression API. (And yeah, to me, that’s a great part of SQLAlchemy; I’ve hardly used its ORM in comparison.)

                                                                                                                      I went from Python + heavy use of the SQLAlchemy expression API to Go and got by OK with just stdlib, but part of that was that the work in Go had far less complicated queries most of the time. So, not the best comparison maybe.

                                                                                                                      1. 2

                                                                                                                        I support the advice to not use mappers like ORMs, but I also agree with what you said. The middle ground seems to be query builders.

                                                                                                                        If you use Postgres as your DBMS by any chance, I advise you to make sure that the query abstraction layer of your choice doesn’t do query parameterization by string interleaving but utilizes PQexecParams underneath instead.

                                                                                                                        1. 1

                                                                                                                          I haven’t used it but I think Dropbox built a library for that. https://godoc.org/github.com/dropbox/godropbox/database/sqlbuilder

                                                                                                                      1. 4

                                                                                                                        Looks like you have a misunderstanding of EXPLAIN ANALYSE’s output. The first step in the query plan for DB view is the bitmap index scan, then bitmap heapscan, then the sort; not the other way around.

                                                                                                                        The order is “inside out”.

                                                                                                                        1. 5

                                                                                                                          To add to that, explain.depesz.com is really helpful.

                                                                                                                          1. 1

                                                                                                                            Thanks, in fact I wondered about that because it seemed weird. 🤦‍♂

                                                                                                                            Too far inbetween EXPLAIN ANALYZE’s for me… perhaps luckily? :D

                                                                                                                          1. 4

                                                                                                                            Whenever I arrive at a point where the answer to “why did it take so long” is “the underlying query took this much time”, the next step for me is see what query was generated exactly and what the plan is (EXPLAIN), with actual timings of the execution if possible (EXPLAIN ANALYZE).

                                                                                                                            I wonder if the newly created index will help when you want to fetch multiple courier_ids each with their most recent location. With a query like

                                                                                                                            SELECT cl.*
                                                                                                                            FROM couriers c
                                                                                                                            CROSS JOIN LATERAL (
                                                                                                                              SELECT id, courier_id, location, time, accuracy, inserted_at, updated_at
                                                                                                                              FROM courier_locatins cl
                                                                                                                              WHERE cl.courier_id = c.id
                                                                                                                              ORDER BY time DESC 
                                                                                                                              LIMIT 1
                                                                                                                            ) AS cl
                                                                                                                            WHERE c.id IN (...)
                                                                                                                            

                                                                                                                            it might be even possible to use the old two indexes (perhaps with the condition that the index on column “time” be created in reversed order; CREATE INDEX courier_location_time_btree ON courier_locations USING btree (time DESC). The multicolumn index would likely benefit from descending order as well. Thinking about it further, a BRIN index might be better still).

                                                                                                                            There is a lot of guessing in this comment because I don’t have the data and I lack the intuition to know better how the query planner would work. There are people in #postgresql on Freenode who could tell just from looking at your case and after getting a few answers from you.

                                                                                                                            1. 1

                                                                                                                              Hey, thanks for your comment! Haven’t investigated this case yet, as we mostly display single couriers or if we don’t we make multiple requests (at the moment either way). Doing the index on descending is pretty nice, I feel like I should try that out.

                                                                                                                              The different index types as well - true I didn’t investigate them here at all. I usually only do when my current solution won’t help anymore 😅 I should read up about them again!

                                                                                                                            1. 6

                                                                                                                              This video is what I use when countering the myths about how C was designed. It goes to the papers and constraints that led Richards’ team to chop ALOL into BCPL. Then, the modifications from BCPL to B to C. Understanding the context of why C was the way it was might help folks understand why we should upgrade given we’re no longer in those constraints either in hardware or programming features.

                                                                                                                              1. 10

                                                                                                                                I think there is a reasonable argument that C won on its merits. The following is a list of some languages that were available in 1975 and my opinion of why they lost out to C. C is pretty much the only language on this list with a portable implementation that ran on minicomputers.

                                                                                                                                Algol 60 - call by name is expensive, not really intended for system software

                                                                                                                                Algol 68 - complex to implement, standard uses an obscure formal semantics, requires a runtime, compilers did not emerge for years

                                                                                                                                Algol W - first implementation was for IBM mainframes in an infix assembly language, few other implementations

                                                                                                                                BCPL - untyped, inferior to C in some ways, limited support for byte addressing

                                                                                                                                BLISS - semantics for assignment are unusual, untyped, no portable compiler, only for DEC architectures

                                                                                                                                Coral66 - British military standard, may not have had recursion

                                                                                                                                Fortran 66 - not really suited to system software, although a number of vendors wrote operating systems in an extended Fortran

                                                                                                                                Forth - different programming model, mostly interpreted

                                                                                                                                IMP72 - implemented mostly on supercomputers, low level of abstraction (Fortran II), complex (extensible) grammar

                                                                                                                                Jovial73 - DoD standard, no standard IO

                                                                                                                                LRLtran - no implementations for minicomputers

                                                                                                                                MAD - low level of abstraction, implementations ran on mainframes

                                                                                                                                NELIAC - low level of abstraction

                                                                                                                                Pascal - weak standard, no separate compilation, Wirth moved on to new languages

                                                                                                                                PL.8 - internal to IBM, compiler ran on mainframes

                                                                                                                                PL/I - complicated to implement, early implementations were slow

                                                                                                                                PL/S - internal to IBM, compiler ran on mainframes

                                                                                                                                RTL/2 - British language for realtime systems, probably unknown in the US.

                                                                                                                                Simula 67 - uses garbage collection, inventors wanted license fees

                                                                                                                                1. 2

                                                                                                                                  Great list. Remember that there’s two parts to this: one is how they designed it; one is what happened later. Your list seems to be what happened later after comparing its design to everything else unmodified. Whereas, mine says they’d have cherry-picked the best of anything on that list modifying it for their situation. In each case, they’d pick whatever was safest or cleanest by default switching to unsafe only where necessary. As hardware improved, the safety and maintainability would improve.

                                                                                                                                  That’s the approach Wirth took with Modula-2 and the rest of his languages. Most others did as well doing languages for safety or programming in the large. It’s now the standard way to do things with many citing an escape from the safety and maintainability problems of C. So, the evidence leans toward Wirth’s choice.

                                                                                                                                2. 1

                                                                                                                                  If I wanted to both re-write indent(1) in not-C and continue to distribute it as a part of FreeBSD, NetBSD, OpenBSD - which programming language should I “upgrade” to? What choice do I have?

                                                                                                                                  1. 2

                                                                                                                                    My top choices for contenders would be rust, zig, myrddin and nim. zig being the closest to C with many fixes.

                                                                                                                                    1. 4

                                                                                                                                      One issue with rust currently is that building the compiler will dominate compile times until most of the distribution is ported to rust.

                                                                                                                                      1. 3

                                                                                                                                        What about Wirth’s new Oberon-07?

                                                                                                                                        Recently it has got a new promising little compiler to C, OBNC.

                                                                                                                                        1. 2

                                                                                                                                          Hadn’t seen that, will check it out.

                                                                                                                                          1. 1

                                                                                                                                            I’d really appreciate your opinion, since you cited Myrddin, which is my favourite contender for the package system of Jehanne.

                                                                                                                                            I do not really know any of the two (I used Pascal years ago.. but Oberon seem better).

                                                                                                                                            But of all the C alternatives I could decide to integrate in Jehanne (the way Perl was integrated in Unix) these seem the best two candidates for their balance between simplicity and practicality.

                                                                                                                                            Wirth’s Oberon win on simplicity, but Obi’s Myrddin win on practicality (according to my shallow understanding so far… take this with a grain of salt!)

                                                                                                                                            1. 2

                                                                                                                                              FWIW, Myrddin is probably going to be the easiest to port to Jehanne, since it already runs on Plan 9, and has a very small system abstraction layer.

                                                                                                                                              1. 1

                                                                                                                                                Hi Orib! Yes, you are right! Myrddin is the most practical choice for Jehanne.

                                                                                                                                                Also it provides language features I like, such as ADT and pattern matching, and it already has a practical standard library.
                                                                                                                                                But honestly I haven’t had the time to try your hints: I saved them from my irc log, but… really I didn’t had the time… :-(

                                                                                                                                                Nevertheless I’m also rather fashinated by Oberon-07: Wirth keeps cleaning it, removing redundant features. I know this add pressure to the library and applicative code, but…

                                                                                                                                                I think you can see the affinity with my strive for simplicity in Jehanne.

                                                                                                                                        2. 4

                                                                                                                                          All of those fall over on portability. Rust is amd64 and i386 only, myrddin is amd64 only, and building the zig compiler requires llvm. nim has the best story with amd64, i386, ppc and arm, which still isn’t enough.

                                                                                                                                          1. 1

                                                                                                                                            I think you are wrong about rust, there have been plenty of posts of embedded arm and other processors targetted by rust. LLVM has lots of targets and can compile itself, so it is relatively portable, though extremely complex.

                                                                                                                                            1. 3

                                                                                                                                              Is rust on other architectures done natively or by cross-compiling? I don’t know about the others but OpenBSD requires that the base install can build itself on every architecture.

                                                                                                                                              1. 1

                                                                                                                                                https://forge.rust-lang.org/platform-support.html - it seems like rustc can run on at least 5-6 architectures. and the groundwork is there for more.

                                                                                                                                                Zig itself has two stdlibs, one is based on libc so I bet that it could run on more platforms.

                                                                                                                                                1. 6

                                                                                                                                                  He is right, the only platforms at the moment able to self-build rust are amd64 & i386. OpenBSD requires much more. You participated in a previous thread so you know that rust in the base system is not likely to happen. Hence rust is not the answer to:

                                                                                                                                                  If I wanted to both re-write indent(1) in not-C and continue to distribute it as a part of FreeBSD, NetBSD, OpenBSD - which programming language should I “upgrade” to? What choice do I have?

                                                                                                                                                  With the current status quo, the only language fitting the above question I believe is Perl.

                                                                                                                                                  1. 2

                                                                                                                                                    https://github.com/Microsoft/checkedc seems to be one of the more practical approaches to upgrading C, though obviously not ready.

                                                                                                                                        3. 2

                                                                                                                                          Maybe Vala. It compiles to C but has dependency on GObject.

                                                                                                                                          1. 2

                                                                                                                                            The answer is C++. Every architecture that OpenBSD currently supports has a C++ compiler in base (well actually compXX.tgz). I’d imagine the answer is similar for FreeBSD and NetBSD. You may be able to get away with C++11 on the popular architectures but I think the less popular ones you’re stuck with C++03 or even C++98.

                                                                                                                                            1. 1

                                                                                                                                              The general choice is anything that compiles to C. If they’re picky about coding standards, what you would have then is a cultural argument instead of one on language capabilities. They wouldn’t allow something better. Then, you might be forced to do what folks like Per Brinch Hansen did back when hardware couldn’t run ALGOL: write in one language you can’t share for its benefits with a second version done against that in language you can share. To automate that, I recommended a while back someone make a safe, clean, superset language that’s compatible with C plus exports to readable code in that language. Throw in real macros and REPL for an extra reason to use it.

                                                                                                                                              Then, we don’t have a CVSup-style situation where author uses a safe, maintainable, high-level language for its benefits but people eventually rewrite that stuff in C anyway.