1. 1

    You eventually need to normalize the fractions somehow and I don’t see this in your work or documentation.

    Andrew (the author of the “User-specified ordering with fractions” wiki page) has given a query to update all fractions across the set of rows to be ordered. That query could be further simplified if you use deferred exclusion constraint instead of a unique constraint, though he prefers the latter.

    1. 1

      OK, good call I’ll implement a similar function for that. As a side note, I wonder how long it takes before the growing terms of the fractions created by rational_intermediate() begin to cause problems and should get renormalized?

    1. 22

      This article is great except for No 3: learning how hardware works. C will teach you how PDP-11 hardware works with some extensions, but not modern hardware. They have different models. The article then mentions computer architecture and assembly are things they teach students. Those plus online articles with examples on specific topics will teach the hardware. So, they’re already doing the right thing even if maybe saying the wrong thing in No. 3.

      Maybe one other modification. There’s quite a lot of tools, esp reimplementations or clones, written in non-C languages. Trend started getting big with Java and .NET with things like Rust and Go making some more waves. There’s also a tendency to write things in themselves. I bring it up because even the Python example isn’t true if you use a Python written in Python, recent interpreter tutorials in Go language, or something like that. You can benefit from understanding the implementation language and/or debugger of whatever you’re using in some situations. That’s not always C, though.

      1. 14

        Agreed. I’ll add that even C’s status as a lingua franca is largely due to the omnipresence of unix, unix-derived, and posix-influenced operating systems. That is, understanding C is still necessary to, for example, link non-ruby extensions to ruby code. That wouldn’t be the case if VMS had ended up dominant, or lisp machines.

        In that way, C is important to study for historical context. Personally, I’d try to find a series of exercises to demonstrate how much different current computer architecture is from what C assumes, and use that as a jumping point to discuss how relevant C’s semantic model is today, and what tradeoffs were made. That could spin out either to designing a language which maps to today’s hardware more completely and correctly, or to discussions of modern optimizing compilers and how far abstracted a language can become and still compile to efficient code.

        A final note: no language “helps you think like a computer”. Our rich history shows that we teach computers how to think, and there’s remarkable flexibility there. Even at the low levels of memory, we’ve seen binary, ternary, binary-coded-decimal, and I’m sure other approaches, all within the first couple decades of computers’ existence. Phrasing it as the original author did implies a limited understanding of what computers can do.

        1. 8

          C will teach you how PDP-11 hardware works with some extensions, but not modern hardware. They have different models.

          I keep hearing this meme, but pdp11 hardware is similar enough to modern hardware in every way that C exposes. Except, arguably, with the exception of NUMA and inter-processor effects.

          1. 10

            You just countered it yourself even with that given prevalence of multicores and multiprocessors. Then there’s cache hierarchies, SIMD, maybe alignment differences (memory is fuzzy), effects of security features, and so on.

            They’d be better of just reading on modern, computer hardware and ways of using it properly.

            1. 6

              Given that none of these are represented directly in assembly, would you also say that the assembly model is a poor fit for modeling modern assembly?

              I mean, it’s a good argument to make, but the attempts to make assembly model the hardware more closely seem to be vaporware so far.

              1. 6

                Hmm. They’re represented more directly than with C given there’s no translation to be done to the ISA. Some like SIMD, atomics, etc will be actual instructions on specific architectures. So, Id say learning hardware and ASM is still better than learning C if wanting to know what resulting ASM is doing on that hardware. Im leaning toward yes.

                There is some discrepency between assembly and hardware on highly-complex architectures, though. The RISC’s and microcontrollers will have less, though.

            2. 1

              Not helped by the C/Unix paradigm switching us from “feature-rich interconnected systems” like in the 1960s to “fast, dumb, and cheap” CPUs of today.

            3. 2

              I really don’t see how C is supposed to teach me how PDP-11 hardware works. C is my primary programming language and I have nearly no knowledge about PDP-11, so I don’t see what you mean. The way I see it is that the C standard is just a contract between language implementors and language users; it has no assumptions about the hardware. The C abstract machine is sufficiently abstract to implement it as a software-level interpreter.

              1. 1

                As in this video of its history, the C language was designed specifically for the hardware it ran on due to its extremely-limited resources. It was based heavily on BCPL, which invented “programmer is in control,” that was what features of ALGOL could compile on another limited machine called an EDSAC. Even being byte-oriented versus word-oriented was due to PDP-7 being byte-oriented vs EDSAC that allowed word-oriented. After a lot of software was written in it, two things happened:

                (a) Specific hardware implementations tried to be compatible to it in stack or memory models so that program’s written for C’s abstract machine would go fast. Although possibly good for PDP-11 hardware, this compatibility would mean many missed opportunities for both safety/security and optimization as hardware improved. These things, though, are what you might learn about hardware studying C.

                (b) Hardware vendors competing with each other on performance, concurrency, energy usage, and security both extended their architectures and made them more heterogenous than before. The C model didn’t just diverge from these: new languages were invented (esp in HPC) so programmers could easily use them via something that gives a mental model closer to what hardware does. The default was hand-coded assembly that got called in C or Fortran apps, though. Yes, HPC often used Fortran since it’s model gave them better performance than C’s on numerical applications even on hardware designed for C’s abstract machine. Even though easy on hardware, the C model introduced too much uncertainty about programmers’ intent for compilers to optimize those routines.

                For this reason, it’s better to just study hardware to learn hardware. Plus, the various languages either designed for max use of that hardware or that the hardware itself is designed for. C language is an option for the latter.

                “ it has no assumptions about the hardware”

                It assumes the hardware will give people direct control over pointers and memory in ways that can break programs. Recent work tries to fix the damage that came from keeping the PDP-11 model all this time. There were also languages that handled them safely by default unless told otherwise using overflow or bounds checks. SPARK eliminated them for most of its code with compiler substituting pointers in where it’s safe to do so. It’s also harder in general to make C programs enforce POLA with hardware or OS mechanisms versus a language with that generated for you or having true macros to hide boilerplate.

                “ The C abstract machine is sufficiently abstract to implement it as a software-level interpreter.”

                You can implement any piece of hardware as a software-level interpreter. It’s just slower. Simulation is also a standard part of hardware development. I don’t think whether it can be interpreted matters. Question is: how much does it match what people are doing with hardware vs just studying hardware, assembly for that hardware, or other languages designed for that hardware?

                1. 3

                  I admit that the history of C and also history of implementations of C do give some insight into computers and how they’ve evolved into what we have now. I do agree that hardware, operating systems and the language have been all evolving at the same time and have made impact on each other. That’s not what I’m disagreeing with.

                  I don’t see a hint of proof that knowledge about the C programming language (as defined by its current standard) gives you any knowledge about any kind of hardware. In other words, I don’t believe you can learn anything practical about hardware just from learning C.

                  To extend what I’ve already said, the C abstract machine is sufficiently abstract to implement it as a software interpreter and it matters since it proves that C draws clear boundaries between expected behavior and implementation details, which include how a certain piece of hardware might behave. It does impose constraints on all compliant implementations, but that tells you nothing about what “runs under the hood” when you run things on your computer; an implementation might be a typical, bare-bones PC, or a simulated piece of hardware, or a human brain. So the fact that one can simulate hardware is not relevant to the fact, that you still can’t draw practical assumptions about its behavior just from knowing C. The C abstract machine is neither hardware nor software.

                  Question is: how much does it match what people are doing with hardware vs just studying hardware, assembly for that hardware, or other languages designed for that hardware?

                  What people do with hardware is directly related to knowledge about that particular piece of hardware, the language implementation they’re using, and so on. That doesn’t prove that C helps you understand that or any other piece of hardware. For example, people do study assembly generated by their gcc running on Linux to think about what their Intel CPU will do, but that kind of knowledge doesn’t come from knowing C - it comes from observing and analyzing behavior of that particular implementation directly and behavior of that particular piece of hardware indirectly (since modern compilers have to have knowledge about it, to some extent). The most you can do is try and determine whether the generated code is in accordance with the chosen standard.

                  1. 1

                    In that case, it seems we mostly agree about its connection to learning hardware. Thanjs for elaborating.

            1. 1

              Struggling through these has made it clear to me how outdated my SQL knowledge is. Can anyone recommend a favorite guide to all the neat stuff in SQL:2003 for a postgresql/mariadb user?

              1. 2

                I don’t know of any guides, but I found the Postgresql docs to be good enough: Window function list and window function usage syntax.

                1. 2

                  https://momjian.us/main/presentations/sql.html Postgres-centric presentations. Not specifically about SQL:2003. I admit I haven’t seen them all, but I attented one of the “Window Magic” presentations and it was very good. Which is not surprising given the author, Bruce Momjian :)

                  1. 3

                    Since NetBSD gets less credit, I’ll add that the things that make it portable also made it get a lot of use in embedded and CompSci. Basically, spots where customization happens at the kernel code level in a lot of places. The folks crazy enough to try to rewrite a UNIX in a new language also always started with NetBSD since it’s easiest to rewrite. None are easy, mind you, but they didn’t think they’d have a chance with FreeBSD or OpenBSD for different reasons.

                    1. 1

                      How portable is NetBSD, compared to the other BSDs?

                      1. 3

                        I have no idea haha. I’ve just read up on a lot of work by people building on various OS’s. In UNIX land, those building on NetBSD seemed to have an easier time. Here’s three things many said:

                        1. It wasn’t huge like FreeBSD or Linux.

                        2. Its focus on portability made it easier to change.

                        3. Its community was welcoming or helpful to those trying to rewrite it for various reasons.

                        I’m not sure how true these are in general or currently since I don’t code BSD/Linux kernels or userlands. I just kept seeing a lot of people doing BSD mods say those things about NetBSD specifically. Probably better for developers of various BSD’s to chime in at this point since we have at least three represented in Lobsters community.

                        1. 4

                          The NetBSD rump kernel concept is one real-world demonstration that the abstraction layers are at least fairly good. It’s not clear you couldn’t do something similar with another OS, but NetBSD seems to have managed it with quite little actual porting needed (drivers run completely unmodified).

                        2. 2

                          They used to joke that it even runs on your toaster. Obviously, most toasters don’t have an embedded OS, but, I think the joke implies something about how portable they desire it to be.

                          1. 5
                            1. 2

                              Ha! I hadn’t seen this, but I’m certainly not surprised!

                            2. 3

                              Obviously, most toasters don’t have an embedded OS

                              Yet.

                              The obvious use case for such a device is cryptocurrency mining.

                              1. 5

                                The obvious use case for such a device is cryptocurrency mining.

                                Yep, This should generate enough heat to burn a few toast :+)

                        1. 2

                          I haven’t done any research so I don’t have an opinion on the matter, but I just recalled recently seeing http://shape-of-code.coding-guidelines.com/2017/11/21/grace-hopper-manager-after-briefly-being-a-programmer/

                          1. 5

                            Here are some reasons for not loving Ada:

                            • It is not type safe, even if you only use features that were meant to be type safe. In other words, Ada’s type system fails at being a type system. “Almost safe” languages (Ada, Eiffel, etc.) are actually more dangerous than C, since at least C programmers are permanently aware that they are walking through a semantic minefield, whereas users of “almost safe” languages often think they aren’t.

                            • Much of Ada’s type system exists to automate the insertion of runtime safety checks, just like much of pre-templates C++’s type system exists to automate the insertion of runtime type coercions and cleanup operations. Now, automation is nice, but, are types the right tool for this job? I do not think so. Macros offer a more compelling alternative.

                            • With Ada, you can pick at most two between safety, efficiency and an implementation-independent semantics. Safely getting rid of the onerous runtime checks requires external analyses not evidently justified by the Ada specification. A formal semantics for the whole of Ada (not just the subset without dynamic resource management) is very much missing.

                            1. 5

                              at least C programmers are permanently aware that they are walking through a semantic minefield, whereas users of “almost safe” languages often think they aren’t.

                              This is a very good point, and in my experience it also applies to functional purity. In an inherently impure language like C or Java, you’re always aware that shared mutable state is present and consequently you’re on guard for it, so you generally won’t build abstractions that depend on purity in order to work. In a language like Haskell or Elm, on the other hand, you know that the compiler actually offers strong purity guarantees at the type level, so you can use functional abstractions on a large scale with confidence. The real problem lies in the middle, when you’re using a language or a library that is pure by convention but will accept impure functions as well, and you build functional abstractions on it that break in really subtle and unexpected ways when mutability and nondeterminism come into play. I’d say an example of the latter category is some modern JS frameworks like React, which depend heavily on the programmer keeping track of what subset of their code has to be pure and what subset can be imperative, all in a language that has no compile-time ability to distinguish the two.

                              1. 2

                                It is not type safe, even if you only use features that were meant to be type safe. In other words, Ada’s type system fails at being a type system. “Almost safe” languages (Ada, Eiffel, etc.) are actually more dangerous than C, since at least C programmers are permanently aware that they are walking through a semantic minefield, whereas users of “almost safe” languages often think they aren’t.

                                That example doesn’t support the point because it’s using an Unchecked package, which exist solely to side step the type system. They’re the Ada equivalents of Haskell’s Unsafe functions.

                                Much of Ada’s type system exists to automate the insertion of runtime safety checks, just like much of pre-templates C++’s type system exists to automate the insertion of runtime type coercions and cleanup operations. Now, automation is nice, but, are types the right tool for this job? I do not think so. Macros offer a more compelling alternative.

                                I don’t think I agree with the premise that Ada’s type system exists to automate the insertion of runtime safety checks. It can be viewed in that way, but so can any other type system. In reality, an Ada compiler is free to skip any runtime check that it can detect is unnecessary.

                                1. 2

                                  That example doesn’t support the point because it’s using an Unchecked package, which exist solely to side step the type system.

                                  Re-read the code. The Conversion function doesn’t use anything from any Unchecked package. The Unchecked_Conversion function is only imported for comparison purposes.

                                  It can be viewed in that way, but so can any other type system.

                                  That would be wrong. For certain type systems, e.g. ML’s, it is a theorem that legal programs can be striped of all type information without altering the dynamic semantics of the language.

                                  In reality, an Ada compiler is free to skip any runtime check that it can detect is unnecessary.

                                  Yes, but it is in general undecidable whether a specific check can be safely elided. Things wouldn’t be so bad with a formal semantics for Ada, allowing programmers to verify by themselves that specific checks can be safely elided.

                                  EDIT: Fixed identifier. Added last sentence.

                                2. 2

                                  I would like to add another reason: not much choice between implementations of Ada.

                                  1. 2

                                    This one doesn’t bother me at all, although I can see why others would consider it a problem.

                                1. 1

                                  Personally, I think Ada is not ready to replace C in a BSD base system. It doesn’t have mature competing implementations under friendly license. As far as I know, the only free Ada implementation is a GNU one.

                                  Rust is even less ready. It doesn’t even have a standard, so even if I wanted to start a new Rust implementation, that would be a show stopper. At the present time, creating anything new that has to be durable in Rust is like writing in Python 2.

                                  Changes in an operating system require work, obviously. If Rust came with the workforce ready to rewrite a base (like in BSDs) from C to Rust, then that might be the first argument in favor of doing it. Claimed language safety is not it, especially not since all the static and dynamic analysis work that has been done over the last decade.

                                  1. 2

                                    For PgBouncer there are a few modes it can be run in: session pooling and transaction pooling.

                                    There is also the statement pooling mode. Which is like transaction pooling mode, but one that doesn’t allow use of multi-statement transactions.

                                    What you want to more efficiently manage your connections is transaction pooling. Transaction pooling will grant a connection when you run BEGIN; and return the transaction when you COMMIT;

                                    I don’t know what the intention of the above sentence was. Certainly a pgBouncer instance running in transaction pooling mode will not require you to wrap every statement in a BEGIN-COMMIT block. Every statement in Postgres is already wrapped in a transaction, even simplest ones like SELECT 1;

                                    Note: with transaction pooling you do need to make sure you turn off prepared statements which Rails turns on by default.

                                    There are more Postgres features that transaction pooling mode will not let you use properly or at all. SET/RESET, LISTEN, currval()/lastval() are some of the more popular ones. Annoyingly, it’s popular for web frameworks to use lastval() to retrieve the id of the last inserted row (instead of utilizing Postgres’s INSERT … RETURNING …).

                                      1. 1

                                        Sadly, pgtune doesn’t seem to be maintained or at least updated to newer Postgres versions. And there’s too much going on from one PG release to another for pgtune to stay relevant for years.

                                      1. 1

                                        Anyone know why Postgres doesn’t just check (at startup) how fast sequential reads are and how fast random seeks are?

                                        I’m not generally a fan of turning something into a config setting when the machine can figure it out for you…

                                        1. 3

                                          With Oracle (and I assume the same is possible with Postgres) you can have each table on a different drive, with table partitioning the same table could span across multiple drives (ie. if you know records older than one year are not access often you could partition them onto a slower drive). This makes an automated test on startup not enough to determine the actual cost.

                                          1. 2

                                            With Oracle (and I assume the same is possible with Postgres) you can have each table on a different drive

                                            Yes, that’s achievable with tablespaces: https://www.postgresql.org/docs/current/static/manage-ag-tablespaces.html

                                            You can set different values of seq_page_cost, random_page_cost and effective_io_concurrency per tablespace.

                                          2. 2

                                            Difficult to make a reliable benchmark? You don’t want the answer to change after a restart with cached data, etc. which implies a minimum threshold of rigor and I’m not sure how enthused I’d be if my database were conducting such a rigorous test automatically.

                                          1. 5

                                            This has been listed on a (a bit outdated) PG wiki page about tuning: https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server

                                            Another popular thing people tend to update is cpu_tuple_cost.

                                            The book “PostgreSQL 9.6 High Performance” should give significantly more information about that and many other things - for a price, of course.

                                            1. 1

                                              The book “PostgreSQL 9.6 High Performance” should give significantly more information about that and many other things - for a price, of course.

                                              Is this an updated version of the excellent “PostgreSQL 9.0 High Performance”? I liked the book, especially as it takes a deep look at how to evaluate the system below the database. (which is useful in any DB context)

                                              The comment here seems to indicate that the update didn’t work well: https://www.amazon.com/product-reviews/1784392979/ref=cm_cr_dp_d_cmps_btm?ie=UTF8&reviewerType=all_reviews

                                              1. 1

                                                It is an updated version. I read the 9.0 book and I liked it. I haven’t read the 9.6 version, only skimmed through it and read the table of contents. The updated version can’t be as bad as the review claims, for example it does talk about Block Range Indexes which were introduced in PG 9.5 or the parallel stuff that went into PG 9.6. I can’t comment on the editing.

                                            1. 4

                                              I believe HAProxy still has the “no crashes in production reported in XX years” which still makes it king, and I wonder if this has had a thorough security audit being C++. It can’t hurt to have more players, though.

                                              1. 2

                                                HAProxy can’t actually serve content though, so you still need something like nginx, Apache or lighttpd behind it.

                                                1. 2

                                                  I’ve been enjoying experimenting with haproxy and application code written in Go.

                                                2. 1

                                                  We had one version of HAProxy crash every 49.7 days. See https://www.mail-archive.com/haproxy@formilux.org/msg25807.html for details and search through the mailing list for more details if you want.

                                                1. 1

                                                  A better solution, based on writable CTEs, is described in the comments under the blog post.

                                                  1. 1

                                                    The article confused me very much. I had to read the original question and the first part of the article once again, to conclude what the point was. I now think it boils down to this: if pointer p points to one of the elements of array A of size S, then the p >= A && p < A + S will yield true, but if a pointer x points to something outside the allocated region, x >= A && x < A + S may also yield true (or yield false or terminate the program or do anything else, because the comparison invokes undefined behavior).

                                                    1. 2

                                                      … web apps fix those but then re-introduce their own very similar mistakes: SQL injection, XSS, XSRF, header injection, MIME confusion, and so on. This leads to a simple thesis: I put it to you that it’s impossible to write secure web apps.

                                                      Not sure about that conclusion - SQL injection is preventable by prepared statements, XSS and XSRF are preventable by CORS. These aren’t considerations made in the design phase relating to the architecture that I’d agree are hard/impossible to prevent, these are exploits that should be handled by abstraction layers in the frameworks you use.

                                                      1. 2

                                                        SQL injection is preventable by prepared statements

                                                        Or more precisely: parameterized queries, where parameters are never interpreted as part of the query. The distinction between the two is important at least in Postgres, because prepared statements require more work on the client side than parameterized queries do.

                                                        1. 2

                                                          Just like memory overflows and double-frees should be solved by abstractions in the language you use :)

                                                          1. 1

                                                            XSS ans XSRF are not preventable by CORS to my knowledge. at the opposite, they are intinsic to web technology and bypass CORS.

                                                          1. 1

                                                            That’s the PuTTY guy… Interesting

                                                            1. 1

                                                              Of the things I knew, also the NASM guy and the Coroutines in C guy. His How to Report Bugs Effectively also seems popular.

                                                            1. 8

                                                              I’ve been enjoying the back-and-forth with @peter at https://lobste.rs/s/w4auk6/more_shell_less_egg_all_this#c_xln3fi. However, the deep nesting of this thread is getting really silly.

                                                              I find that the mailing list mode helps a great deal.

                                                              https://lobste.rs/s/jg3eet

                                                              1. 1

                                                                I’m a great fan of the feature.

                                                              1. 1
                                                                • exotic archs, endianess: the boat has sailed, deal with it

                                                                How are we meant to parse this?

                                                                1. 4

                                                                  They’re not supporting exotic architectures anymore.

                                                                  1. 3

                                                                    2017 has exactly four vanilla architectures: x86, amd64, arm, and arm64.

                                                                    1. 3

                                                                      I should mention that Rust does support more than just that :p https://forge.rust-lang.org/platform-support.html

                                                                      1. 1

                                                                        Oops, I meant to be speaking about the zeitgeist rather than just Rust.

                                                                        The other architectures I see in Rust’s Tier 2 support are MIPS, PPC and asm.js. Familiar names but not exactly what I would call vanilla these days.

                                                                    2. 3

                                                                      Time to buy a real computer, not one of those sparc toys.

                                                                      1. 3

                                                                        I like to couple toy computers (ARM, sparc64) with toy operating systems (BSDs). It’s fun because toys are for fun :)

                                                                    1. 3

                                                                      I think this might be the talk:

                                                                      https://youtu.be/8wuW8lfsVGc?t=1h27m9s