Nearly all the data comes from theGNU/Linux Distribution Timeline, which sadly has not be kept up to date.

As an ex-compiler writer have have plenty of sympathy for those trying to make a living selling compilers, but times change.

The problem used to be competition from many small compiler companies, then the amount of memory available on developer machines exploded, which is the critical factor holding back open source compilers.

An extreme example of unportable C is the book Mastering C Pointers: Tools for Programming Power, which was castigated recently. To be fair, that book has other flaws rather than being in a different camp, but I think that fuels some of the intensity of passion against it.

This… rather grossly undersells how much is wrong with that book. The author didn’t understand scope, for crying out loud, and never had a grasp of how C organized memory, even in the high-level handwavy “C Abstract Machine” sense the standard is written to.

There are better examples of unportable C, such as pretty much any non-trivial C program written for MS-DOS, especially the ones which did things like manually writing to video memory to get the best graphics performance. Of course, pretty much all embedded C would fit here as well, but you’ll actually be able to get and read the source of some of those MS-DOS programs.

In so doing, the committee had to converge on a computational model that would somehow encompass all targets. This turned out to be quite difficult, because there were a lot of targets out there that would be considered strange and exotic; arithmetic is not even guaranteed to be twos complement (the alternative is ones complement), word sizes might not be a power of 2, and other things.

Another example would be saturation semantics for overflow, as opposed to wraparound. DSPs use saturation semantics, so going off the top end of the scale plateaus, instead of causing a weird jagged waveform.

As for the rest, it’s a hard problem. Selectively turning off optimization for specific functions would be useful for some codebases, but aggressive optimization isn’t the only problem here: Optimization doesn’t cause your long type to suddenly be the wrong size to hold a pointer on some machines but not others. Annotating the code with machine-checked assumptions about type size, overflow behavior, and maybe other things would allow intelligent warnings about stupid code, but… well… try to get anyone to do it.

It’s a terrible source. As the wiki says: “The C rules and recommendations in this wiki are a work in progress and reflect the current thinking of the secure coding community. Because this is a development website, many pages are incomplete or contain errors. “

My review of the 2008 edition of the published guidelines pointed out that they were full of errors and omissions. So now they have started wiki for people to fix their problems.

Amusing that this type of person goes unnoticed when many of us tell people what we do. I fall into this category as I was describing in another thread. What she misses is many of us aren’t rich or the work sustainable: many people in lower to middle classes sacrificing money or status to do deep research and development in a field that interests them and/or they find necessary. They might also not like the priorities of commercial or government funding groups.

For instance, I thought figuring out how to make computers that don’t fail or get hacked was a thing we desperately needed. I believed both livelihoods and lives were at stake. That we had access to them was a social good that neither the markets nor FOSS were really serving. It was also an interesting, deep, rabbit hole of a problem crossing many sub-fields of IT, economics, and psychology. That she misses people without money doing it altruistically surprises me more given she wrote the report on FOSS developers working with little to no money or contributions on critical stuff that mattered to them. Same kind of thing I think with different work output.

Still a good write-up that will draw attention to the concept. We might get more people doing it or publishing what they’re doing. I think most of us don’t publish enough. We should accept some of the troubles of that since the ideas get out there more. I also like this quote focusing on the obsessive nature of deep, independent research:

“I understand, then, why researchers flock to the safety of institutions. Imagine studying something that nobody else is studying, for reasons you can’t really articulate, without knowing what the outcome of your work will be. For the truly obsessed person, the need for validation isn’t about ego; it’s about sanity. You want to know there’s some meaning behind the dizzying mental labyrinth that you simultaneously can’t escape and also never want to leave.”

Research in software engineering does not need lots of hardware, the funding needed to be a gentleman scientist in this field is money to live and basic computing equipment.

The startup cost is high, after all it is necessary to be reasonably expert in the field. But again, this is the personal time needed to spend time reading a lot and gaining practical experience.

It feels like we are through the valley of releases that needed to restore Python 2 compatibility. Good to see releases with a few innovations that will make my life easier.

I have always thought that Herbert Schildt’s C books contained more mistakes than any other C books out there.

Clive Feather’s ‘famous’ review and a review of a later book by seebs (of obfuscated fame): C: The complete nonsense

Ok, the summer project has just started and he has not read the standard yet.

“undefined behavior: behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements”

Let’s hope he reads the C standard before implementing any checks. A freely available pdf that is not the actual C standard you buy in the shops (which has a fancy ISO cove r on it)

“A minimalist knowledge approach to software engineering is cost effective because most code does not exist long enough to make it worthwhile investing in reducing future maintenance costs. Yes, it is more expensive for those that survive to become commonly used, but think of all the savings from not investing in those that did not survive.”

This is something that I’m probably going to have to think more on. @derek-jones might even have data to support it in his collection. My data, though, indicated that most real-world projects from the 1960’s on up to present times run into problems late in the lifecycle they have to fix. Those fixes usually cost more in money or reputation. Some groups spent small, upfront investment preventing most problems like that. They claim it usually paid off in various ways. This was especially true if software was long-lasting. There were times when the quality cost more overall on top of a thrown-together project.

Another issue is is that pervasively-buggy software conditioned users to expect that it’s normal. This reduces demand or competitive advantage of high-quality, mass-market software. Many firms, esp small or startups, can profitably supply buggy software so long as it meets a need and they fix the bugs. In enterprise market, you can even sell software that barely works or doesn’t at all so long as it appeared to meet a need making someone in the company look good. So, this needs to be factored into the decision of whether to engineer software vs throw it together.

I still say lean toward well-documented, easy-to-change software just in case you get stuck with it. You can also charge more in many markets with better rep. Use the amount of QA practices that the market will pay for. If they pay nothing, use stuff that costs about nothing like interface checks, usage-based testing, and fuzzing. If they’ll pay significant amount, add more design/code review, analysis/testing, slices of specialist talent (eg UI or security), improvements to dependencies, and so on.

Having got one business prediction right, I will stick my neck out and make another one (plus the obvious one that the world will not end because of this purchase).

Microsoft should have bought GitHub after the LinkedIn purchase.

I’d offer to buy the companies or groups controlling most of the mining power. It’s usually an oligopoly like in the for-profit, non-crypto markets. Even possible to pay off individual executives to cut deals to lower the price. The resulting buy would probably be way, way, way less than $100 billion for Bitcoin. Hell, it might be less than a billion. That’s owning it, too, rather than a bribe to sabotage it in some way that looks like an accident. That could be mere millions.

Cutting free from the ball and chain makes it easier for committee members to add more complexity.

There are PoW-less cryptocurrencies being developed. If these gain traction and turn out to be secure, then we can leave behind the first generation of cryptos based on PoW.

Ok, it’s time to stop developing software, get some Verilog books, and go after them coins. That’s what we need to do people. :)

On a serious note, one thing the author didn’t mention that might reduce barrier to entry are structured ASIC’s. They use a bit more energy, they’re a bit slower, and have much lower cost to develop/deploy. The main company doing it is eASIC with their Nextreme’s. I remember their 90nm option had prototyping of fifty to sixty grand for a pile of chips at one point. Get it working on an FPGA designed for a conversion. Then, see if they can put it on a Nextreme.

pdf graphs to csv is the most interesting project :-)

The author has a pretty explicit bias toward Earley parsing, and using LL(k) over LR(k) methods (which I mostly share). There’s a wealth of information here, but he’s missing a couple other significant strands of generalized parsing research outside of Marpa’s particular Earley variant (perhaps because they don’t fit within his narrative).

GLR (Generalized LR):

• Tomita, LR parsers for natural languages, 1984.
• Tomita, Efficient parsing for natural language, 1986.
• Scott and Johnstone, Generalised bottom up parsers with reduced stack activity, 2005. This segues into their later work with Earley and GLL.

GLL (Generalized LL):

• Scott and Johnstone, GLL Parsing, 2010.
• Afroozeh and Izmaylova, Faster, Practical GLL Parsing, 2015.

Outside of Earley, GLL seems like a very practical generalized parsing approach. Instaparse is one implementation.

Earley / Shared Packed Parse Forests:

• Scott, SPPF-style parsing from Earley recognisers, 2008 (and several related papers by Scott and Johnstone). Note: I’ve never been able to get the approach described in the paper implemented correctly, and not for lack of trying.

Earley Intersection Parsing (not sure if there’s a canonical name for this):

• Bar-Hillel, Perles, and Shamir, On formal properties of simple phrase structure grammars in Zeitschrift für Phonetik, Sprachwissenschaft und Kommunikationsforschung, 1961. Proves some important results about intersecting automata and context free grammars.
• Chapter 13 (“Parsing as Intersection”) of Grune and Jacobs (also cited elsewhere in his timeline), particularly 13.4 (pgs. 437-439): This describes Bar-Hillel’s Intersection Parsing approach using contemporary CS & parsing terminology, and then suggests combining it with Earley parsing. While the basic intersection parsing method produces an astonishing amount of dead-end nodes to garbage-collect later, Earley parsers limit searching to productions reachable from the start symbol. If the completer is modified to produce non-terminals in the intersection parsing format (which is easy), intersection parsing nicely handles producing the parse forest (with structural sharing, when ambiguity produces multiple derivations).

I’ve been working on a C library for Earley Intersection parsing, and an awk-like, pattern-match-directed language based on it, but working on testing them thoroughly tends to lead me to working on theft instead.

Amazing. Well written, well researched and concise. Parsing is such an interesting problem - it arises naturally almost anywhere you find text.
I’ve read the phrase “parsing is a solved problem” so often in discussions about it but this article puts that in its place. Thoroughly.