you need to provide a reason more substantive than theoretical compiler optimizations that are shown by hand waving

Read the post, then; there are substantive reasons in it. I’m not engaging with you if you’re going to start by misrepresenting reasoned arguments as “hand waving”.

This is essentially the same as one of the arguments I addressed in the post. Although in certain types of programs, particularly lower-level languages (like C and Rust) where the code is written directly by a human programmer, there probably are not going to be many cases where the optimisations enabled by having overflow be undefined will kick in. However, if the program makes heave use of templates or generated code, or is produced by transpiling a higher-level language with a naive transpiler, then it could do (I’ll conceded this is theoretical in that I can’t really give a concrete example). The mechanism by which the optimisation works is well understood and it isn’t too difficult to produce an artificial example where the optimisation grants a significant speed-up in the generated code.

Also, in the case of Rust programs, you can’t really reliably assess the impact of wrapping on overflow unless there is an option to make overflow undefined behaviour. Is there such an option in Rust?

This cannot be entirely true. As a reducto ad absurdum, it would be possible in principle to laboriously define all the things that compilers currently do with undefined behaviour and make that the new definition of the behaviour, and there would then be zero performance impact

Clearly that would be absurd, and it’s certainly not what I meant by “remove all undefined behaviour”. Your “possible in principle” suggestion is practically speaking completely impossible, and what I said was true if you don’t take such a ridiculously liberal interpretation of it. Let’s not play word games here.

C compiler writers might argue that removing all undefined behaviour without compromising performance would be prohibitively expensive

They might, but that’s not what I argued in the post.

I don’t think it’s been shown that that generalises to realistic programs or to a compiler that was willing to put a bit more effort in.

There’s no strong evidence that it does, nor that it wouldn’t ever do so.

I suspect that one issue is that compilers may manifest different runtime behaviour for undefined behaviour, depending on what specific code the compiler decided to generate for a particular source sequence. In theory you could document this with sufficient effort, but the documentation would not necessarily be useful; it would wind up saying ‘the generated code may do any of the following depending on factors beyond your useful control’.

(A canonical case of ‘your results depend on how the compiler generates code’, although I don’t know if it depends on undefined behaviour, is x86 floating point calculations, where your calculations may be performed with extra precision depending on whether the compiler kept everything in 80-bit FPU registers, spilled some to memory (clipping to 64 bits or less), or used SSE (which is always 64-bit max).)

It’s not only possible: Ive seen formal semantics that define various undefined behaviors just like you said. People writing C compilers can definitely do it if they wanted to.

-fwrapv

Undefined behaviour practically guarantees all the things that you’ve just said you don’t want.

No, it doesn’t. In making this claim, you are implying that code which invokes undefined behaviour still has defined semantics. Certainly, the compiler won’t guess what was intended and then try to use undefined behaviour to try and implement that; it assumes that what it was told was intended (i.e. what was expressed by the code, according to the semantics of the language) is what was intended. That’s a reasonable assumption to make given the nature and purpose of a compiler.

It would be hard to do worse than deleting overflow checks that the programmer wrote into the code, but only sometimes, which is the current behaviour.

I agree that, if the compiler could divine that some check (which it would otherwise optimise away) is meant to be a wrapping overflow check for some particular preceding operation, it would be nice if the compiler could issue a strong warning. I do not believe it should ever silently “fix” the problem by applying wrapping semantics to the relevant preceding operations, because that is going to lead to C programmers increasingly believing that overflow does have wrapping behaviour, and I don’t think that is a good idea even if wrapping was the ideal overflow behavior.

One question is, would assuming that most/all code that fits that general pattern is an indication that wrapping behaviour was required by previous operations and compiling accordingly (with a warning), have a negative impact on optimisations and performance? I think the answer is “probably not for most existing programs”, but I also think it could clearly have a significant impact potentially. This could be extended generally to a question about whether wrapping semantics generally could impact optimisation (which I think has the same answer).

Another question is, is wrapping behaviour generally useful, other than for purpose of these overflow checks? I’ve address that in the post; I think the answer is clearly no.

However, is wrapping better behaviour (disregarding performance) than undefined behaviour? From a safety perspective it may be slightly better, since the effect of overflow bugs is more constrained, but the bugs can still happen and can still be exploited. This is anecdotal, but most of the overflow-related security holes that I’ve seen, other than the small number due to compiler-removed post-overflow checks, have been directly caused by the wrapping and not by any other associated undefined behaviour.

So:

• I’m not convinced that wrapping semantics will never significantly impact performance in real programs
• I don’t see wrapping as a useful behaviour, other than for implementing erroneous post-overflow checks, which can and should anway be re-written as correct pre-overflow checks
• I’m not convinced that wrapping on overflow is much better than UB on overflow, in practice (yes, in theory UB can do very nasty things. But for this particular case of UB, practically speaking, the effects are usually constrained). But let’s assume that we want to avoid arbitrary UB. In that case:
• Trap-on-overflow is a superior alternative because it’s safer, except in cases where performance is critical (and in those cases, wrapping might not be the right choice either).

Getting back to what you said, the only way to not delete any overflow checks is to enforce wrapping semantics everywhere, and I disagree with that due to the points above.

Indeed you can’t be sure, which is why a responsible compiler should fail-safe rather than fail-dangerous. A missed optimization opportunity is much better than a security bug.

Agreed - that’s why trapping and not wrapping is the right behaviour.

Thanks for bringing up some reasonable discussion. I don’t think there’s a totally objective right/wrong answer at this point - but I hope you see some validity in the points I’ve raised above.

(some small edits made after posting to improve readability).

so from the C standard I can both conclude that sizeof(int) == 4 or 8 and for int i, i+1 > i is a theorem so a test if(i+1 <= i) panic(); is “bat-shit crazy”? Think about it. Testing to see if addition of fixed length ints overflows is not only mathematically sound, but it matches the operation of all the dominant processors - that’s how fixed point 2s complement math works which is why almost all processors incorporate an overflow bit or similar. Ints are not integers.

I am much more uncertain about other undefined behaviors, for example null dereference

It would be easy enough to define that as causing immediate termination

Easy enough to define it that way, sure, but I don’t think it would be a popular move in the embedded world – on MMU-less systems where the hardware might not trap it, seems like that would force the compiler to insert runtime checks before every pointer dereference.

It would be easy enough to define that as causing immediate termination; the real question is whether this would be worth doing.

Nobody is asking for C implementations to force traps on null dereference. Nobody. So why are you trying to explain it would be hard or have negative consequences?

I’d be very interested in what your semantic issues with Rust are.

A proper answer to that would need me to sit down for an hour (or more) and go through again the material on Rust to remember the issues I had. Some of them aren’t very significant, some of them are definitely subjective. I should qualify: I’ve barely actually used Rust, just looked at it a number of times and had second-hand exposure via friends who’ve been using it extensively. The main thing I can remember off the top of my head that I didn’t like is that you get move semantics by default when passing objects to functions, except when the type implements the Copyable trait (in which case you get a copy), so the presence or absence of a trait changes the semantics of an operation. This is subtle and, potentially, confusing (though the error message is pretty direct). I’d rather have a syntactic distinction in the function call syntax to specify “I want this parameter moved” vs copied.

Other things that bother me are lack of exceptions (I realise this was most likely a design decision, just not one that I agree with) and limited metaprogramming (the “hygienic macro” facility, when I looked at it, appeared a bit half-baked; but then, I’m comparing to C++ which has very extensive metaprogramming facilities, even if they have awful syntax).

I can assure you after attending the All Hands that this is definitely a hot topic, but also a hard one.

Yep, understood.

I’m happy that you took a look at the language, even if you came away wanting.

I’ll be continuing to watch closely. I’m very interested in Rust. I honestly think that some of the ideas it’s brought to the table will change the way future languages are designed.

Hmm?? This sounds very interesting! Please tell me more about it.

In linux: prctl(PR_SET_CHILD_SUBREAPER, 1); In DragonFlyBSD (and apparently FreeBSD too, I see): procctl(P_PID, getpid(), PROC_REAP_ACQUIRE, NULL);

In both cases this marks the current process as a “reaper” - any child/grandchild process which double-forks or otherwise becomes orphaned will be reparented to this process rather than to init. Dinit uses this already to be able to supervise forking processes, but it still needs to be able to determine the pid (by reading it from a pid file). There’s the possibility though of inserting a per-service supervisor process which can then be used to keep track of all the processes that a particular service generates - although it still doesn’t provide a clean way to terminate them; I think you really do need cgroups or jails for that.

its existence doesn’t justify stopping a program due to a failed allocation.

Yes, especially since overcommit can be turned off, which should largely (if not always - I’m not sure) prevent the OOM killer from acting.

Right. I was saying just let malloc return NULL and let the program deal with it instead of basically lying about whether the allocation succeeded or not. I disable memory overcommit on most of my systems.

I’ve put tens of thousands of lines of C into production, including multiple Linux kernel drivers. In one case, those kernel drivers were critical-path code on a device used in strain testing the wings of an airplane that you might’ve flown in by now.

I’m not a stranger to the kernel; I just left that world. Behavior like Linus’ in that email was part of the reason, though far from the only reason.

With all of that said: having written a bunch of systems software shouldn’t be a prerequisite for suggesting that we avoid attacking people personally when they make programming mistakes, or what we suspect are programming mistakes.

“brain damaged” is a meme among old school hackers which isn’t as strong of a word as you think.

Yikes. That “meme” is a whole other thing I don’t even care to unpack right now.

I don’t use hipster as an insult or to imply wrongness, but I do use it to invalidate his point. Gary is a Ruby developer. Linus is a kernel developer. The worlds are far removed from each other.

Gotcha. Kernal developer == real old-school hacker. Ruby developer == script kiddie hipster. Are we really still having this argument in 2018?

Right, “professionalism” implies that you only need to be nice to somebody when you want them to something for you or want their money. This should actually be about “respect”, whether or not you want a Linux contributor to do something for you or want their money.

If I look at who the contributors to the Linux kernel are, it would certainly appear to be a professional endeavor.

A large chunk of contributions to the kernel are made by people who are getting paid by the companies they work for to contribute. Sounds like a professional setting to me.

Linus himself is working in a professional capacity. He’s employed by the Linux Foundation to work on Linux. The fact he is employed to work on an open source project that he founded doesn’t make that situation non-professional.

It’s the same philosophy as network-level dependencies. A web app that depends on a mail service for some operations is not going to shutdown or wait to boot if the mail service is down. Each dependency should have a tunable retry logic, usually with an exponential backoff.

either those that depend on it will die or they will handle it fine

If they die, and are configured to restart, they will keep bouncing up and down while the dependency is down? I think having dependency resolution is definitely better than that. Restart the dependency, then the dependent.

You might like my own dinit (https://github.com/davmac314/dinit). It somewhat aims for that - handle services and dependencies, leave everything else to the pre-existing toolchain. It’s not quite finished but it’s becoming quite usable and I’ve been booting my system with it for some time now.

Au contraire. As I showed, C standard does not need to graft on a clumsy and painful anti-alias mechanism and programmers don’t need to go though stupid contortions with allocation of buffers that disappear under optimization , because the compiler does not need it. My code does’t have alignment problems. The justification for pointer alias rules is false. The end.

Incidentally, what POSIX does say (from 2.4.1 Signal Generation and Delivery):

The determination of which action is taken in response to a signal is made at the time the signal is delivered, allowing for any changes since the time of generation. This determination is independent of the means by which the signal was originally generated. If a subsequent occurrence of a pending signal is generated, it is implementation-defined as to whether the signal is delivered or accepted more than once in circumstances other than those in which queuing is required

Specific cases which require queuing, which I could find, are:

Per-process timers may be created that notify the process of timer expirations by queuing a realtime extended signal.

In 2.4.2:

When a signal is generated by the sigqueue() function or any signal-generating function that supports the specification of an application-defined value, the signal shall be marked pending and, if the SA_SIGINFO flag is set for that signal, the signal shall be queued to the process along with the application-specified signal value. Multiple occurrences of signals so generated are queued in FIFO order. It is unspecified whether signals so generated are queued when the SA_SIGINFO flag is not set for that signal.

The rationale section for the timer_create() states:

The specified timer facilities may deliver realtime signals (that is, queued signals) on implementations that support this option. Since realtime applications cannot afford to lose notifications of asynchronous events, like timer expirations or asynchronous I/O completions, it must be possible to ensure that sufficient resources exist to deliver the signal when the event occurs.

So, this implies that aynchronous I/O completion as well as timer timeout must be able to queue a signal without failing. That’s about the only specific mentions of this I can find. Note that asynchronous I/O uses a “struct sigevent” argument just as timer_create() does. Note also the implication that non-realtime signals (of which SIGCHLD is one) do not need to support queuing.

About SIGCHLD, in 2.4.3 Signal Actions:

When a process stops, a SIGCHLD signal shall be generated for its parent process, unless the parent process has set the SA_NOCLDSTOP flag.

Here it says “generated” rather than “queued”, implying that SIGCHLD can be coalesced.

until it returns ECHILD.

Sorry, until it returns 0.

Interesting. I had a similar thought (with respect to wait) when reading the OP, but wasn’t sure. It would be a nice follow up question for the author. Maybe if you know which child was terminated, you can be more precise with which child you attempt to reap? Otherwise, if you have to try many of them, that might have performance implications?

I still have not seen a well worked out example of how UB produces real optimizations

Some examples here (found by a search just now): https://kristerw.blogspot.com/2016/02/how-undefined-signed-overflow-enables.html

SPECint isn’t necessarily a good way to assess the affect of these particular optimisations.

I run stuff on CPU’s with different number of cores… that’s a really good one for knocking loose undefined behaviour bugs.

What do you mean by that? Just something like 2 vs 4 or as different as vs a 3 or 6 core that’s not a multiple of 2? Now that you mention it, I wonder if other interaction bugs could show up in SoC’s with mix of high-power and low-energy cores running things interacting. Maybe running on them could expose more bugs, too.

But of course, there is often no warning.

There is a false choice between inefficient code with run time bounds checking and compiler “optimizations” that break working code

Optimisations don’t break working code. They cause broken code to have different observable behaviour.

And, despite the claims in this article, there are cases where trivial optimisations such as assuming that signed integer arithmetic operations won’t overflow can lead to significant speedups (it’s just that these cases are not something as trivial as a single isolated loop).

Great. Let’s see an example.

I don’t have a code example to hand, and as I said they’re not trivial, but that doesn’t mean it’s not true. Since it can eliminate whole code paths, it can affect the efficacy of for example value range propagation, affect inlining decisions, and have other flow-on effects.

I think this attitude comes from a fundamental antipathy to the design of C or a basic misunderstanding of how it is used

I disagree.

My file:

#include <assert.h>

int foo(int x)
{
  assert(x >= 0);
  return x + 5;
}

My file after running it through the C preprocessor:

# 1 "x.c"
# 1 "/usr/include/assert.h"
 
 

 
 
 

 
# 12

# 15

#ident      "@(#)assert.h   1.10    04/05/18 SMI"

# 21

# 26
extern void __assert(const char *, const char *, int);
# 31

# 35

# 37

 
# 44

# 46

# 52

# 63

# 2 "x.c"

int foo(int x)
{
   ( void ) ( ( x >= 0 ) || ( __assert ( "x >= 0" , "x.c" , 5 ) , 0 ) );
  return x + 5;
}
#ident "acomp: Sun C 5.12 SunOS_sparc 2011/11/16"

Not an __atttribute__ to be found. This C compiler can now generate code as if x is never a negative value.

Which is almost, but not quite what one would want….

I’m not sure I understand you. assert() will abort if the expression given is false. That’s what it does. It also prints where the expression was (it’s part of the standard). If you don’t want to abort, don’t call assert(). If you expect that assert() is a compile-time check, well, it’s not.

I certainly feel there is room for compiler and optimizer writers work with design by contract style programmers to have a “mutually beneficial” two way conversation with the programmers when they write asserts.

There’s only so far that can go though. Put your foo() function in another file, and no C compiler can warn you.

assert() is also a standard C function, which means the compiler can have built-in knowledge of its semantics (much like a C compiler can replace a call to memmove() with inline assembly). The fact that GCC uses its __attribute__ extension for this doesn’t apply to all other compilers.

Since __builtin_unreachable only exists to guide the compiler, p has to be an expression with no side effects, and then the compiler can optimise it out because you don’t use its result.

The C standard is not the C language. It is a committee report attempting to codify the language. It is not made up of laws of physics - it can be wrong and can change. My argument is that the standard is wrong. Feel free to disagree, but please don’t treat the current instance of the standard as if they were beyond discussion.

In fact, I do want a different standard: one that is closer to my idea what the rules of the language should be in order to make the language useful, beautiful, and closer to the spirit of the design.

The compiler and linker don’t have total freedom to change layouts even in the current standard - otherwise, for example, memcpy would not work. Note: “Except for bit-fields, objects are composed of contiguous sequences of one or more bytes, the number, order, and encoding of which are either explicitly specified or implementation-defined.”

struct a{ int x[100];}; char *b = malloc(sizeof(int)*101; struct a *y = (struct a *)b; if(sizeof(struct a)) != sizeof(int)*100 ) panic(“this implementation of C won’t work for us\n”); …. do stuff … y->x[100] = checksum(y);

But worse,

message = readsocket(); for(i = 0; i < message->numberbytes; i++) if( i > MAX)use(m->payload[i]))

if the compiler can assume the index is never greater than the array size and MAX is greater than array size, according to you it should be able to “optimize” away the check.

How ever, if the compiler can prove that the check always succeeds, then the check is useless and the programmer has written useless code and the compiler rightly elides it.

This is one of the key problems with UB. The compiler can assume there is no UB. Therefore the check is assumed unnecessary. Compilers don’t do this right now, but that’s the interpretation that is claimed to be correct. In fact, in many cases the compiler assumes that the code will not behave the way that the generated code does behave. This is nutty.

I’m not questioning whether unsafe Rust is part of Rust. I’m questioning whether aliasing pointers with different types and dereferencing them will always give defined behaviour. (And maybe it will; but it seems an odd decision to make, seeing as Rust is generally ok with unsafe code producing being able to produce undefined behaviour, and this doesn’t seem like a worthwhile behaviour to define).

The people wanting “faster startup” are also wanting “fast teardown”

Yeah, I guess I agree that they should both be fast, but if we were measuring for real, I’d measure them separately.

I’m not sure that’s a useful distinction.

If latency matters then it could be. If you’re spawning a process to handle network requests for example then the startup time affects latency but the teardown time doesn’t, unless the load gets too high.

I don’t know Qt Quick and Clutter, so I can’t comment. Definitely in the first release, JavaFX was positioned as a competitor to Silverlight and Air. JavaFX 1 was all about the XML to make it declarative and script-driven. With JavaFX 2 they seemed to back away from that positioning a bit.

As far as the expected complement of widgets, yep. They’re all there. You don’t have to deal with CSS or skinning if you don’t want to. Just instantiate the components and go. All the interactions you’d expect are there. It’s not like an OpenGL rebuild of a GUI toolkit. It’s a GUI toolkit that also makes it easy to show OpenGL.

(The docs can be a little misleading here again, because they talk about a “stage” and a “scene”, which makes it sound like a 3d engine. It’s not.)

Another thing I liked is that they extended the JavaBeans getter/setter model with active property objects. Instead of creating a custom listener interface for everything like in AWT, you can just bind to specific properties and get notified about changes to those. That gives you a narrower interface that’s used more broadly… which makes it super-easy to use from Clojure. :-)

One thing I dislike is that you’re expected to have your main class be a subclass of javafx.application.Application. They want to take control of initial class loading and application startup. There’s a goofy workaround needed if you want (like me) to have main be a Clojure function. That’s not of general interest though so I won’t clutter this post up with it.

Thanks for the correction! So it’s more like the XML layouts for Android?