Threads for gavinhoward
-
I can’t believe I wrote the Usenet post about Dylan referenced in this article 22 years ago - after all that time are there any mainstream languages using conditions/restarts?
-
Smalltalk doesn’t have exceptions as a language-level construct. Instead, blocks (closures/lambdas) can return from the scope in which the block was created (assuming it’s still on the stack). You implement exceptions by pushing a handler block onto the top of a global stack, invoking it at the ‘throw’ point, and having it return. Smalltalk ‘stacks’ are actually lists of activation records where every local is a field of the object and amenable to introspection, so you can also use it to walk up the stack and build resumable or restartable exceptions. Not really a mainstream language though.
Perhaps more interesting, the language-agnostic parts of the SEH mechanism on Windows fully support resumable and restartable exceptions. There are some comments about this in the public source in the MSVC redistributable things. To my knowledge, no language has ever actually used them. This is much easier with SEH than with Itanium-style unwinding. The Itanium ABI does a two-pass unwind, where one pass finds cleanups and catch handlers, the second pass runs them. This means that the stack is destroyed by the time that you get to the catch block: each cleanup runs in the stack frame that it’s unwinding through, so implicitly destroys anything that has been unwound through by trampling over its stack. In contrast, the SEH mechanism invokes ‘funclets’, new functions that run on top of the stack with a pointer to the stack pointer for the frame that they’re cleaning up. This means that it’s possible to quite easily shuffle the order in which they are executed and allow a far-off handler to decide, based on the exception object and any other state that it has access to, that it wants to just adjust the object and resume execution from a point in the frame that threw the exception or control the unwinding process further.
Oh, and one of the comments in the article talks about
longjmping out of a signal handler. Never do this.setjmpstores only the callee-save register state: it is a function call and so the caller is responsible for saving any caller-save state. Signals are delivered at arbitrary points, not just at call points, and so you will corrupt some subset of your register state if you do this.setcontextexists specifically to jump out of signal handlers. Similarly, you should notthrowout of signal handlers. In theory, DWARF unwind metadata can express everything that you need for this (the FreeBSD signal trampolines now have complete unwind info, so you get back into the calling frame correctly) but both LLVM and GCC assume that exceptions are thrown only at call sites and so it is very likely that the information will be incorrect for the top frame. It will usually work if the top frame doesn’t try to catch the exception, because generally spills and reloads happen in the prolog / epilog and so the unwind state for non-catching functions will correctly restore everything that the parent frame needs.-
Oh, and one of the comments in the article talks about longjmping out of a signal handler. Never do this. setjmp stores only the callee-save register state: it is a function call and so the caller is responsible for saving any caller-save state. Signals are delivered at arbitrary points, not just at call points, and so you will corrupt some subset of your register state if you do this.
I think you’re wrong here.
The POSIX standard says this about invoking
longjmp()from a signal handler:The behavior of async-signal-safe functions, as defined by this section, is as specified by POSIX.1, regardless of invocation from a signal-catching function. This is the only intended meaning of the statement that async-signal-safe functions may be used in signal-catching functions without restriction. … Note that although
longjmp()andsiglongjmp()are in the list of async-signal-safe functions, there are restrictions on subsequent behavior after the function is called from a signal-catching function. This is because the code executing afterlongjmp()orsiglongjmp()can call any unsafe functions with the same danger as calling those unsafe functions directly from the signal handler. Applications that uselongjmp()orsiglongjmp()out of signal handlers require rigorous protection in order to be portable.It also says this:
Although
longjmp()is an async-signal-safe function, if it is invoked from a signal handler which interrupted a non-async-signal-safe function or equivalent (such as the processing equivalent toexit()performed after a return from the initial call tomain()), the behavior of any subsequent call to a non-async-signal-safe function or equivalent is undefined.So it seems the extra restriction mentioned before is that
longjmp()andsiglongjmp()cannot be used from a signal handler that interrupted a non-async-signal-safe function.There are other restrictions. The standard says this about them:
If the most recent invocation of
setjmp()with the corresponding jmp_buf occurred in another thread, or if there is no such invocation, or if the function containing the invocation ofsetjmp()has terminated execution in the interim, or if the invocation ofsetjmp()was within the scope of an identifier with variably modified type and execution has left that scope in the interim, the behavior is undefined.So the function with the
setjmp()invocation must be in the same thread and still on the stack (i.e., the invoking function has not returned), and it also has to be in scope (i.e., execution must be in the same C scope as thesetjmp()invocation).In addition:
All accessible objects have values, and all other components of the abstract machine have state (for example, floating-point status flags and open files), as of the time
longjmp()was called, except that the values of objects of automatic storage duration are unspecified if they meet all the following conditions:-
They are local to the function containing the corresponding
setjmp()invocation. -
They do not have
volatile-qualified type. -
They are changed between the
setjmp()invocation andlongjmp()call.
All three of those conditions have to exist for the value of local variables to be unspecified. If you have a pointer to something in the stack, but in a function above, you’re fine; the pointer doesn’t change, and the contents at the pointer are specified. If you have a pointer to a heap allocation, the contents of the heap allocation are specified. And so on.
All of this is to say that you are broadly right, that code should usually not
longjmp()out of signal handlers. But to say never seems a bit much, if you do everything right. Of course, like crypto, you should only do it if you know what you’re doing.Also, I believe that the quotes above mean that if an implementation does not save caller-save registers, it is wrong. The reason is that the compiler could have lifted one local into a caller-save register. If that local does not change between the initial
setjmp()and thelongjmp()back to it, that caller-save register should have the same value, and if it doesn’t, I argue that the implementation does not follow the standard, that the implementation is wrong, not the application, especially since it is legal for an implementation to use a macro. In fact, the standard has several restrictions on howsetjmp()can be invoked to make it easier to implement as a macro:An application shall ensure that an invocation of setjmp() appears in one of the following contexts only:
-
The entire controlling expression of a selection or iteration statement
-
One operand of a relational or equality operator with the other operand an integral constant expression, with the resulting expression being the entire controlling expression of a selection or iteration statement
-
The operand of a unary ‘!’ operator with the resulting expression being the entire controlling expression of a selection or iteration
-
The entire expression of an expression statement (possibly cast to void)
If the invocation appears in any other context, the behavior is undefined.
Source: I implemented a
longjmp()out of a signal handler in mybcand followed all of the relevant standard restrictions I quoted above. It was hard, yes, and I had to have signal “locks” for the signal handler to tell if execution was in a non-async-signal-safe function (in which case, it sets a flag, and returns normally, and when the signal lock is removed, the jump happens then).-
Also, I believe that the quotes above mean that if an implementation does not save caller-save registers, it is wrong.
If that is the case, then there are no correct implementations. For example, the glibc version saves only 8 registers on x86-64. Similarly, the FreeBSD version stores 8 integer registers. This means that they are not storing all of the integer register set, let alone any floating-point or vector state. This is in contrast to
ucontext, which does store the entire state. Thesigprefixed versions also store the signal mask and restore it.It looks as if
ucontextis finally gone from the latest version of POSIX, but the previous version said this:When a signal handler is executed, the current user context is saved and a new context is created. If the thread leaves the signal handler via longjmp(), then it is unspecified whether the context at the time of the corresponding setjmp() call is restored and thus whether future calls to getcontext() provide an accurate representation of the current context, since the context restored by longjmp() does not necessarily contain all the information that setcontext() requires. Signal handlers should use siglongjmp() or setcontext() instead.
Explicitly:
longjmpdoes not store the entire context, any temporary registers may not be stored.Also, I believe that the quotes above mean that if an implementation does not save caller-save registers, it is wrong. The reason is that the compiler could have lifted one local into a caller-save register. If that local does not change between the initial setjmp() and the longjmp() back to it, that caller-save register should have the same value, and if it doesn’t, I argue that the implementation does not follow the standard, that the implementation is wrong, not the application, especially since it is legal for an implementation to use a macro.
No, this is why
setjmpexplicitly requires that all local mutable state that you access after the return is held involatilevariables: to force the compiler to explicitly save them to the stack (or elsewhere) and reload them. If they are local, not volatile, and have changed, then accessing them is UB. Any subset of these is fine:- If they are not volatile and are local, but have not changed, then they must be stored either on the stack or in a callee-save register because otherwise the function call (for
setjmp) may clobber them and so that’s fine. - If they are local and have not changed, but are not volatile, then they will be stored in the same location (register or stack slot) in the use point as before the
setjmpcall (which must be something that’s preserved across calls). - If they are not local, then any local modifications must have been written back to the global before
setjmpand any access aftersetjmpreturns must reload.
Source: I implemented a longjmp() out of a signal handler in my bc and followed all of the relevant standard restrictions I quoted above. It was hard, yes, and I had to have signal “locks” for the signal handler to tell if execution was in a non-async-signal-safe function (in which case, it sets a flag, and returns normally, and when the signal lock is removed, the jump happens then).
I hope that you mean
siglongjmp, notlongjmp, because otherwise your signal mask will be left in an undefined state (in particular, because you’re not resuming viasigreturn, the signal that you’re currently handling will be masked, which may cause very surprising behaviour and incorrect results if you rely on the signals for correctness). It’s not UB, it’s well specified, it’s just impossible to reason about in the general case.Note that your signal-lock approach works only if you have an exhaustive list of async signal unsafe functions. This means that it cannot work in a program that links libraries other than libc. Jumping out of signal handlers and doing stack unwinding also requires that you don’t use
pthread_cleanup, GCC’s__attribute__((cleanup)), or C++ RAII in your program. I honestly can’t conceive of a situation where I’d consider that a good tradeoff.-
EDIT: I am dumb. I realized that under the ABI’s currently in use, treating caller save registers normally would actually be correct. So there’s no special code, and my
bcwill work correctly under all compilers. I’m leaving this comment up as a lesson to me and for posterity.I hope that you mean siglongjmp, not longjmp, because otherwise your signal mask will be left in an undefined state (in particular, because you’re not resuming via sigreturn, the signal that you’re currently handling will be masked, which may cause very surprising behaviour and incorrect results if you rely on the signals for correctness). It’s not UB, it’s well specified, it’s just impossible to reason about in the general case.
Yes, I used
siglongjmp(). I apologize if this was wrong, but in your original post, it sounded like you thoughtsiglongjmp()should not be used either, so I lumpedlongjmp()andsiglongjmp()together.It looks as if ucontext is finally gone from the latest version of POSIX, but the previous version said this:
When a signal handler is executed, the current user context is saved and a new context is created. If the thread leaves the signal handler via longjmp(), then it is unspecified whether the context at the time of the corresponding setjmp() call is restored and thus whether future calls to getcontext() provide an accurate representation of the current context, since the context restored by longjmp() does not necessarily contain all the information that setcontext() requires. Signal handlers should use siglongjmp() or setcontext() instead.
Explicitly: longjmp does not store the entire context, any temporary registers may not be stored.
Yes, that looks like
longjmp()does not have to restore the full context according to POSIX. (C99 might still require it, and C99 controls; more on that later.) However, it looks likesiglongjmp()does. That paragraph mentions them separately, and it explicitly says thatsiglongjmp()should be used because of the restrictions on the use oflongjmp(), which implies thatsiglongjmp()does not have those restrictions. And ifsiglongjmp()does not have those restrictions, then it should restore all of the register context.I agree that it does not have to set floating-point context or reset open files, etc., since those are explicitly called out. But it certainly seems to me like
siglongjmp()is expected to restore them.No, this is why setjmp explicitly requires that all local mutable state that you access after the return is held in volatile variables: to force the compiler to explicitly save them to the stack (or elsewhere) and reload them.
This sounds completely wrong, so I just went through POSIX and checked: those is no such restriction on the compiler, specifically the
c99utility, nor anywhere else it mentions compilers. I also checked the C99 standard, and it says the exact same thing as POSIX.I also searched the C99 standard to see if there were restrictions on handling of objects of automatic storage duration, and there were none.
So there could be, and are, several C99 compilers that follow the standard, such as tcc, cproc, icc, chibicc, and others, and yet, if someone compiles my
bcwith one of those compilers on FreeBSD, mybccould fail to work properly.But even with Clang, does it either treat
setjmp()orsigsetjmp()specially or refuse to promote local variables to caller-save registers? Can you point me to the code in Clang that does either one?Is a failure in my
bcthe fault of the compiler? I argue that the C standard does not have any restriction that puts the compiler at fault.Is a failure in my
bcmy fault? If I had usedlongjmp()instead ofsiglongjmp()in the signal handler, yes, it would be my fault. But I did not, and I followed all of the other restrictions on applications, including not relying on any floating-point state.Thus, I argue that since neither C99 nor POSIX has a restriction on compilers that require special handling of
setjmp(), and since POSIX has no restrictions on applications that I did not follow, neither I nor compilers can be at fault.In addition, because POSIX says about
setjmp():The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2017 defers to the ISO C standard.
This means the C99 standard controls, and the C99 standard says:
The environment of a call to the setjmp macro consists of information sufficient for a call to the longjmp function to return execution to the correct block and invocation of that block, were it called recursively.
The line “information sufficient” is crucial; I argue that it is a restriction on the implementation, not the compiler or the application.
Now, we can argue what “sufficient information” means, and I think it might be different to every platform (well, ABI), but if FreeBSD and Linux want to obey the POSIX standard, I think each of them needs to have a conversation about what
sigsetjmp()andsetjmp()should save. Looking into this for this discussion has made me realize just how little platforms have actually considered what those require.So yes, I argue that glibc and FreeBSD are wrong with respect to the POSIX standard. They might decide they are right for what they want.
Now, why has my
bcworked so well on those platforms? Well, the standard compilers might be helping, but it might also be because mybchas been lucky to this point so far. I don’t like that thought.So let’s have that conversation, especially on FreeBSD, especially 13.1 just hit, and my
bchas begun to be used a lot more. I want mybcto work right, and I’m sure FreeBSD does too.Note that your signal-lock approach works only if you have an exhaustive list of async signal unsafe functions. This means that it cannot work in a program that links libraries other than libc.
This is correct. I do want to have zero dependencies other than libc, so it wasn’t a problem.
Jumping out of signal handlers and doing stack unwinding also requires that you don’t use pthread_cleanup, GCC’s
__attribute__((cleanup)), or C++ RAII in your program. I honestly can’t conceive of a situation where I’d consider that a good tradeoff.Because of portability. If I had used any of those, my
bcwould not be nearly as portable as it is. Making it work on Windows (with the exception of command-line history) was simple. That would not have been the case if I had used any of those, except C++ RAII. But you already know my opinion about C++, so that wasn’t an option either.But if you are unsatisfied with that answer, then I have another: in the project where I was thinking about implementing a
malloc(), I implemented a heap-based stack with the abiilty to call destructors. With a little clever use of macros, I can apply it to every function in my code, and it is 100% portable! And it also does stack traces!So I did give myself something like C++ RAII in C.
EDIT: I just want to add that this heap-based stack also made it possible for me to implement conditions and restarts in C, to put things back on topic.
-
No, this is why setjmp explicitly requires that all local mutable state that you access after the return is held in volatile variables: to force the compiler to explicitly save them to the stack (or elsewhere) and reload them.
This sounds completely wrong, so I just went through POSIX and checked: those is no such restriction on the compiler, specifically the c99 utility, nor anywhere else it mentions compilers. I also checked the C99 standard, and it says the exact same thing as POSIX.
It’s implicit from the definition of
volatile. The compiler must not elide memory accesses that are present in the C abstract machine when accessingvolatilevariables. If you read avolatile intthen the compiler must emit anint-sized load. It therefore follows that, if you read avolatilevariable, call a function, modify the variable, and then read the variable again, the compiler must emit a load, a call, a store, and a load. The C spec has a lot of verbiage about this.But even with Clang, does it either treat setjmp() or sigsetjmp() specially or refuse to promote local variables to caller-save registers? Can you point me to the code in Clang that does either one?
Clang does have a little bit of special handling for
setjmp, but the same handling that it has forvforkand similar returns-twice functions. It doesn’t need anything else, because (from the bit of POSIX that I referred to, which mirrors similar text in ISO C) if you modify a non-volatilelocal in between the first return and the second and then try to access it the results are UB. Specifically, POSIX says:All accessible objects have values, and all other components of the abstract machine have state (for example, floating-point status flags and open files), as of the time longjmp() was called, except that the values of objects of automatic storage duration are unspecified if they meet all the following conditions:
-
They are local to the function containing the corresponding setjmp() invocation.
-
They do not have volatile-qualified type.
-
They are changed between the setjmp() invocation and longjmp() call.
This is confusingly written with a negation of a universally quantified thing, rather than an existentially quantified thing, but applying De Morgan’s laws, we can translate it into a more human-readable form: It is undefined behaviour if you access any local variable that is volatile qualified and is modified between the
setjmpandlongjmpcalls.As I said, this works because of other requirements. The compiler must assume (unless it can prove otherwise via escape / alias analysis. The fact that setjmp is either a compiler builtin or an assembly sequence blocks this analysis):
- Any modification to a global or heap object is visible to code executing in a function call and so must store any changes to such objects back to memory.
- Any function call may modify any global or heap object and so it must reload any values that it reads from such locations.
- Loads and stores of
volatilevariables, including locals, must not be elided, and so any modification of avolatilelocal must preserve the write to the stack. - Local variables must be stored in a location that is not clobbered by a call. If a local variable (even a non-
volatileone) is written before thesetjmpcall, then it must be stored either on the stack, or in a callee-save register, or it will be clobbered by the call.
These are not special rules for
setjmp, they fall out of the C abstract machine and are requirements on any ABI that implements that abstract machine.Note: I’m using call a shorthand here. You cannot implement
setjmpin C because it needs to do some things that are permitted within the C abstract machine only bysetjmpitself. The standard allows it to be a macro so that it can be implemented in inline assembly, rather than as a call to an assembly routine. In this case, the inline assembly will mark all of the registers that are not preserved as clobbered. Inline assembly is non-standard and it’s up to the C implementer to use whatever non-standard extensions they want (or external assembly routines) to implement things like this. On Itanium,setjmpwas actually implemented on top of libunwind, with the jump buffer just storing a token that told the unwinder where to jump (this meant that it ran all cleanups on the way up the stack, which was a very nice side effect, and madelongjmpsafe in exception-safe C++ code as well).Oh, and there’s a really stupid reason why
setjmpis often a macro: the standard defines it to take the argument by value. In C, if it really is a function, you need something like#define setjmp(env) real_setjmp(&env). GCC’s__builtin_setjmpactually takes ajmp_buf&as an argument and so C++ reference types end up leaking very slightly into C. Yay.Note that there probably are some compiler transforms that make even this a bit dubious. The compiler may not elide loads or stores of
volatilevalues, nor reorder them with respect to other accesses to the same value, but it is free to reorder them with respect to other operations._Atomicintroduces restrictions with respect to other accesses, so you might actually need_Atomic(volatile int)for anintthat’s guaranteed to have the expected value on return from asiglongjmpout of a signal handler. This does not contradict what the quoted section says: the value that it will have without the_Atomicspecifier is still well-defined, it’s just defined to be one of a set of possible values in the state space defined by the C abstract machine (and, in some cases, that’s what you actually want).This is particularly true for jumping into the top stack frame. Consider:
jmp_buf env; volatile int x = 0; volatile int y = 0; volatile int z = 1; if (sigsetjmp(env) == 0) { int y; x = 1; y = z / y; // SIGFPE delivered here, signal handler calls `siglongjmp`. } else { printf("%d\n", x); }This is permitted to print
0. The compiler is free to reorder the code in the block to:z_0 = load z; y_0 = load y; y_1 = z_0 / y_0; // SIGFPE happens here store y y_1 store x 1If you used
_Atomic(volatile int)instead ofvolatile intfor the three locals then (I think) this would not be permitted because each of these would be a sequentially-consistent operation and so store toxmay not be reordered with respect to the loads ofzandy. I believe you can also fix it by making each of the locals in a singlevolatilestruct, because then the rule that prevents reordering memory accesses to the samevolatileobject would kick in.Note that this example will, I believe, do what you expect when compiled with clang at the moment because LLVM is fairly conservative with respect to reordering
volatileaccesses. There are some FIXMEs in the code about this because the conservatism largely dates from a pre-C[++]11 world when_Atomicdidn’t exist and so people had to fudge it withvolatileand pray.In spite of the fact that it is, technically, possible to write correct code that jumps out of a signal handler, my opinion stands. The three hardest parts of the C specification to understand are
[sig]setjmp, signals, andvolatile. Anything that requires a programmer to understand and use all of these to implement correctly is going to be unmaintainable code. Even if you are smart enough to get it right, the next person to try to modify the code might be someone like me, and I’m definitely not. -
-
- If they are not volatile and are local, but have not changed, then they must be stored either on the stack or in a callee-save register because otherwise the function call (for
-
longjmp() and siglongjmp() cannot be used from a signal handler that interrupted a non-async-signal-safe function.
Okay sure the text says that.
But that requirement, of knowing what function you have interrupted, is ludicrously hard to arrange except when the signal handler is installed in the same function it is triggered from. Certainly entirely unsuitable for a generic mechanism that might wrap arbitrary user code.
-
-
-
-
Yep, added in 0.5
https://doc.rust-lang.org/0.5/core/condition.html https://github.com/rust-lang/rust/blob/master/RELEASES.md#version-05-2012-12-21
Can’t find a good code example though.
-
See also the issue where they were removed – that issue & the see also links show why it didn’t work for Rust:
-
-
-
-
-
Copied from my post on the orange site.
I’m glad that the terminal 80-character limit was mentioned and that humans struggle with longer lines. In fact, the optimal line length is 50-75 characters 1, which is just right for 80 characters with 1+ levels of indentation.
That last one is the reason why I still keep a hard 80-character limit on my own code, even though I have a 4K monitor, and a big one too (43 inches).
But doing so actually has a great side effoct: keeping an 80-character limit with a 4K monitor means I can have four files open side-by-side. 2 Or a file open in four different places. Or a file and its associated header both open to two different places.
It also means that I can see the code completely while debugging with four columns. 3
-
Hard-wrapping at 80 characters means that those who prefer less than 80 characters are stuck with the formatting that you imposed upon them, and those that prefer more than 80 characters are also stuck with the formatting that you imposed upon them.
Furthermore, if I need a larger font size than you do, then suddenly I can’t fit four of your 80-character hard-wrapped files onto my screen anymore, because if I turn off soft-wrap then I have to scroll, and if I turn on soft-wrap the results are unreadable for blocks of consecutive hard-wrapped lines.
Respect people’s editor preferences. Don’t hard-wrap your code. Soft-wrap has existed for decades.
-
Soft-wrapping completely ignores indentation. That’s not a workable solution either, and it will continue to be until editors can not only soft-wrap, but soft-wrap and place the extra code at the same indentation that the line has.
It’s this:
stuff = more_stuff + way_more_stuff - you_know_who_is_going_to_get_you * there_is_a_sad_sort_of_clanging * (he_who_must_not_be_named / infinity_and_beyond + wont_you_be_my_neighbor * star_wars_vs_star_trek)(pretending that the hard wrap I put in was actually a soft-wrap because lobste.rs creates a scroll bar) versus
stuff = more_stuff + way_more_stuff - you_know_who_is_going_to_get_you * there_is_a_sad_sort_of_clanging * (he_who_must_not_be_named / infinity_and_beyond + wont_you_be_my_neighbor * star_wars_vs_star_trek)Until that is possible, hard-wrap with a code formatter that can do it for you in the editor with a hotkey is better than soft-wrap. You mention in a comment in this thread that this is possible, and that it doesn’t exist because people haven’t done it. Have you considered that most people don’t care?
If you want soft-wrapping everywhere, might I suggest writing your own editor, or language, or compiler that does what you want? In other words, show me the code. And the existence of legally blind people who can’t use soft-wrapping means that your claim in the thread that “then everyone benefits from that point onward” is patently false.
Also, note that I said that I do it for my own code, where other people don’t matter because they are not even allowed to contribute, so I’m not “imposing” it on anyone. If you require me to soft-wrap, you are imposing on me.
Let me have my opinion in my own code.
Also, I note above a comment you made asking why a legally blind programmer could not use soft-wrap. Might I just suggest that you are being a little too evangelical about soft-wrapping if you’re:
- Complaining that I impose my style on other people in my own code that others are not allowed to contribute to, and
- Asking for justification why someone who struggles with sight might not like soft-wrapping?
But to answer your question about why a legally blind person would not want soft-wrapping, perhaps their effective vision cone is a really small angle such that they cannot see the line number, or its non-existence, at the beginning of a line, and so they use indentation as an easy way to see if a particular line is a continuation of a line above or not. And soft-wrapping would blow that away.
In short, your complaining about me not respecting other people’s editor preferences in my own project and questioning the requirements of partially disabled people is not a good look. It’s too evangelical, and ironically, it does not respect other people’s preferences.
Soft-wrapping is not the end-all be-all. Far from it. In fact, from the comments here, I would venture to say that it is not the preference for the majority of people.
Please accept our preferences and not try to change them.
In fact, my original comment only expressed how I felt and how I work. I did not say anyone else was wrong in their opinion of wanting longer lines or soft-wrapping; I merely talked about what I prefer and why.
You did not accept my preferences, as though you believe hard-wrapping is bad for programming in general. Might I respectfully suggest that if it’s what works for the majority of programmers, including legally blind programmers, that you are wrong?
I wouldn’t say or imply that soft-wrapping is bad for programming in general; why do you imply that hard-wrapping is? It makes no sense, and you sound like an obsessed evangelist to me.
-
-
-
Note: This is copied from my comment on the orange site.
I appreciate this article bacause this is an important distinction to make. In fact, it is so important that I am willing to rewrite code in order to know the names and contact information of all of the people that my dependencies depend on, as well as having some sort of professional relationship with them.
For example, in a project I am working on, I need a database, a way to talk over the Internet, and cryptography. Obviously, I know what database to use: SQLite (D. Richard Hipp). Obviously, I know what dependency to use for talking over the Internet: curl (Daniel Stenberg).
Cryptography is harder, but I finally settled on BearSSL (Thomas Pornin). BearSSL does not give me everything I want, however; since I want OPAQUE (a way for clients to not give their password to a potentially malicious server), I need that. BearSSL also does not give me a “KSF,” or key-stretching function, which OPAQUE requires, though I can use Argon2i for that.
The reference implementation for Argon2i unfortunately seems dead, even though I know the names of the people who made it. I don’t know if they will respond if I contact them, while I do know that D. Richard Hipp, Daniel Stenberg, and Thomas Pornin will respond. So in order to make sure I always have a point of contact with all of my dependencies, I am going to write Argon2i, BLAKE2b (needed by Argon2i), and OPAQUE myself.
Bad idea? Yeah, don’t roll your own crypto. But I am studying hard, and I intend to get my crypto audited before publishing.
The end result, however, is very worth it: my dependencies will be well-known, and I know each of the authors personally, albeit through email.
And down the road, if I manage to make some money, I can kick some of it back to them in exchange for their previous help. In turn, they’ll be happy to continue the relationship.
That’s how Open Source works at its best: it depends on relationships, and on giving back to those relationships. I think that that is what this article is trying to say, and I whole-heartedly agree.
-
What you’ve written resonates with me, but it seems to push a lot of people to the side. People with families may not have the time to learn the algorithmic details of say, BLAKE2d. Others with learning difficulties or large knowledge gaps in the thing they want to recreate may never be able to achieve an implementation in a reasonable amount of time.
And then there’s the other side: maybe these FOSS authors really don’t want to know you. Maybe the truth is, they write software because they like it, feel a sense of real achievement due to being helpful to humanity, but want nothing more. The semi-recent events of authors being ridiculed for “lack of support” is a similar example of “don’t bother me”.
Trust though is huge and that’s why we have digital signatures for software packages, source code tarballs, and whatever else we want to maybe trust. I think the motto “don’t trust, verify” is heavily underrated, and should be applied more diligently to our dependencies. I believe this tool is helping with that.
The context will determine the degree of trust you require.
-
You bring up good points. And yet…
As it so happens, I have a wife. I have very little time, and I have knowledge gaps in cryptography. I am not going to be able to do this in a reasonable amount of time. Yet I am still doing it because I feel strongly about doing my software right. Not many people want to. I do want to, and I am betting that once I do publish something, people would want to use it.
As I mentioned on the orange site, I already have an email relationship with two of the three programmers. All three of them make their money off of the particular software I am going to use as dependencies (well, one makes their money off of consulting in the same area as the software he made), so if all else fails, I can throw money at them in the same way that others already do.
That’s why my three requirements for dependencies included some way to pay for something. First, as TFA says, that’s important for Open Source, and second, it will probably be important to my business later.
And as for trust, well, if I have paid them, part of the product I will pay them for is to have them declare that their software is fit for my purposes. (Basically, to nullify the “no warranty” clause of licenses, so I would have some sort of warranty.) While that doesn’t take care of the problem, there are laws that govern the use of warranties, so I would have some recourse. And when they take money from me, they would probably have more incentive to keep me happy, i.e., they would have incentive to be trustworthy.
Still not enough for complete trust, but hey, it’s only three dependencies. That makes it easier to audit them personally.
-
Yet I am still doing it because I feel strongly about doing my software right.
Devil’s advocate: And so any other way is doing software wrong?
I already have an email relationship with two of the three programmers.
So in this particular case it’s possible / they accept relationships, but this isn’t true of all scenarios…
The trust part yeah, there are a lot of ways to approach it. I wasn’t dismissing anything you said, just augmenting it with my own comment 🙂
Thank you for the insightful conversation! Makes you think.
-
Devil’s advocate: And so any other way is doing software wrong?
If you’re paid for it, yes. If not, no. But I do want to be paid for mine eventually. Like the three fellows I will be depending on.
So in this particular case it’s possible / they accept relationships, but this isn’t true of all scenarios…
Correct. I had to do the leg work first before deciding on them. And if they decide to cut me off, I’m going to have to replace their software with something else or write it myself.
Thank you for the insightful conversation! Makes you think.
You’re welcome. And thank you. :)
-
If you’re paid for it, yes
This just in: getting paid is bad. I think respecting licenses and creating software off the works of others is how we advance as a society (and indeed, is how countless other creative pursuits have evolved). Purists in this respect don’t usually get very far is my cynical take here.
-
-
-
-
-
-
-
-
What does evil mean in this case? To me, it means a EULA that:
- gives more advantage to the software creator/seller than the customer, when it should be equal, and/or
- absolves the creator/seller of responsibility.
With that in mind, I think Mr. Mitchell is right in general.
In practice, how many EULA’s are not evil? I think Mr. Mitchell has probably seen a few, probably written a few. I think that may be because his work is probably mostly between two business customers.
But as an individual software consumer, I have never seen such a EULA. I’ve only seen EULA’s that are “evil” (which I define as a EULA that gives more advantage to the software creator than the customer, when it should be equal), at least in the software that I use.
That is why I use Open Source software as much as possible: that extreme is better than the other, of being locked down by EULA’s that advantage the creator over me or absolve the creator of responsibility.
This is in addition to a few of the other problems mentioned by comments on the orange site.
So in theory, practice and theory are no different; in practice, they are.
But it would be a good world if Mr. Mitchell was right in practice, as long as customers always had access to source code and the ability to modify it, and the creator/seller accepted responsibility for it.
-
It’s also a rare case of arguing for a strawman. ;)
The customer can fix bugs and make improvements. Better yet, while they pay, they can foist this off on the seller. What the customer could usually only hope for in the best case with open source—a timely response from a responsible person—they now get by right, under contract.
The problem is that for all practical purposes such an EULA doesn’t exist. No software developer can be expected to do any work without an incentive to do that. If there’s a problem the vendor isn’t personally having and leaving it unfixed does not lead to significant user base shrinking, they will not do it, and who can blame them?
EULAs with negative financial incentives for the vendor to fix bugs/add features or solve your problems kinda do exist, but they are out of reach even for average business users. There are also many ways to game those incentives by sticking to their letter without keeping their spirit, e.g. giving a reply within the SLA timeframe but not supplying any actually useful information there. And then even the most well-intentioned support providers should be wary of any guarantees to solve the problem, because anyone can take you to court and claim you didn’t solve their problem. That’s not counting cases when a problem really can be solved.
If people can share modified versions, there may be hope that someone is having the same problem as you but better programming skills and will make a patch. In a “use, but don’t distribute” model there’s no way for anyone to share it, so even if someone fixes it, you will not benefit from it.
P.S. For the record, I do make money from open-source software with a “pay for precompiled binaries and support” model, and we do have a EULA covering the binaries with proprietary and trademarked artwork embedded in them (exactly like RedHat and Mozilla do). But we don’t give any guarantees of solving user’s problem because it would be false advertising. We don’t game SLA response time guarantees that we give, because we all had multiple proprietary vendors play that trick on us, and we hate it.
-
The problem is that for all practical purposes such an EULA doesn’t exist. No software developer can be expected to do any work without an incentive to do that.
Licenses like that get signed all the time.
The vendor’s support commitment requires them to respond timely to requests. Where those requests identify bugs, the result is a fix, which the vendor’s maintenance commitment requires it to provide. If the vendor falls down on its support commitment, the customer gets credits against its fees, and possibly an out from the contract. The vendor’s incentivized to avoid that.
Blank-check commitments for new features aren’t so common, for obvious reasons. I have seen many deals where particular roadmap points got included in the terms. But even without a hard commitment to develop new features, support request → roadmap item → minor release stories are very common. Features existing customers ask for can be features potential customers want.
-
Maybe if you pay hundreds thousands dollars… Haven’t seen anything like that from proprietary vendors in the thousands or tens of thousands dollar range.
I have seen bugs in very expensive software go unfixed for months and years just because they weren’t affecting the majority of users, and it wasn’t a real threat to the vendor. I did see many prompt fixes, mind you, but my point is that in the vast majority of cases there’s no guarantee that your bug will be fixed, no matter how critical is it to you.
-
I have definitely seen good support SLAs in that range of ARR. And happy customers who never had to send a ticket!
I hear you on languishing bugs in pricey software. It happens. Frustratingly, sometimes it’s just efficient.
I have also seen side deals where customers ponied up extra to get a bug squashed or a feature added, even one arguably covered by their existing support-maintenance deal. The remedies under that deal weren’t enough to move the vendor, and the loss from the customer walking came in under the cost of the fix. Another day at the office.
I think we’d probably be better off in a wold with more source available deals like the one I outlined in the blog post: fix it yourself if you want to, just send the patch back to the vendor. But often enough, the vendor’s still the one who can get the work done with lowest cost, especially when the customer isn’t a “software company”. The vendor has the people who know the code.
-
-
-
-
In practice, how many EULA’s are not evil? I think Mr. Mitchell has probably seen a few, probably written a few. I think that may be because his work is probably mostly between two business customers.
But as an individual software consumer, I have never seen such a EULA. …
I really appreciate this comment.
I was 50-50 on the second part of the blog post, after the
<hr>. The bit where I speculate on why this black-white view persists. I knew my speculation’s informed, but speculation’s always a jump. I worried it would distract or detract more than it added. I’m glad I went with it now.The HN discussion already has two or three threads where I see someone with software deals experience butting heads with someone who only sees take-it-or-leave-it terms. Here you’ve hit the same insight with introspection, without all the noise and squabble. It’s nice to be here on lobste.rs ;-D
The relevant bit of my post was here:
[The black-and-white view of licensing] reinforced a stark, limited view of the industry—characteristic of young and commercially sheltered coders dealing entirely with dominant apps from megacorps, on one hand, and open source from hackers, on the other—as all relevant reality. There wasn’t any burgeoning gray area of independent software sales in this picture…. No experience of negotiation—engineering terms to circumstances—to speak of. No rallying call to develop those chops.
When a vendor has you over a barrel—market power, exclusive deal—the terms they deign give are more than likely bad news. Same with the software, for that matter. Microsoft doesn’t need to give us source code to keep Word in pole position. If a hardware vendor controls their tech, the dev kit can be lousy.
As solo devs cruising the web for code, we have no leverage. The cost of just discussing terms with us would eat all our value as potential customers. So terms are take-it-or-leave-it.
The reason I’m on about this to solo devs and indie firms is that I think we could get a lot more out of ourselves and each other if we accepted that deals engineering is an art we can get good at that could unlock a ton of latent potential. Some of this is business basics: many firms now offer multiple take-it-or-leave-it deals, at different “tiers”, with a features grid. As programmers, we have the skills to do that and so much more. And we have the skills to automate it—software configuration, legal terms, payment, the whole shebang.
-
I’m the author of the article that Mr. Mitchell is responding to.
tl;dr: I think Mr. Mitchell has good points, but I also feel like he did not address the parts of my post that actually discouraged me.
I didn’t expect to have anyone respond with a blog post like this. That said, Mr. Mitchell has some good points.
First, he is right that my original article is too absolutist. I will own that mistake. Please remember, however, that the post was written in the middle of a depressive episode, when I was feeling more discouraged than I had been in a long time. I wasn’t thinking completely straight.
Second, he makes a good point about trying hard to serve people, but what he missed is that I had done that. In the types of software I want to write, I could see very well that closed source was not going to fly with my intended audience. @alexandria said (in this thread):
People will only pay for closed source software if they can’t already acquire a ‘good enough’ alternative for free.
As far as build systems and version control systems go (the top two projects I am working on), there are plenty of ‘good enough’ alternatives that are Open Source, so developers only want Open Source, for the most part. Of course, there are companies that will use closed source, if it is better, but individual developers, by and large, don’t, especially because they often don’t need the features that make the closed source ones better.
The reason I am targeting individual developers is because they are the most likely to be “early adopters” of new technology. I was hoping that, after getting enough of them as users, they would convince their employers to start using my software.
Third, and this follows from the previous point, Mr. Mitchell has himself said that, over time, software moves from closed source to Open Source, then to public domain. I had read that post of his before posting mine, and I understood what he was saying. I guess what I did not say well is that, for the software I was planning to write, the transition from closed source to Open Source has already happened, which means that going closed source would not work.
Fourth, I don’t feel like Mr. Mitchell addressed the points from my original post that actually made me discouraged: the appropriation of Open Source by companies who fail to give back or outright steal (by violating FOSS licenses). Those things are the reason I was discouraged because, even if I do get individual programmers to use my software, then they get their employers to use it, what will prevent their employers from just ripping me off and violating the license I put on my code?
The fact that developers will use closed source in some cases, as Mr. Mitchell says, does nothing to address these problems.
With that said, I don’t regret putting that post out and submitting it here and to the orange site, and that’s because my “call to action” at the bottom, asking people to contact me if they knew things that could encourage me, worked. A lot of people emailed me with encouragement, and that eventually brought me out of my funk.
One person in particular, who I won’t name in case he does not want to be named, wrote to me about an article he wrote a long time ago about how it’s possible to make a living selling closed source software that is only closed source temporarily before being open sourced after a certain amount of time. He helped me see why that works, and he also helped me figure out a method for not violating my ethics in doing so.
(My ethics include always providing source code to my users, like an architect should provide blueprints to the owner of the building they designed. What this person helped me realize is that even if it’s closed source, I can provide source code to my customers with a license that prevents them from redistributing it. Yes, this is an argument for copyright still applying to software.)
That, along with Mr. Mitchell’s assertion that all software eventually moves toward the public domain, helped me form a plan.
First, I’m going to get the software ready, of course. But when it’s ready, I’m going to release it under the terms of the two most radioactive licenses possible: the SSPL and the AGPL, and users will have to comply with the terms of both. This shouldn’t matter for individual developers.
However, it will matter to companies, so next, I will do my best to make sure my users know that they can ask me to relicense more permissively once they are asking their employers to use my software. When they ask, I will.
I will also develop closed source add-ons that companies can use, and these add-ons will be open sourced after a certain period.
In essence, my software will follow the transition from closed source to copyleft to permissive licensing that Mr. Mitchell described; it’s just that instead of more permissively-licensed competitors rising up and out-competing me, I’ll relicense my stuff to prevent the need for competitors to do so.
That does beg the question of why I wouldn’t start permissively licensed in the first place, and the answer is to make my software radioactive enough that at least some companies won’t touch it at the beginning. It’s a game of numbers because I don’t have to prevent all companies from ripping me off, just enough of them. And after my software grows more important (if that ever happens), then I suspect that companies would be less likely to rip me off, even if the software is permissively licensed.
It’s funny, but Mr. Mitchell did help me, in a way, with his post about the lifecycle of software licenses.
-
Forgive me for catching up with you here! I’d made a note to send you an e-mail, properly, which I try to do whenever I blog a response to someone else’s post. But Saturday caught up with me. I got the post out, and didn’t look at my to-dos again until this morning.
Note to self: Publish the post, then send the e-mail. Don’t put it off!
I hear you on discouragement and depression. Man is that real, and I’m inspired by how honest and open you are about it. I wish I’d thought more about where you may have been mentally as I wrote, and done more to emphasize a helpful rather than corrective tone. If I’d caught you still in it, and come across too harsh—easy to read that way when you’re down, don’t I know it!—I could have done you wrong. I’m happy to read that my post found you standing firmer on your feet. I just got lucky there. Also that others stepped up with so much encouragement. A little faith in community restored.
As for company and user misbehavior: oh yeah, that’s real. And I’m really fucking tired of it. And I’m probably doing disservice by taking it as a given, whenever I write. By focusing just on the jump from frustration to resignation, without honoring the frustration to begin with, my post falls short of a complete picture. Your notes there are very well taken.
On licensing, I’d encourage you to consider a license choice that more clearly expresses your intentions. A mad-science hybrid of AGPL and SSPL will definitely freak people out. But if what you really want to say is “businesses needs to talk to me about a deal”, you might find that better expressed through a noncommercial license like PolyForm Noncommercial or Prosperity, which also allows free trials. More experimentally, you might find Big Time interesting.
Whichever way you go, good luck!
PS: No need for “Mr. Mitchell”, unless you prefer that way. Kyle, or kemitchell, do me fine. And kyle@kemitchell.com anytime.
-
First, I’m going to get the software ready, of course. But when it’s ready, I’m going to release it under the terms of the two most radioactive licenses possible: the SSPL and the AGPL, and users will have to comply with the terms of both. This shouldn’t matter for individual developers.
However, it will matter to companies, so next, I will do my best to make sure my users know that they can ask me to relicense more permissively once they are asking their employers to use my software. When they ask, I will.
That depends a lot on the company. I’d have to check our policy but I believe it means that we could use it, we could maintain our internal fork, but we’d need to jump through approval hoops to contribute anything back. Licenses like the AGPL are problematic if we want to incorporate them into a product or ship them to customers (or, in general, do anything that involves the code leaving the company) but they are fine for use.
The critical thing for a company (which I’d assume @kemitchell knows, since he is a corporate lawyer and this is literally his day job) is minimising risk. The license is one aspect of this. Dual licensing doesn’t really help here because it lets you choose between risks (the risk the supplier will go away for the proprietary license versus the risks associated with a less permissive license). If your dual license allows people to pay for a more permissive license (e.g. MIT) then you now have a risk that someone will distribute the version that they receive.
For a single developer, the largest risk that a company is likely to worry about is the bus factor. If you get hit by a bus, what happens to the software? That’s a massive risk for anything that’s going to be a core part of the work flow. There’s a big difference between buying a proprietary product from a big company and a proprietary product from some guy, especially if it’s a product with a lot of users and that is bringing in a lot of revenue.
Open vs closed is one of the less important concerns within an overall risk discussion for most companies.
-
That depends a lot on the company. I’d have to check our policy but I believe it means that we could use it, we could maintain our internal fork, but we’d need to jump through approval hoops to contribute anything back. Licenses like the AGPL are problematic if we want to incorporate them into a product or ship them to customers (or, in general, do anything that involves the code leaving the company) but they are fine for use.
That actually sounds perfect, to be honest, including not giving back code. I’m interested in companies contributing back in general, but for my own purposes, I’d rather not incorporate code copyrighted by Microsoft into my repo.
That said, I don’t really like the SSPL and will probably remove the requirement for it soonish after the code is published.
The critical thing for a company (which I’d assume @kemitchell knows, since he is a corporate lawyer and this is literally his day job) is minimising risk. The license is one aspect of this.
I think I understand the position companies have on risk, and I want to do my best to make risk minimization the real product I am selling.
Dual licensing doesn’t really help here because it lets you choose between risks (the risk the supplier will go away for the proprietary license versus the risks associated with a less permissive license). If your dual license allows people to pay for a more permissive license (e.g. MIT) then you now have a risk that someone will distribute the version that they receive.
Those risks are partially why I’m not going to dual license the core.
For a single developer, the largest risk that a company is likely to worry about is the bus factor. If you get hit by a bus, what happens to the software? That’s a massive risk for anything that’s going to be a core part of the work flow.
Yes, I agree, and it is a weakness of what I would like to do. But I do have some techniques for reducing the impact of the risk from the bus factor.
First, I document my code heavily. You can see this with my
bcespecially. The development manual inbcis the largest file, by far, in that repo. But I didn’t stop there. I commented every piece of code heavily so that someone else could go in, follow what I was doing, and be able to understand it. This reduces the impact of the bus factor by making it so users can have the backup plan of fixing bugs themselves if I get hit by a bus, and that backup plan has a chance of working.Second, I create extensive test suites. Once again,
bcis the example. The test suite is so extensive that I feel comfortable making massive experimental changes and just running the test suite (usually under Valgrind or ASan) to see if there was any regression. Should I get hit by a bus, the test suite then becomes a tool for anyone else wanting to make changes to do so without fear, just like me, which I believe reduces the impact of the bus factor.Third, companies can pay for the privilege of making the time factor a non-issue, and by “time factor,” I mean the possibility that I don’t have enough time or motivation to respond to their bug reports in a timely manner. But that’s the risk that they themselves have to mitigate; I can’t help with that.
There’s a big difference between buying a proprietary product from a big company and a proprietary product from some guy, especially if it’s a product with a lot of users and that is bringing in a lot of revenue.
I agree. In fact, it’s why I am doing all of the stuff I mentioned above. Doing those sorts of things brings a one-man project closer to a product from a big company. I think it’s why a project like Curl, which basically has a bus factor near 1, is so successful and widely used.
Sorry, said to much, but tl;dr, you are right about risk, I know you are right, and I’m doing my best to mitigate that.
-
I’d also add that risk and perceived risk are both important. It sounds as if the risk is low but I’m not sure what to suggest for reducing the perceived risk. A company has to do a lot of analysis of your code to understand how difficult it would be for someone to take over but that’s probably more work than most companies would do. You might be able to do some equivalent of underwriting: have some other company like Red Hat or Canonical provide maintenance contracts that they subcontract to you.
With curl, the reason that the risk is low is that, to a first approximation, everybody depends on curl. This means that, if Daniel were hit by a bus then everyone is equally screwed. If you are not the biggest company depending on curl then you can depend on someone else leading the effort to pick up maintenance costs.
-
-
-
-
This article incorrectly states that Zig has “colored”
asyncfunctions. In reality, Zig async functions do not suffer from function coloring.Yes, you can write virtually any software in Zig, but should you? My experience in maintaining high-level code in Rust and C99 says NO.
Maybe gain some experience with Zig in order to draw this conclusion about Zig?
-
Not sure if he changed the text but the article mentions the async color problem such that it could be considered applying generally. But the article doesn’t link that to Zig explicitly or did I miss it?
It would be fair to mention how Zig solved it as he mentions it for Go.
-
This response illustrates the number one reason I am not a fan of Zig: its proponents, like the proponents of Rust, are not entirely honest about it.
In reality, Zig async functions do not suffer from function coloring.
This is a lie. In fact, that article, while a great piece of persuasive writing, is also mostly a lie.
It tells the truth in one question in the FAQ:
Q: SO I DON’T EVEN HAVE TO THINK ABOUT NORMAL FUNCTIONS VS COROUTINES IN MY LIBRARY?
No, occasionally you will have to. As an example, if you’re allowing your users to pass to your library function pointers at runtime, you will need to make sure to use the right calling convention based on whether the function is async or not. You normally don’t have to think about it because the compiler is able to do the work for you at compile-time, but that can’t happen for runtime-known values.
In other words, Zig still suffers from the function coloring problem at runtime. If you do async in a static way, the compiler will be able to cheese the function coloring problem away. In essence, the compiler hides the function coloring problem from you when it can.
But when you do it at runtime and the compiler can’t cheese it, you still have the function coloring problem.
I think it is a good achievement to make the compiler able to hide it most of the time, but please be honest about it.
-
Calling this dishonest and a lie is incredibly uncharitable interpretation of what is written. Even if you’re right that it’s techncially incorrect, at worst it’s a simplification made in good faith to be able to talk about the problem, no more of a lie than teaching newtoniam mechanics as laws of physics in middle school is a lie because of special relativity, or teaching special relativity in high school is a lie because of general relativity.
Also, I’m not familiar with zig, but from your description I think you’re wrong to claim that functions are colored. Your refutation of that argument is that function pointers are colored, but functions are a distinct entity from function pointers - and one used much more frequently in most programming languages that have both concepts. Potentially I’m misunderstanding something here though, there is definitely room for subtlety.
-
Calling this a dishonest and a lie is incredibly uncharitable interpretation of what is written.
No, it’s not. The reason is because they know the truth, and yet, they still claim that Zig functions are not colored. It is dishonest to do so.
It would be completely honest to claim that “the Zig compiler can make it appear that Zig functions are not colored.” That is entirely honest, and I doubt it would lose them any fans.
But to claim that Zig functions not colored is a straight up lie.
Even if you’re right that it’s techncially incorrect,
I quoted Kristoff directly saying that Zig functions are colored. How could I be wrong?
at worst it’s a simplification made in good faith to be able to talk about the problem, no more of a lie than teaching newtoniam mechanics as laws of physics in middle school is a lie because of special relativity, or teaching special relativity in high school is a lie because of general relativity.
There are simplifications that work, and there are simplifications that don’t.
Case in point: relativity. Are you ever, in your life, going to encounter a situation where relativity matters? Unless you’re working with rockets or GPS, probably not.
But how likely is it that you’re going to run into a situation where Zig’s compiler fails to hide the coloring of functions? Quite likely.
Here’s why: while Kristoff did warn library authors about the function coloring at runtime, I doubt many of them pay attention because of the repetition of “Zig functions are not colored” that you hear all of the time from Andrew and the rest. It’s so prevalent that even non-contributors who don’t understand the truth jump into comments here on lobste.rs and on the orange site to defend Zig whenever someone writes a post about async.
So by repeating the lie so much, Zig programmers are taught implicitly to ignore the truthful warning in Kristoff’s post.
Thus, libraries get written. They are written ignoring the function coloring problem because the library authors have been implicitly told to do so. Some of those libraries take function pointers for good reasons. Those libraries are buggy.
Then those libraries get used. The library users do not pay attention to the function coloring problem because they have been implicitly told to do so.
And that’s how you get bugs.
It doesn’t even need to be libraries. In my
bc, I use function pointers internally to select the correct operation. It’s in C, but if it had been in Zig, and I had used async, I would probably have been burned by it if I did not know that Zig functions are colored.Also, I’m not familiar with zig, but from your description I think you’re wrong to claim that functions are not colored. Your refutation of that argument is that function pointers are colored, but functions are a distinct entity from function pointers - and one used much more frequently in most programming languages that have both concepts. Potentially I’m misunderstanding something here though, there is definitely room for subtlety.
You are absolutely misunderstanding.
How can function pointers be colored? They are merely pointers to functions. They are data. Data is not colored; code is colored. Thus, function pointers (data that just points to functions) can’t be colored but functions (containers for code) can be.
If data could be colored, you would not be able to print the value of the pointer without jumping through hoops, but I bet if you did the Zig equivalent of
printf("%p\n", function_pointer);it will work just fine.So if there is coloring in Zig, and Kristoff’s post does admit there is, then it has to be functions that are colored, not function pointers.
In Kristoff’s post, there is this comment in some of the example code:
// Note how the function definition doesn't require any static // `async` marking. The compiler can deduce when a function is // async based on its usage of `await`.He says “when a function is async…” An async/non-async dichotomy means there is function coloring.
What the compiler does is automagically detect async functions (as Kristoff says) and inserts the correct code to call it according to its color. That doesn’t mean the color is gone; it means that the compiler is hiding it from you.
For a language whose designer eschews operator overloading because it hides function calls, it feels disingenuous to me to hide how functions are being called.
All of this means that Zig functions are still colored. It’s just that, at compile time, it can hide that from you. At runtime, however, it can’t.
And that is why Zig functions are colored.
-
I have a hard time following all the animosity in your replies. Maybe I’m just not used to having fans on the internet :^)
In my article, and whenever discussing function coloring, I, and I guess most people, define “function coloring” the problem of having to mark functions as
asyncand having to prepend their invocation withawait, when you want to get their result. The famous article by Bob Nystrom, “What Color is Your Function?” also focuses entirely on the problem of syntactical interoperability between normal, non-async code and async, and how the second infects codebases by forcing every other function to be tagged async, which in turn forcesawaits to be sprinkled around.In my article I opened mentioning aio-libs, which is a very clear cut example of this problem: those people are forced to reinvent the wheel (ie reimplement existing packages) because the original codebases simply cannot be reasonably used in the context of an async application.
This is the problem that Zig solves. One library codebase that, with proper care, can run in both contexts and take advantage of parallelism when available. No async-std, no aio-libs, etc. This works because Zig changes the meaning and usage of
asyncandawaitcompared to all other programming languages (that use async/await).You seem to be focused on the fact that by doing async you will introduce continuations in your program. Yes, you will. Nobody said you won’t. What you define as “cheesing” (lmao) is a practical tool that can save a lot of wasted effort. I guess you could say that levers and gears cheesed the need for more physical human labor, from that perspective.
Sure, syntax and the resulting computational model aren’t completely detached: if you do have continuations in your code, then you will need to think about how your application is going to behave. Duh, but the point is libraries. Go download OkRedis. Write an async application with it, then write a blocking applicaton with it. You will be able to do both, while importing the same exact declarations from my library, and while also enjoying speedups in the async version, if you allowed for concurrent operations to happen in your code.
But how likely is it that you’re going to run into a situation where Zig’s compiler fails to hide the coloring of functions? Quite likely.
Thus, libraries get written. They are written ignoring the function coloring problem because the library authors have been implicitly told to do so. Some of those libraries take function pointers for good reasons. Those libraries are buggy.No. Aside from the fact that you normally just pass function identifiers around, instead of pointers, function pointers have a type and that type also tells you (and the compiler) what the right calling convention is. On top of that, library authors are most absolutely not asked to ignore asyncness. In OkRedis I have a few spots where I explicitly change the behavior of the Redis client based on whether we’re in async mode or not.
The point, to stress it one last time, is that you don’t need to have two different library codebases that require duplicated effort, and that in the single codebase needed, you’re going to only have to make a few changes to account for asyncness. In fact, in OkRedis I only have one place where I needed to account for that: in the Client struct. Every other piece of code in the entire library behaves correctly without needing any change. Pretty neat, if you ask me.
-
I have a hard time following all the animosity in your replies. Maybe I’m just not used to having fans on the internet :^)
The “animosity” (I was more defending myself vigorously) comes from Andrew swearing at me and accusing me, which he might have had a reason.
In his post, he claimed I said he was maliciously lying, but I only said that he was lying. I separate unintentional lies from intentional lies, and I believe all of you are unintentionally lying. Because I realized he thought that, I made sure to tell him that and tell him what I would like to see.
In my article, and whenever discussing function coloring, I, and I guess most people, define “function coloring” the problem of having to mark functions as async and having to prepend their invocation with await, when you want to get their result. The famous article by Bob Nystrom, “What Color is Your Function?” also focuses entirely on the problem of syntactical interoperability between normal, non-async code and async,
In Bob Nystrom’s post, this is how he defined function coloring:
The way you call a function depends on its color.
That’s it.
Most people associate color with async and await because that’s how JavaScript, the language from his post, does it. But that’s not how he defined it.
After playing with Zig’s function pointers, I can say with confidence that his definition, “The way you call a function depends on its color,” does apply to Zig.
and how the second infects codebases by forcing every other function to be tagged async, which in turn forces awaits to be sprinkled around.
This is what Zig does better. It limits the blast radius of async/await. But it’s still there. See the examples from my latest reply to Andrew. I had to mark a call site with
@asyncCall, including making a frame. But then, I couldn’t call theblue()function because it still wasn’t async. So if I were to make it work, I would have to makeblue()async. And I could do that while still making the program crash half the time.(Side note: I don’t know how to write out the type of async function. Changing
blue()to async is not working with the[2]@TypeOf(blue)trick that I am using. It’s still giving me the same compile error.)In my article I opened mentioning aio-libs, which is a very clear cut example of this problem: those people are forced to reinvent the wheel (ie reimplement existing packages) because the original codebases simply cannot be reasonably used in the context of an async application.
This is the problem that Zig solves. One library codebase that, with proper care, can run in both contexts and take advantage of parallelism when available. No async-std, no aio-libs, etc. This works because Zig changes the meaning and usage of async and await compared to all other programming languages (that use async/await).
This is not what you are telling people, however. You are telling them that Zig does not have function colors. Those two are orthogonal.
And I also doubt that Zig actually solves that problem. I do not know Zig, and it took me all of 30 minutes to 1) find a compiler bug and 2) find an example where you cannot run code in both contexts.
You seem to be focused on the fact that by doing async you will introduce continuations in your program. Yes, you will. Nobody said you won’t. What you define as “cheesing” (***) is a practical tool that can save a lot of wasted effort. I guess you could say that levers and gears cheesed the need for more physical human labor, from that perspective.
I have no idea what swear word you used there (I have a filter that literally turns swear words into three asterisks like you see there), but this is why I am not happy with Andrew. Now, I am not happy with you.
I used “cheesing” because while it is certainly a time saver, it’s still cheating. Yes, levers and gears cheese the application of force. That’s not a bad thing. Computers are supposed to be mental levers or “bicycles for the mind.” Cheesing is a good thing.
And yes, I am focused on introducing continuations into the program because there is a better way to introduce continuations and still get concurrency.
In fact, I am going to write a blog post about that better way. It’s called structured concurrency, and it introduces continuations by using closures to push data down the stack.
Sure, syntax and the resulting computational model aren’t completely detached: if you do have continuations in your code, then you will need to think about how your application is going to behave. Duh, but the point is libraries. Go download OkRedis. Write an async application with it, then write a blocking applicaton with it. You will be able to do both, while importing the same exact declarations from my library, and while also enjoying speedups in the async version, if you allowed for concurrent operations to happen in your code.
Where’s the catch? There’s always a catch. Please tell me the catch.
In fact, this whole thing is about me asking you, Andrew, and the others to be honest about what catches there are in Zig’s async story.
Likewise, I’m going to have to be honest about what catches there are to structured concurrency, and you can hold me to that when the blog post comes out.
No. Aside from the fact that you normally just pass function identifiers around, instead of pointers, function pointers have a type and that type also tells you (and the compiler) what the right calling convention is.
That is just an admission that functions are colored, if they have different types.
On top of that, library authors are most absolutely not asked to ignore asyncness. In OkRedis I have a few spots where I explicitly change the behavior of the Redis client based on whether we’re in async mode or not.
They are not explicitly asked. I said “implicitly” for a reason. “It’s not what programming languages do, it’s what they [and their communities] shepherd you to.” By telling everyone that Zig does not have function colors, you are training them to not think about it, even the library authors. As such, you then have to find those library authors, tell them to think about it, and explain why. It would save you and Andrew time if you just were upfront about what Zig does and does not do. And you would have, on average, better libraries.
The point, to stress it one last time, is that you don’t need to have two different library codebases that require duplicated effort, and that in the single codebase needed, you’re going to only have to make a few changes to account for asyncness. In fact, in OkRedis I only have one place where I needed to account for that: in the Client struct. Every other piece of code in the entire library behaves correctly without needing any change. Pretty neat, if you ask me.
That is neat. I agree. I just want Zig users to understand that, not be blissfully unaware of it.
-
The “animosity” (I was more defending myself vigorously) comes from Andrew swearing at me and accusing me, which he might have had a reason.
You called me a liar in the first comment you wrote.
Where’s the catch? There’s always a catch. Please tell me the catch.
Since I’m such a liar, why don’t you write some code and show me, and everyone else, where the catch is.
-
Since I’m such a liar, why don’t you write some code and show me, and everyone else, where the catch is.
Well, I don’t need to write code, but I can use your own words. You said that, “Every suspend needs to be matched by a corresponding resume” or there is undefined behavior. When asked if that could be a compiler warning, you said, “That’s unfortunately impossible, as far as I know.”
That’s the catch.
-
Why would you even use
suspendandresumein a normal application? Those are low level primitives. I didn’t use either in any part of my blog post, and in fact you won’t find them inside OkRedis either. Unless you’re writing an event loop and wiring it to epoll or io_uring, you only needasyncandawait.This is not a philosophical debate: talk is cheap, as they say, so show me the code. I showed you mine, it’s OkRedis.
-
Why would you even use suspend and resume in a normal application? Those are low level primitives.
Then why are they the first primitives you introduce to new users in the Zig documentation? They should have been last, with a clear warning about their caveats, if you even have them in the main documentation at all.
This is not a philosophical debate: talk is cheap, as they say, so show me the code. I showed you mine, it’s OkRedis.
I’m not going to download OkRedis or write code with it. I only learned enough Zig to make my examples to Andrew compile, and I have begun to not like Zig at all. It’s confusing and a mess, in my opinion.
But if you think that the examples I gave Andrew are not good enough, I don’t know what to tell you. I guess we’ll see if they are good enough for the people that read my blog post on it.
But I do have another question: people around Zig have said that its async story does not require an event loop, but none have explained why. Can you explain why?
-
Then why are they the first primitives you introduce to new users in the Zig documentation? They should have been last, with a clear warning about their caveats, if you even have them in the main documentation at all.
They’re the basic building block used to manipulate async frames (Zig’s continuations). First you complained that my blog post didn’t talk about how async frames work, and that I meant to deceive people by not talking about it, then you read the language reference and say it should not even mention the language features that implement async frames.
With your attitude in this entire discussion, you put yourself in a position where you have an incentive to not understand things, even well established computer science concepts such as continuations. If we talk at a high level, it’s a lie, if we get into the details, it’s confusing (and at this point we know what you mean to say: designed to be confusing). I can’t help you once you go there.
I’m looking forward to reading your blog post, although in all frankness you should consider doing some introspection before diving into it.
-
They’re the basic building block used to manipulate async frames (Zig’s continuations). First you complained that my blog post didn’t talk about how async frames work, and that I meant to deceive people by not talking about it, then you read the language reference and say it should not even mention the language features that implement async frames.
That’s the language reference? I thought it was the getting started documentation. Those details are not good to put in documentation for getting started, but I agree that they are good for a language reference. I would still put them last, though.
With your attitude in this entire discussion, you put yourself in a position where you have an incentive to not understand things, even well established computer science concepts such as continuations.
That’s a little ad hominem. I can understand continuations and not understand how they are used in Zig because the language reference is confusing. And yes, it is confusing.
If we talk at a high level, it’s a lie, if we get into the details, it’s confusing
It turns out that the problem is in your documentation and in your blog post. You can talk about it at a high level as long as your language about it is accurate. You can talk about the low level details once the high level subtleties are clarified.
(and at this point we know what you mean to say: designed to be confusing). I can’t help you once you go there.
I do not believe Zig was designed to be confusing, but after using it, I can safely say that the language design was not well done to prevent such confusion.
As an example, and as far as I understand at the moment, the way Zig “gets around” the function colors problem is to reuse the
asyncandawaitkeywords slightly differently than other languages and usessuspendto actually make a function async. So in typical code,asyncandawaitdo not have the function coloring problem. Which is great and all, but the subtleties of using them are usually lost on programmers coming from other languages.When I first heard about Zig, by the way, I was excited about it. This was back in 2018, I think, during the part of its evolution where it had comptime but not much more complexity above C. I thought comptime was great (that opinion has changed, but that’s a different story), and that the language looked promising.
Fast forward to today: Zig is immensely more complex than it was back then, and I don’t see what that complexity has bought.
That’s not a problem in and of itself, but complexity does make things harder, which means the documentation should be clearer and more precise. And the marketing should be the same.
My beef with Zig boils down to those things not happening.
Well, okay, I do have another beef with Zig: it sets the wrong tone. Programming languages, once used, set the tone for the industry, and I think Zig sets the wrong tone. So does Rust for that matter. But I can talk about that more in my blog post.
I’m looking forward to reading your blog post, although in all frankness you should consider doing some introspection before diving into it.
I have done introspection. I’ve learned where the function coloring problem actually is in Zig, and I’ve adopted new language to not come off in the wrong way. And I’ll do that in my blog post.
-
-
-
-
-
-
-
-
-
-
-
For me, the coloring problem describes both the static and runtime semantics. Does Zig handle the case where a function called with async enters some random syscall or grabs a mutex that blocks for a long time and isn’t explicitly handled by whatever the runtime system is or does that end up blocking the execution of other async tasks?
The reason why the runtime semantics matter to me when it comes to concurrency is because if you can block threads, then you implicitly always have a bounded semaphore (your threadpool) that you have to think about at all times or your theoretically correct concurrency algorithm can actually deadlock. That detail is unfortunately leaked.
-
If you grab a standard library mutex in evented I/O mode then it interacts with the event loop, suspending the async function rather than e.g. a futex() syscall. The same code works in both contexts:
mutex.lock(); defer mutex.unlock();There are no function colors here; it will do the correct thing in evented I/O mode and blocking I/O mode. The person who authored the zig package using a mutex does not have to be aware of the intent of the application code.
This is what the Zig language supports. Let me check the status of this feature in the standard library… looks like it’s implemented for file system reads/writes but it’s still todo for mutexes, sleep, and other kinds of I/O. This is all still quite experimental. If you’re looking for a reason to not use Zig, it’s that - not being stable yet. But you can’t say that Zig has the same async function coloring problem as other languages since it’s doing something radically different.
-
Thanks for the explanation and standard library status information.
I think the ability to make a function async at call time rather than at definition time is the best idea in Go’s concurrency design, and so, bringing something like that to a language with a much smaller runtime and no garbage collector is exciting. I look forward to seeing how this, and all of the other interesting ideas in Zig, comes together.
(p.s. thanks so much for
zig cc)
-
-
-
-
@pmeunier, I have questions. I’m interested in patch theory and patch-based version control.
However, I just can’t understand many things about it.
Here’s what I understand:
- The idea of using patches instead of snapshots.
- Patch commutation and how that helps with better merging.
- Most of being able to figure out if a patch depends on another patch.
- Pijul’s pristine can store conflicts.
What I don’t understand:
- Darcs merge algorithm.
- Why it goes exponential and when.
- Why storing conflicts in the pristine avoids the problem.
- Is Pijul’s pristine just a cache that can also store conflicts?
If you can explain those things to someone who struggles to read academic material, that would be great, although I do know that your work has been stolen before, so if you don’t want to explain, I understand.
-
I don’t know how Darcs worked, but it doesn’t have a datastructure independent from the patches: the patches are applied to a plain text version of the repository. In the absence of conflicts, every patch applied needs to be checked with all the patches since the last tag for commutation. Applying n patches cannot be faster than n^2.
When there are conflicts, I’m not sure anyone knew why it went exponential time, but it’s apparently fixed now (still quadratic, whereas Pijul is log).
Pijul’s pristine is not “just a cache”, it’s a CRDT. You can think of it as a cache if you want.
-
I have to say good for DHH for managing to make something out of Open Source. But I agree with @kline: just because DHH reaped rewards, while not intending to, is no argument for others to follow in his footsteps.
I make the argument why in The Social Contract of Open Source, but essentially, Open Source is draining two scarce resources: time and energy of maintainers. If companies do not want those scarce resources to dry up, they should help buffer them with another resource: money. And since time is money, giving money to maintainers may actually give them more time.
Companies need to learn that they need to take care of FOSS in order for FOSS to take care of them.
-
I don’t necessarily disagree with anything you said, but it’s weirdly slanted toward companies. You can just make stuff for yourself, and like minded people, and if companies contact you, simply ignore them.
To me the problem is that once you take money from companies, you are consciously or unconsciously beholden to their interests. I’d rather have that contract explicit in the form of a paycheck
-
I guess I could have made that clearer, that I think FOSS maintainers have the right to enforce the social contract in any way they want. That does include ignoring companies if that is what is desired. And that is the desire in many cases. I think I slanted it toward companies because they get upset when FOSS maintainers do what the maintainers want instead of what the companies want.
I think that however FOSS maintainers enforce the social contract is fine, including taking a paycheck. For me, personally, I suck at getting hired, so getting companies to pay me as a contractor would probably be easier.
-
-
Money doesn’t get me any more time, though. Unless maybe it’s enough money for me to quit any other jobs I have, and I happen to want to quit those jobs
-
Hard agree. I’d say there are two issues:
- Most OSS projects do not make it easy or obvious how to support them financially.
- Most OSS projects do not require financial support in their license.
2 kinda solves 1. Put, in plain English, how and when you expect the company to pay.
Using the MIT or BSD license while also asking businesses to pay is a recipe for disappointment.
-
-
gavinhoward
1 year ago
|
link
| on:
Dev corrupts NPM libs 'colors' and 'faker' breaking thousands of apps
I just want to address one line in the “Dependency Risk and Funding” post:
Daniel Stenberg of curl doesn’t wield that power (and probably also doesn’t want to).
Knowing what I know about Daniel, he probably does not want that power, as the author says.
However, he absolutely has that power.
Daniel could hide deliberate vulnerabilities in Curl that would allow him to take control of every machine running it. He could also hide code that would destroy those machines. In fact, he could hide code in Curl to delete itself and whatever code is using it, as well as mirrors of it, thus effectively wiping Curl off of the face of the Earth, even more so than what Marak did.
Just because people mirror Daniel’s code does not mean he doesn’t have the power to do serious damage.
-
However, he absolutely has that power.
I think you missed the point. Let me explain.
I have commit and release power over a very popular library (Harfbuzz) that goes into millions of machines too. I also have distro packager signing keys for more than one Linux distro including Arch Linux. The issue here is not commit or even release power, the issue is visibility. I know full well that every freaking character I commit gets scrutinized by several folks with even more programming chops than myself. Even if I turned evil and wanted to hijack something I would be called up short and tarred and feathered so fast I’d never recover.
Daniel is in a similar boat. Not only is he a known entity but the code he writes is directly scrutinized by others and he would have to be very devious indeed over a long haul to get something really dangerous past all the watchers.
The NPM and other similar ecosystems with deep dependency trees (where most people writing and release apps don’t even know where most of the code is coming from at compile time) are different. It is ultimately quite easy to write and maintain something trivial but “useful” and then hijack high profile projects with a dependency attack in a way that it is not easy for actual maintainers of high profile projects to do directly on their own projects.
I believe that’s what the article was referring to when it said Daniel doesn’t have the same power. He would have to work a lot harder to get even something trivial through compared to how a lone actor deep in a Node dependency tree could so easily affect so many projects.
-
-
gettalong
1 year ago
|
link
| on:
How Effective Is Copyleft with Commercial Licensing at Making Money?
I have gone a similar route the past years with HexaPDF. My goal was to get an adequate side income next to my 40h job, so it was clear for me that I wouldn’t sell support as main income because that would take too much time away from the main development.
The PDF library is dual-licensed AGPL and a commercial license. The reason for the license choice was mainly that I wanted to provide a command line tool for manipulating PDFs. So everyone can use the AGPL version without much thinking about the license terms. And companies will choose the commercial license. And this works. I’m not sure how dual-licensing would work in your case, e.g. with a build system.
HexaPDF fills a niche where no other similar product existed/exists. And it still took a long time to get somewhere business-wise. I guess that mainly comes down to me not doing enough marketing and sales, and having a product that is not needed by that many companies.
I started the company in 2018 and now, 3.5 years later, I have about 25 paying customers. With about double the number I will have the side income I initially targeted. We will see how it goes :)
If you want to have 2 times the typical developer salary, I would do a market research and see how many companies would benefit from your software. If your product is better, than the companies will happily pay.
-
I’m not sure how dual-licensing would work in your case, e.g. with a build system.
It’s going to be distributable as a library, but it’s also going to be a distributed build system, like Bazel.
If you want to have 2 times the typical developer salary, I would do a market research and see how many companies would benefit from your software. If your product is better, than the companies will happily pay.
I hope this is true, though I suspect that I also have to prove to the companies that it’s better. And not just better, but far better (because of inertia). I think that may be the hardest part.
-
I hope this is true, though I suspect that I also have to prove to the companies that it’s better. And not just better, but far better (because of inertia). I think that may be the hardest part.
Isn’t that the beauty of dual-licensing? That you can do everything completely in the open and let companies try everything out without them needing to ask any licensing questions or do upfront payment?
I don’t think that your product has to be far better than all the others, it just has to have a business advantage for your customers.
For example, although I think HexaPDF is great (naturally :) I know that it is by far not feature complete and there are commercial libraries in other languages that are much better in various regards. Yet, I have one customer who came across HexaPDF, tested it for their use case and found it superior to all other tools they tried but still not optimal for what they wanted. So I worked with them for months before they bought a license, and in the process made HexaPDF better for everyone.
-
Isn’t that the beauty of dual-licensing? That you can do everything completely in the open and let companies try everything out without them needing to ask any licensing questions or do upfront payment?
Depends on the license. AGPL is famous for being entirely forbidden in some larger companies (e.g. Google), so for people working there, the product doesn’t exist until the other license is acquired.
This doesn’t mean “don’t use the AGPL”, but the dynamic you’re envisioning might not work out for any number of reasons.
-
-
The operating systems to use at Google are well curated - Linux would be https://en.wikipedia.org/wiki/GLinux. As the policy is “no AGPL”, my guess (I work at Google but didn’t check the licenses) is that the GLinux maintainers simply don’t (re-)package such software.
-
-
-
-
-
-
itamarst
1 year ago
|
link
| on:
How Effective Is Copyleft with Commercial Licensing at Making Money?
Lots of companies pay for software. There’s a whole giant industry selling commercial software… which leads to the question of why not making it proprietary.
- What sort of product is it?:
- Who would benefit from availability of source?
- Who would benefit from it being open source (you can give people source code under a proprietary license, so this is a different question than the previous one)?
(I’m working on commercial product, with open source variant with slightly different use case as marketing, and … a bunch of people use the open source tool, and I’ve only gotten a single patch, ever. It’s not clear what being open source does for anyone in this particular example.)
-
You have a good point, so let me answer your questions:
- It is a tool meant for developers: a build system.
- Everyone; it is actually crucial to the software supply chain that the source is available. If the build system is not Open Source (i.e., you can’t compile it yourself), you don’t know if it has been backdoored with a Trusting Trust attack, just like a compiler.
- End users. If it’s only source-available, then companies that distribute software that builds with it could conceivably make it really hard to build their software, even if that software is FOSS or source-available.
But beyond the fact that it is actually crucial to be FOSS for security, there is another big reason: developers will not adopt a non-FOSS tool. If it is FOSS, it has a chance, and if it is not, then it has none.
-
-
But beyond the fact that it is actually crucial to be FOSS for security, there is another big reason: developers will not adopt a non-FOSS tool. If it is FOSS, it has a chance, and if it is not, then it has none.
Open source isn’t a requirement for commercially successful build tools; Incredibuild is a proprietary build system used by Adobe, Amazon, Boeing, Epic Megagames, Intel, Microsoft, and many other companies. Most of the market consists of pragmatists; they’ll adopt a new product if it addresses a major pain point.
Is there a distributed build tool for Rust yet? That may be a market worth pursuing.
-
I did not expect anyone to say that closed-source build systems were used, but you and a sibling named two.
As far as making a distributed build tool for Rust, yeah, I can do that. Thank you.
-
-
It is a tool meant for developers: a build system.
I am curious how are you planning to legally structure dual-licensing of a build system. I believe most (all?) examples of dual-licensing where one license is free/open source involve a copyleft license (commonly GPL). In order to trigger copyleft’ness the user must produce a derivative work of your software (e.g., link to your library). I don’t see how using a build system to build a project results in derivative work. I suppose there are probably some dual-licensed projects based on AGPL but that doesn’t seem to fit the build system either.
I also broadly agree with what others have said about your primary concern (that the companies will steal rather than pay): companies (at least in the western economies) are happy to pay provided prices are reasonable and metrics are sensible (e.g., many would be reluctant to jump though licensing server installation, etc). But companies, especially large ones, are also often conservative/dysfunctional so expect quite a bit of admin overhead (see @kornel comment). For the level of revenue you are looking at (say, ~$300K/year), I would say you will need to hire an admin person unless you are prepared to spend a substantial chunk of your own time doing that.
This is based on my experience running a software company (codesynthesis.com ) with a bunch of dual-licensed products. Ironically, quite a bit of its revenue is currently used to fund the development of a build system (
build2; permissively-licensed under MIT). If you are looking to build a general-purpose build system, plan for a many-year effort (again, talking from experience). Good luck!-
I am curious how are you planning to legally structure dual-licensing of a build system.
It will also be a library.
There are plenty of places in programming where it is necessary to be able to generate tasks, order those tasks to make sure all dependencies are fulfilled, and run those tasks (hopefully as fast as possible).
One such example is a init/supervision system. There are services that need to be started after certain others.
(Sidenote: I’m also working on an init/supervision system, so technically, companies don’t need to make their own with my library. It’s just an example.)
I suppose there are probably some dual-licensed projects based on AGPL but that doesn’t seem to fit the build system either.
This build system will be distributable, like Bazel, so yes, that does apply.
I also broadly agree with what others have said about your primary concern (that the companies will steal rather than pay): companies (at least in the western economies) are happy to pay provided prices are reasonable and metrics are sensible (e.g., many would be reluctant to jump though licensing server installation, etc).
What are reasonable prices, though?
But companies, especially large ones, are also often conservative/dysfunctional so expect quite a bit of admin overhead (see @kornel comment). For the level of revenue you are looking at (say, ~$300K/year), I would say you will need to hire an admin person unless you are prepared to spend a substantial chunk of your own time doing that.
I am going to do it, yes, but I’m also going to be helped by my wife.
This is based on my experience running a software company (codesynthesis.com ) with a bunch of dual-licensed products. Ironically, quite a bit of its revenue is currently used to fund the development of a build system (build2; permissively-licensed under MIT). If you are looking to build a general-purpose build system, plan for a many-year effort (again, talking from experience). Good luck!
Oh, I’m cutting features out of my build system, so I don’t expect it to take that long. Also, I’m not running a business like you are.
Thank you.
-
What are reasonable prices, though?
The video Designing the Ideal Bootstrapped Business has some excellent advice on pricing; the author has sold at least 3 startups.
-
-
-
kornel
edited
1 year ago
|
link
| on:
How Effective Is Copyleft with Commercial Licensing at Making Money?
Companies do pay for software. I’m dual-licensing pngquant. It doesn’t quite pay a developer salary, but it’s worth my time.
I can’t truly know how many companies use it against the license, but OTOH there are many that come to me and happily get a license.
In my experience companies want to pay. They value having a written contract that guarantees they will have support and that the license is truly valid (there’s a real risk that a random free library on the net is incorrectly licensed and the code belongs to someone else, eg due to author’s employment contract.)
I’m not worried at all about companies copying the code. If the task is their core competency, they will write their own thing no matter what. But when it’s not, it doesn’t make sense for them to steal the code – simply because development and maintenance is costly, and it’s taking up their resources they need for their core business. When they buy code it’s because they want to move faster and stay focused, and not reinvent the wheel.
Be warned that the real work that goes into running such software business is not software development. It’s mainly sales and marketing. Answering enquiries, contract negotiation, soul-sucking SAP vendor registration forms, chasing invoices, dealing with US banking being on a different planet than everyone else, and so on.
-
Thank you for your answer!
In my experience companies want to pay. They value having a written contract that guarantees they will have support and that the license is truly valid (there’s a real risk that a random free library on the net is incorrectly licensed and the code belongs to someone else, eg due to author’s employment contract.)
Wow, this completely goes against what it appears like from the outside. I hope you are right.
I’m not worried at all about companies copying the code. If the task is their core competency, they will write their own thing no matter what. But when it’s not, it doesn’t make sense for them to steal the code – simply because development and maintenance is costly, and it’s taking up their resources they need for their core business. When they buy code it’s because they want to move faster and stay focused, and not reinvent the wheel.
I hadn’t thought about that.
-
Wow, this completely goes against what it appears like from the outside. I hope you are right.
Companies are all different but there are a few useful rules of thumb:
- If a company has done something multiple times, they will have a process for it. Anything that has a process is easy.
- In general, companies want most of their employees working on what they regard as their core competency (which may not be what you think it is). They will spend money to outsource things if it helps their employees focus on the things that they’re paid to do.
- Companies tend to be better at optimising for reduced risk than for maximum gain.
The first of these may help you but it probably won’t. Buying software is almost certainly something that any company that you talk to has a process for. Often, this will require buying it from an approved supplier. This means that you will need to either have a reseller’s agreement with one of their approved suppliers (meaning that the reseller takes a big cut) or you will have to go through their process for approvals (this is usually not worth it unless you’re expecting a lot of revenue from the customer and can take a long time).
The second point probably helps you. If you are selling a tool that makes their employees more efficient at doing whatever it is that they’re actually paid to do, then it is probably better for them to buy it from you than it is for them to develop it in house.
The last point is the one that will cause you the most problems. If you are a small company (especially if you are a sole trader) then you are seen as very high risk. If your software is amazing and you get hit by a bus, what happens? Can someone else maintain it with the license that they have? How much will that cost? If they discover a critical bug, what is the chance that you’ll be able to fix it without disrupting their schedules? Is your support contract giving them measurably lower risk than just using the free version?
Beyond that, are they shipping your code? If so, there are all sorts of compliance things that are easy for permissively licensed software, harder for proprietary or copyleft software. It may be cheaper (factoring in the risk) to write something and release it under a permissive license than to use your version.
-
Wow, this completely goes against what it appears like from the outside. I hope you are right.
Why Are Enterprises So Slow? explains a lot of the motivations behind typical enterprise policies. It’s common to have support contracts for everything to mitigate risk.
I hadn’t thought about that.
I have seen this effect firsthand at work. Several of our internal applications have been discontinued because third-party vendors released comparable products.
-
-
-
kemitchell
1 year ago
|
link
| on:
How Effective Is Copyleft with Commercial Licensing at Making Money?
Companies pay for software all the time. You may be reading too many headlines from one corner of the industry.
Dual licensing (AKA selling exceptions) has worked and does work for many firms, large and small, both on its own and in combination with other models, like selling proprietary extensions or complementary software. I keep a very incomplete list of examples at duallicensing.com. There have been many more successful dual-licensing sales than lawsuits by dual-licensing companies against deadbeat users.
Merely sprinkling a business model on top of a project with a few website changes and social media posts almost never yields meaningful money. Not with dual licensing, not with open core, not with services or any other model. You need a model and you need to push. Going into business is adding a whole ’nother project to your life.
Driving paid-license sales will take time and energy. That is time and energy you will not also be able to spend on your software. On the upside, paid-license sales can take substantially less time and energy than developing complementary products, hosting, developing closed, one-off software on contract, or providing high-touch professional services like training. Your project is your project and there won’t be any business need to segment it into free and paid chunks, since what you’re selling is fundamentally permissions, not bits.
-
Dual licensing (AKA selling exceptions) has worked and does work for many firms, large and small, both on its own and in combination with other models, like selling proprietary extensions or complementary software. I keep a very incomplete list of examples at duallicensing.com.
I don’t know how many of your examples are actually small people, but I do know of one: VideoLAN. And the link I had in my original post was the VideoLAN guy talking about how it really hasn’t worked out very well. So while you have examples (and thank you for them; I’m going through them now), I’m a little nervous about how effectively those examples actually make money.
There have been many more successful dual-licensing sales than lawsuits by dual-licensing companies against deadbeat users.
I’ll have to take your lawyer’s word for that, but I do wonder if that’s just because the threat is enough from bigger entities. If I, as an individual, am not enough of a threat, would they care enough to pay as required? I don’t really know.
Merely sprinkling a business model on top of a project with a few website changes and social media posts almost never yields meaningful money. Not with dual licensing, not with open core, not with services or any other model. You need a model and you need to push. Going into business is adding a whole ’nother project to your life.
Agreed. My current business model plan is two-fold: licensing and on-call support. I just don’t think people will pay for that if they can get away with not paying.
Driving paid-license sales will take time and energy. That is time and energy you will not also be able to spend on your software. On the upside, paid-license sales can take substantially less time and energy than developing complementary products, hosting, developing closed, one-off software on contract, or providing high-touch professional services like training.
I do understand that driving sales takes time and energy. Unfortunately, that’s just what I’m going to have to do to make money. I’d rather spend half my time on that than all of my time on someone else’s software.
Your project is your project and there won’t be any business need to segment it into free and paid chunks, since what you’re selling is fundamentally permissions, not bits.
Are you saying I should keep it closed source? I’m not entirely sure what you are saying here.
-
If you think your potential customers are a bunch of big companies and you’re afraid of big companies, I’d suggest you reach out to some founders at companies that successfully license big companies. Or find another line of business.
If you’re looking for validation of the idea that dual licensing doesn’t work because large companies are all big meanies who don’t play fair, I can’t corroborate. I’m sure it happens. And probably more often where the developer obviously lacks spine and cowers. But the dual licensing failure cases I see have a lot more to do with more basic business faults.
-
If you think your potential customers are a bunch of big companies and you’re afraid of big companies, I’d suggest you reach out to some founders at companies that successfully license big companies. Or find another line of business.
That is a fair criticism. I’ll take the L, and I’ll see about doing as you said.
If you’re looking for validation of the idea that dual licensing doesn’t work because large companies are all big meanies who don’t play fair, I can’t corroborate. I’m sure it happens.
My wife, the one with business sense, thinks it won’t work because of this, so it’s not just me. In fact, I was pretty idealistic about it until a month ago. She tried to get me to see sense for years, and I’ve only recently come around.
And probably more often where the developer obviously lacks spine and cowers.
If I had the resources to go after companies in the case that they violated my license, I would happily “grow a spine” and continue with my work. But I don’t have the resources because a lawyer like you doesn’t come cheap.
But the dual licensing failure cases I see have a lot more to do with more basic business faults.
I believe it. I’ve been taking potential business ideas to my wife for years, and having the business sense that she does, she has shot them all down. So I could see it being hard to find the right one.
In other words, I guess I have not found the right one. Good to know.
-
Wasn’t trying to talk you down. But you came in with a question based on a presupposition that contradicts my experience. For what it’s worth, I’m a deals lawyer, not a lawsuits lawyer.
I have seen founders and salespeople have to push on large company users who weren’t initially willing to deal. When the vendor is small, that is definitely an asymmetric conflict. If you find yourself on the smaller side of an asymmetric conflict, you can’t think just in terms of all the big-side resources you don’t have, like how many dollars or bodies or lawyers they have that you don’t. You have to work other leverage. Go talk to founders that have won some of those battles.
For what it’s worth, the VideoLAN comment you cited seemed to have a lot more to say about lack of interest in technical support contracts than dual licensing. That fits with my perception of their software’s primary use case and license choice, which don’t put a lot of users in positions where they need other license terms.
It’s hard to sell tech support for reliable, well documented software. It’s relatively easy to sell technical support to large companies with urgent problems.
-
-
-
-
-
I use a Keyboardio Model 01.
Because I use Dvorak, I have a layer for Qwerty, in case my wife needs to use the keyboard.
Then, of course, I have a layer for the extra keys, including imitating common Qwerty keyboard shortcuts like Ctrl+c, Ctrl+v, etc.
But my real innovation is having a layer that basically pastes the reverse of right side of the keyboard onto the left side, and it is activated by a chord on the left side.
What this means is that when I am using certain shortcut-heavy programs that also require mouse, like Blender, I can control the mouse with my right hand while handling nearly all keyboard work with my left hand. It saves me a lot of moving my right hand between the mouse and the keyboard.
-
Since companies steal Open Source software without a care in the world, what’s to stop companies from stealing Rig and embedding it into their proprietary software?
Nothing. Nothing has ever been able to prevent this. Even if you used a nonfree license they could do this if they thought the benefit to them outweighed the odds that you can afford to sue.
But what’s the alternative? Not make software at all? I’d rather help people than not, and also let’s be honest I make software for me and because I can’t not.
-
Author here.
I understand your position. It was my position up until a week ago.
But I fear that if companies steal my code, they may end up doing far more harm than my code does good otherwise. I could be overthinking it, but it’s also a build system which could be used to backdoor just about anything.
-
But as I said, what is the alternative? Maybe you personally have the option to just not work on software. I don’t really, I’m not sure who I would even be. But moreover, if everyone who cares stops trying then all software would be evil all the time and that seems… Worse?
-
You have really good points.
I guess the question is: can companies be as creative as Open Source authors?
If so, then yes, it would be worse not to have Open Source anyway because we would have the same software, just evil no matter what.
If not, then Open Source may actually provide creativity for the evil those companies do because without Open Source, they may not be able to accomplish as much.
I don’t know the answer to the above question. I guess my discouragement is based on the feeling that Open Source is fueling the evil creativity of companies. If someone could show me that that feeling is wrong, I’d be so happy.
-
“Open Source software I do write could end up harming more users than it helps.”
I think structure is important. Structure, moreso than license, governs how software will be used. I never really believed in the copyleft licenses providing any sort of ethical insurance or even guaranteeing that modifications get contributed back. I agree that copyleft is mostly just wishful thinking when the rubber hits the road.
BUT! Software is much more than its license. Can some greedy or power hungry group “steal” BitTorrent or ipfs and make tons of money or collect tons of data from it? I’m sure there are examples, but surely they are minor/marginal compared to the prevelant “evil” uses of “big data” software like Apache Cassandra or Hadoop.
I believe working on software IS worth it, and in fact, it’s really important: the way the world is structured right now sucks for a lot of people!! A lot of that comes from the structure of the software. I can imagine a lot of software that works for the common man, but wouldn’t benefit the rich and greedy. Such software would be structured differently, it would have different goals / constraints, and most likely it wouldn’t make much money.
Critically, such software does not exist right now. Or at least it is incomplete or in disrepair. Its up to us to build it / fix it. Open Source just happens to be the most practical way to do that.
-
I hope you’re right, but I don’t see how to structure Open Source in such a way to prevent companies from stealing it.
-
If you’re hell-bent on this goal, try writing software that is useful to, and extensible by, regular people and not super useful to beg tech. For example, most of the tech produced at JMP.chat is not useful to big tech because it only does thing they don’t want and that don’t make sense at Big Scale but the stack and product are super useful to the individuals who use it.
-
-
-
-
-
-
-
It doesn’t feel like a solid line of thinking.
From his conclusion:
-
“I can’t get a job” - Apparently because all software companies are evil. Except not all of them are. And there are plenty nonprofits out there that are really trying to help people. He’ll just have to take a small paycut.
-
“I can’t make money from writing Open Source software” - Sure, it’s harder, but not impossible. Also, that’s not really the point of OSS.
-
“Open Source software I do write could end up harming more users than it helps.” - Yes, software can be used for bad purposes. So can hammers. Is it unethical to create hammers?
There are real issues with open-source, that can be seen in some of the examples he gives, but he doesn’t address them in any meaningful way. Sorry if I’m being insensitive, but it sounds more like whining than philosophy to me.
-
Nothing on the site feels like a solid line of thinking to me. The author believes and argues, at length, for things that are demonstrably false. For example, they think that vaccines are more dangerous than COVID-19.
But I think you’re over-reading the piece. There’s an implied “For Me” in the question the title asks. Clearly this individual can’t get a job working on FOSS, they can’t otherwise make money from it, and they fear that it harms users in some way more than it helps them. If they’ve done that math, it obviously isn’t worth working on it for them.
I don’t think it’s generally applicable either, and I think your analysis of it is correct in that context. But I’m not sure that argument was being made.
-
“Releasing open source software under capitalism just leads to exploitation. But releasing closed source software under capitalism is even worse! I guess there just isn’t any good solution here.”
Maybe the problem isn’t the software, my dude.
-
Author here.
It doesn’t feel like a solid line of thinking.
I fully admit it’s not; this is a post where I put feelings in more than anything else. I feel discouraged because I always believed that I could make a good difference in the world with FOSS. That belief has been recently challenged, and I’m feeling a little lost as a result.
From his conclusion:
Perhaps I should not have mentioned any of that, but the real point of mentioning them was to say that I had to leave the software industry earlier, but not software. Now, it feels like I have to leave software itself.
Yes, software can be used for bad purposes. So can hammers. Is it unethical to create hammers?
No, it’s not, and I guess I did not articulate well. What I was trying to say is that, even though I try to write my code as ethically as possible, it would still do more harm to users. I think software is different from hammers in that the harm that can be done by them scales far faster than the harm that can be done by things like hammers.
For example, a hammer can be wielded for harm, but it can only harm as many people as the bad guy can personally reach. But software can be taken by a global company and harm billions in the process.
And that’s not even to say that create software is unethical; I don’t think it would be unethical for me to make Rig and to make it Open Source because I would try to do everything I can to make it helpful while preventing harm.
It just feels like the harm scales so much that no matter the ethics, it could end up accomplishing the opposite of what I want.
There are real issues with open-source, that can be seen in some of the examples he gives, but he doesn’t address them in any meaningful way.
I admit in the post that I don’t have answers. Instead the real point of the post was the last line inviting people to contact me with their thoughts.
-
Merry Christmas, Gavin. This problem is solved. It will be rolled out after the New Year. Up to individual maintainers to adopt it. I can let you know when it drops in case you miss it.
-
I presume you mean the problem of scalable harm? Your post is really cryptic.
Regardless, I’m intrigued, so yes, please let me know when it drops.
-
-
-
-
I feel kinda bad about how far my angry rant about the state of the industry has gone and how many people it has touched. I’m sorry if my angry nihilistic feelings influenced this line of thinking at all. I don’t know what to do about it.
-
Oh, no, you didn’t make me feel this way.
Behind the scenes, what really happened is that your article (among others) woke me up to what companies actually do. I asked my wife, and it turns out that she had been trying to get me to understand this for years; my idealism had just been blinding me to it. It was talking to my wife that actually brought the discouragement, though you could say it was really ignorance that caused it by blinding me to reality until reality slapped me.
-
And yet, you have not really harmed people. Without giving a parade of horribles, I think that you already know about the crimes and harms being perpetuated by our industry, and writing an angry rant does not stack up to what any such paradegoer has done.
It is typical and understandable that reading or writing about history, including the history of our field, is uncomfortable and produces negative feelings. But you must remember that the endless positivity of our society serves to shame historians for their honesty without fixing the problems of the past.
-
-
Yes. It’s not worth working on open source without compensation, but Free Software is still worthwhile.
I notice that you did not mention AGPL or other licenses which are known to repel corporations; why not?
-
Author here.
I notice that you did not mention AGPL or other licenses which are known to repel corporations; why not?
GitHub Copilot. If it weren’t for that, I’d license my current code under AGPL and relicense later after getting my licenses reviewed by a lawyer.
Good question; sorry for not making it clear.
-
-
That’s a nice feature.
Thank you for telling me, and sorry about that.
-
-
-
-
-
My experience with Nix in the past has been slightly less advanced/dynamic (mainly NixOS and simple packages) but the performance point was a major factor to me. I understand that Flakes are meant to address some of this, but as it stands, Nix evaluations can get really slow. I’d personally love to see something closer to the speed of an
apk add.I’d be curious if there is a “simpler” version of Nix that could exist which gets speed ups from different constraints. For example, I’ve found please to be faster than most bazel projects, partly due to being written in go and having less of a startup cost, but also because the build setup seems to be simpler.
I think that the root of the problem might be that Nix is a package build system, not a development build system, and so is build with different assumptions. I wonder if there’s a way to build a good tool that does both package builds (tracks package dependencies, build binary artifacts, has install hooks) and a build tool (tracks file dependencies, has non-build rules such as linting, and caches artifacts for dev not installation). I’m just spitballing but it seems to me like trying to reconcile these two different systems might force a useful set of constraints that results in a fast & simple build system? (though it could just as easily go the other way and become unwieldy and complex).
-
Nix is a package build system, not a development build system
Ah, but this is exactly the point :-)
There is nothing fundamental about Nix that prevents it from covering both, other than significant performance costs of representing the large build graphs of software itself (rather than a simplified system build graph of wrapped secondary build systems). At TVL we have buildGo and buildLisp as “Bazel-like” build systems written in Nix, and we do use them for our own tools, but evaluation performance suffers significantly and stops us from adding more development-focused features that we would like to see.
In fact this was a big driver behind the original motivation that led to us making a Nix fork, and then eventually starting Tvix.
-
I wonder if there’s a way to build a good tool that does both package builds (tracks package dependencies, build binary artifacts, has install hooks) and a build tool (tracks file dependencies, has non-build rules such as linting, and caches artifacts for dev not installation).
I believe there is! It mostly comes from a paper called A Sound and Optimal Incremental Build System with Dynamic Dependencies (PDF), which is not my work (although I’m currently working on an implementation of the ideas).
There are three key things needed:
- Dynamic dependencies.
- Flexible “file stamps.”
- Targets can execute arbitrary code, instead of just commands.
The first item is needed based on the fact that dependencies can change based on the configuration needed for the build of a package. Say you have a package that needs libcurl, but only if users enable network features.
It is also needed to import targets from another build. I’ll use the libcurl example above. If your package’s build target has libcurl as a dependency, then it should be able to import libcurl’s build files and then continue the build making the dependencies of libcurl’s build target dependencies of your package’s build targets.
In other words, dynamic dependencies allow a build to properly import the builds of its dependencies.
The second item is the secret sauce and is, I believe, the greatest idea from the paper. The paper calls them “file stamps,” and I call them “stampers.” They are basically arbitrary code that returns a Boolean showing whether or not a target needs updating or not.
A Make-like target’s stampers would check if the file
mtimeis less than any of its dependencies. A more sophisticated one might check that any file attributes of a target’s dependencies have changed. Another might hash a file.The third is needed because otherwise, you can’t express some builds, but tying it with dynamic dependencies is also the bridge between building in the large (package managers) and building in the small (“normal” build systems).
Why does this tie it all together? Well, first consider trying to implement a network-based caching system. In most build systems, it’s a special thing, but in a build system with the above the things, you just need write a target that:
- Uses a custom stamper that checks the hash of a file, and if it is changed, checks the network for a cached built version of the new version of the file.
- If such a cache version exists, make updating that target mean downloading the cache version; otherwise, make updating the target mean building it as normal.
Voila! Caching in the build system with no special code.
That, plus being able to import targets from other build files is what ties packages together and what allows the build system to tie package management and software building together.
I’ll leave it as an exercise to the reader to figure out how such a design could be used to implement a Nix-like package manager.
(By the way, the paper uses special code and a special algorithm for handling circular dependencies. I think this is a bad idea. I think this problem is neatly solved by being able to run arbitrary code. Just put mutually dependent targets into the same target, which means targets need to allow multiple outputs, and loop until they reach a fixed point.)
I’m just spitballing but it seems to me like trying to reconcile these two different systems might force a useful set of constraints that results in a fast & simple build system?
I think that design is simple, but you can judge for yourself. As to whether it’s fast, I think that comes down to implementation.
-
-
That is a good question.
To be able to do dynamic dependencies, you basically have to have a “Suspending” scheduler strategy, so that’s what mine will have.
However, because targets can run arbitrary code, and because stampers can as well, my build system doesn’t actually fit in one category because different stampers could implement different rebuilding strategies. In fact, there could be stampers for all of the rebuilding strategies.
So, technically, my build system could fill all four slots under the “Suspending” scheduler strategy in the far right column in table 2 on page 27.
In fact, packages will probably be build files that use deep constructive traces, thus making my build system act like Nix for packages, while in-project build files will use any of the other three strategies as appropriate. For example, a massive project run by Google would probably use “Constructive Traces” for caching and farming out to a build farm, medium projects would probably use “Verifying Traces” to ensure the flakiness of
mtimedidn’t cause unnecessary cleans, and small projects would use “Dirty Bit” because the build would be fast enough that flakiness wouldn’t matter.This will be what makes my build system solve the problem of scaling from the smallest builds to medium builds to the biggest builds. That is, if it actually does solve the scaling problem, which is a BIG “if”. I hope and think it will, but ideas are cheap; execution is everything.
Edit: I forgot to add that my build system also will have a feature that allows you to limit its power in a build script along three axes: the power of targets, the power of dependencies, and the power of stampers. Limiting the last one is what causes my build system to fill those four slots under the “Suspending” scheduler strategy, but I forgot about the ability to limit the power of dependencies. Basically, you can turn off dynamic dependencies, which would effectively make my build system use the “Topological” scheduler strategy. Combine that with the ability to fill all four rebuilder strategy slots, and my build system will be able to fill 8 out of the 12.
Filling the other four is not necessary because anything you can do with a “Restarting” scheduler you can do with a “Suspending” scheduler. And restarting can be more complicated to implement.
-
-
How so? I did not find the presentation lacking in any such respects.
Defining person to be man or woman is obviously false and needlessly exclusionary. The talk is itself quite good and there’s a reason I’m working from it, but that production probably wouldn’t fly today.
This is not false, exclusionary, or transphobic or any other sort of “problematic”. It was simply an illustrative example of defining a concept as a union of existing concepts, that everybody in the intended audience could easily understand. It is not, and is not pretending to, say anything at all about sex, gender, personhood, or anything else. Pretending otherwise is disingenuous at best.
It’s worth noting that the production isn’t great, and Steele says so himself in the typed notes. There’s no need to accuse anyone of anything that the original author hasn’t already conceded.
From the typed version:
I’m flagging this reply because I find it kind of dismissive. I also think that style of rhetoric will invite a low standard of discussion.
What do I see here? An abridged transcript:
I see this as penalising the author, who so far has contributed positively to this thread.
If it’s very important to you to express this opinion, we can help you do it in a way that’s clearly positive sum
If we can’t disagree with someone, even with someone who has contributed positively, then we can’t have discourse. Your flag is against free discourse.
I’m not trying to discourage disagreement itself; I’m trying to advocate for a style of discussion that leads to the flourishing of all participants.
I clearly have not conveyed myself as well as I intended, and I don’t have the motivation to improve my original comment further. You should probably just ignore it.
Indeed. I wonder if we can deconstruct why that happened and fix it. I think that the main reason that the talk needs to define “man” and “woman” is so that it can talk about great men and great women of history. A modern reconstruction might go through the motions of humorously defining “person”, a two-syllable word, instead.