1. 4

    Surely I’m not going to be the only one expecting a comparison here between go’s. I’m not really well versed in GC but this appears to mirror go’s quite heavily.

    1. 12

      It’s compacting and generational, so that’s a pair of very large differences.

      1. 1

        My understanding, and I can’t find a link handy, is that the Go team is on a long term path to change their internals to allow for compacting and generational gc. There was something about the Azul guys advising them a year+ ago iirc.

        Edit; I’m not sure what the current status is, haven’t been following, but see this from 2012, look for Gil Tene comments:

        https://groups.google.com/forum/#!topic/golang-dev/GvA0DaCI2BU

        1. 3

          This presentation from this July suggests they’re averse to taking almost any regressions now even if they get good GC throughput out of it. rlh tried freeing garbage at thread (goroutine) exit if the memory wasn’t reachable from another thread at any point, which seemed promising to me but didn’t pan out. aclements did some very clever experiments with fast cryptographic hashing of pointers to allow new tradeoffs, but rlh even seemed doubtful the prospects of that approach in the long term.

          Compacting is a yet harder sell because they don’t want a read barrier and objects moving might make life harder for cgo users.

          Does seem likely we’ll see more work on more reliably meeting folks’ current expectations, like by fixing situations where it’s hard to stop a thread in a tight loop, and we’ll probably see work on reducing garbage through escape analysis, either directly or by doing better at other stuff like inlining. I said more in my long comment, but I suspect Java and Go have gone on sufficiently different paths they might not come back that close together. I could be wrong; things are interesting that way!

          1. 1

            Might be. I’m just going on what I know about the collector’s current state.

        2. 9

          Other comments get at it, but the two are very different internally. Java GCs have been generational, meaning they can collect common short-lived garbage without looking at every live pointer in the heap, and compacting, meaning they pack together live data, which helps them achieve quick allocation and locality that can help processor caches work effectively.

          ZGC is trying to maintain all of that and not pause the app much. Concurrent compacting GCs are hard because you can’t normally atomically update all the pointers to an object at once. To deal with that you need a read barrier or load barrier, something that happens when the app reads a pointer to make sure that it ends up reading the object from the right place. Sometimes (like in Azul C4 I think) this is done with memory-mapping tricks; in ZGC it looks like they do it by checking a few bits in each pointer they read. Anyway, keeping an app running while you move its data out from under it, without slowing it down a lot, is no easier than it sounds. (To the side, generational collectors don’t have to be compacting, but most are. WebKit’s Riptide is an interesting example of the tradeoffs of non-compacting generational.)

          In Go all collections are full collections (not generational) and no heap compaction happens. So Go’s average GC cycle will do more work than a typical Java collector’s average cycle would in an app that allocates equally heavily and has short-lived garbage. Go is by all accounts good at keeping that work in the background. While not tackling generational, they’ve reduced the GC pauses to more or less synchronization points, under 1ms if all the threads of your app can be paused promptly (and they’re interested in making it possible to pause currently-uncooperative threads).

          What Go does have going for it throughput-wise is that the language and tooling make it easier to allocate less, similar to what Coda’s comment said. Java is heavy on references to heap-allocated objects, and it uses indirect calls (virtual method calls) all over the place that make cross-function escape analysis hard (though JVMs still manage to do some, because the JIT can watch the app running and notice that an indirect call’s destination is predictable). Go’s defaults are flipped from that, and existing perf-sensitive Go code is already written with the assumption that allocations are kind of expensive. The presentation ngrilly linked to from one of the Go GC people suggests at a minimum the Go team really doesn’t want to accept any regressions for low-garbage code to get generational-type throughput improvements. I suspect the languages and communities have gone down sufficiently divergent paths about memory and GC that they’re not that likely to come together now, but I could be surprised.

          1. 1

            One question that I don’t have a good feeling for is: could Go offer something like what the JVM has, where there are several distinct garbage collectors with different performance characteristics (high throughput vs. low latency)? I know simplicity has been a selling point, but like Coda said, the abundance of options is fine if you have a really solid default.

            1. 1

              Doubtful they’ll have the user choose; they talk pretty proudly about not offering many knobs.

              One thing Rick Hudson noted in the presentation (worth reading if you’re this deep in) is that if Austin’s clever pointer-hashing-at-GC-time trick works for some programs, the runtime could choose between using it or not based on how well it’s working out on the current workload. (Which it couldn’t easily do if, like, changing GCs meant compiling in different barrier code.) He doesn’t exactly suggest that they’re going to do it, just notes they could.

            2. 1

              This is fantastic! Exactly what I was hoping for!

            3. 3

              There are decades of research and engineering efforts that put Go’s GC and Hotspot apart.

              Go’s GC is a nice introductory project, Hotspot is the real deal.

              1. 3

                Go’s GC designers are not newbies either and have decades of experience: https://blog.golang.org/ismmkeynote

                1. 1

                  Google seems to be the nursing home of many people that had one lucky idea 20 years ago and are content with riding on their fame til retirement, so “famous person X works on it” has not much meaning when associated with Google.

                  The Train GC was quite interesting at its time, but the “invention” of stack maps is just like the “invention” of UTF-8 … if it hadn’t been “invented” by random person A, it would have been invented by random person B a few weeks/months later.

                  Taking everything together, I’m rather unconvinced that Go’s GC will even remotely approach G1, ZGC’s, Shenandoah’s level of sophistication any time soon.

                2. 3

                  For me it is kind of amusing that huge amounts of research and development went into the Hotspot GC but on the other hand there seem to be no sensible defaults because there is often the need to hand tune its parameters. In Go I don’t have to jump through those hoops, and I’m not advised to, but still get very good performance characteristics, at least comparable to (in my humble opinion even better) than for a lot of Java applications.

                  1. 12

                    On the contrary, most Java applications don’t need to be tuned and the default GC ergonomics are just fine. For the G1 collector (introduced in 2009 a few months before Go and made the default a year ago), setting the JVM’s heap size is enough for pretty much all workloads except for those which have always been challenging for garbage collected languages—large, dense reference graphs.

                    The advantages Go has for those workloads are non-scalar value types and excellent tooling for optimizing memory allocation, not a magic garbage collector.

                    (Also, to clarify — HotSpot is generally used to refer to Oracle’s JIT VM, not its garbage collection architecture.)

                    1. 1

                      Thank you for the clarification.

                3. 2

                  I had the same impression while reading the article, although I also don’t know that much about GC.

                1. 1

                  Just finished reading the error handling and generics drafts. Really great work! Exactly what I hoped for the future of Go.

                  1. 8

                    The thread of security issues unveiled during the last few months in Intel CPU and similar architectures is an industrial nightmare. It’s difficult to accept that a whole industry could have been built on a such fragile basis…

                    1. 25

                      I mean, have you seen the software the world runs on?

                      1. 6

                        It’s difficult to accept that a whole industry could have been built on a such fragile basis…

                        See also car software.

                        1. 5

                          For me, it was easy after seeing how much better older stuff was that they ignored for money. Intel did try to make their own better stuff which failed repeatedly. They tried three times with i432, i960, and Itanium. Each had good and bad (i960 my favorite) but Intel was punished hard for all of them. Customers did buy billions in x86’s based on performance (most important), cost, and watts. Predictably, Intel decided to keep doing what people paid billions for instead of what cost them billions. I blame the buyers as much as them given I’ve tried to sell lots of them on secure, usable products that were free or cost almost nothing. Usually unsuccessful due to some aspect of human nature.

                          Like in most other tech products. It was surprising to me that Intel’s products weren’t worse than they are. Well, maybe they were as the bugs keep coming in as those assessing them predicted. They weren’t designed for strong security since market doesn’t pay for that. So, they have a lot of security problems. The hackers ignored them way longer than I thought they would, though.

                          1. 4

                            What shocks me most is how long we have been using these techniques without widespread awareness of these issues or the potential for this class of issues.

                            Some people predicted these problems, sure, but their concerns were mostly dismissed. So over the course of decades, we’ve seen other chip makers and architectures adopt the same techniques and thus enable these same bug classes.

                            For a young initiative like RISC-V, this is a great opportunity. They have not sunk years and years of development in features which may never be entirely safe to implement (speculative execution, hyperthreading, …) and are now able to take these new threats into account, quite early in their development. This could be a boon for industrial adoption, especially while many competitors are forced to rethink so many man-years of performance improvements.

                          1. 8

                            I prefer Vim’s approach of using JSON for communicating with external jobs instead of using Messagepack like Neovim does. I’m also not very partial about inventing a new binary format as a vimimfo replacement.

                            It is my understanding that Neovim wasn’t available on Windows for quite a while. I’m all for shedding features in order to facilitate building new ones, it might take Neovim in interesting new directions, but then the new project should not be considered a drop-in replacement for the old one.

                            (I even kind of like VimScript, despite its weirdness.)

                            1. 11

                              I’m also not very partial about inventing a new binary format as a vimimfo replacement.

                              viminfo is an ad-hoc unstructured format. shada is trivially parsable (and stream-able), because it’s just msgpack. It’s misleading to call it a “new binary format”, it’s just plain old msgpack with some sentinel values for guaranteed round-tripping to/from VimL.

                              It doesn’t make sense to prefer viminfo, which isn’t a format at all.

                              1. 1

                                Anyone using OniVim among us? I’m wondering how does it compare to vim, Atom, VSCode and Sublime Text?

                              1. 22

                                After writing Go for 5 years, I’d recommend Rust for C developers. It’s more complicated than Go for sure, but also has more to offer. The lack of garbage collection and support of generics are definitely a plus compared to Go.

                                Go is a better language for junior devs, but I wouldn’t call C programmers junior. They should be able to digest Rust’s complexity.

                                1. 9

                                  They should be able to digest Rust’s complexity.

                                  Non trivial amount of C programmers are still doing C to avoid additional complexity. Not everyone wants a kitchen & sink programming language.

                                  1. 6

                                    Rust can definitely get overly complex if the developers show no constraint (i.e. type golf), but the control afforded by manual memory management makes up for it, IMHO. Unless it’s a one-run project, performance will eventually matter, and fixing bad allocation practices after the fact is a lot harder than doing it right from the beginning.

                                    1. 1

                                      Couldn’t they just start with a C-like subset of Rust adding from there to their arsenal what extra features they like? It’s what I was going to recommend to those trying it for safety-critical use since they likely know C.

                                      1. 9

                                        I think it’s rather difficult to write rust in a C like manner. This contrasts with go, where you can basically write C code and move the type declarations around and end up with somewhat unidiomatic but working go.

                                        1. 3

                                          I think C++ as a better C works because you still have libc besides the STL, etc. The Rust standard library uses generics, traits, etc. quite heavily and type parameters and lifetime parameters tend to percolate to downstream users.

                                          Though I think a lot of value in Rust is in concepts that may initially add some complexity, such the borrow checker rules.

                                          1. 3

                                            The problem with C++ is its complexity at the language level. I have little hope of teams of people porting various tools for static analysis, verification, and refactoring to it that C and Java already have. Certifying compilers either. C itself is a rough language but smaller. The massive bandwagon behind it caused lots of tooling to be built, esp FOSS. So, I now push for low-level stuff either safer C or something that ties into C’s ecosystem.

                                          2. 4

                                            You could argue the same for C++ (start with C and add extra features). Complexity comes with the whole ecosystem from platform support (OS, arch), compiler complexity (and hence subtle difference in feature implementations) to the language itself (C++ templates, rust macros). It’s challenging to limit oneself to a very specific subset on a single person project, it’s exponentially harder for larger teams to agree on a subset and adhere to it. I guess I just want a safer C not a new C++ replacement which seems to be the target for newer languages (like D & Rust).

                                            1. 4

                                              It’s challenging to limit oneself to a very specific subset on a single person project, it’s exponentially harder for larger teams to agree on a subset and adhere to it.

                                              I see your overall point. It could be tricky. It would probably stay niche. I will note that, in the C and Java worlds, there’s tools that check source code for compliance with coding standards. That could work for a Rust subset as well.

                                              “I guess I just want a safer C not a new C++ replacement which seems to be the target for newer languages (like D & Rust).”

                                              I can’t remember if I asked you what you thought about Cyclone. So, I’m curious about that plus what you or other C programmers would change about such a proposal.

                                              I was thinking something like it with Rust’s affine types and/or reference counting when borrow-checking sucks too much with performance acceptable. Also, unsafe stuff if necessary with the module prefixed with that like Wirth would do. Some kind of module system or linking types to avoid linker errors, too. Seemless use of existing C libraries. Then, an interpreter or REPL for the productivity boost. Extracts to C to use its optimizing and certifying compilers. I’m unsure of what I’d default with on error handling and concurrency. First round at error handling might be error codes since I saw a design for statically checking their correct usage.

                                              1. 3

                                                I can’t remember if I asked you what you thought about Cyclone. So, I’m curious about that plus what you or other C programmers would change about such a proposal.

                                                I looked at it in the past and it felt like a language built on top of C similar to what a checker tool with annotations would do. It felt geared too much towards research versus use and the site itself states:

                                                Cyclone is no longer supported; the core research project has finished and the developers have moved on to other things. (Several of Cyclone’s ideas have made their way into Rust.) Cyclone’s code can be made to work with some effort, but it will not build out of the box on modern (64 bit) platforms).

                                                However if I had to change Cyclone I would at least drop exceptions from it.

                                                I am keeping an eye on zig and that’s closest to how I imagine a potentially successful C replacement - assuming it takes up enough community drive and gets some people developing interesting software with it.

                                                That’s something Go had nailed down really well. The whole standard library (especially their crypto and http libs) being implemented from scratch in Go instead of being bindings were a strong value signal.

                                                1. 2

                                                  re dropping exceptions. Dropping exceptions makes sense. Is there another way of error handling that’s safer or better than C’s that you think might be adoptable in a new, C-like language?

                                                  re Zig. It’s an interesting language. I’m watching it at a distance for ideas.

                                                  re standard library of X in X. Yeah, I agree. I’ve been noticing that pattern with Myrddin, too. They’ve been doing a lot within the language despite how new it is.

                                                  1. 4

                                                    Dropping exceptions makes sense. Is there another way of error handling that’s safer or better than C’s that you think might be adoptable in a new, C-like language?

                                                    Yes, I think Zig actually does that pretty well: https://andrewkelley.me/post/intro-to-zig.html#error-type

                                                    edit: snippet from the zig homepage:

                                                    A fresh take on error handling that resembles what well-written C error handling looks like, minus the boilerplate and verbosity.

                                                    1. 2

                                                      Thanks for the link and tips!

                                        2. 7

                                          Short build/edit/run cycles are appreciated by junior and senior developers alike. Go currently has superior compilation times.

                                          1. 10

                                            Junior and senior developers also enjoy language features such as map, reduce, filter, and generics. Not to mention deterministic memory allocation, soft realtime, forced error checking, zero-cost abstractions, and (of course) memory safety.

                                            1. 3

                                              Junior and senior developers also enjoy language features such as map, reduce, filter, and generics.

                                              Those are great!

                                              deterministic memory allocation, soft realtime, forced error checking, zero-cost abstractions, and (of course) memory safety.

                                              Where are you finding juniors who care about this stuff? (no, really - I would like to know what kind of education got them there).

                                              1. 8

                                                I cared about those things, as a junior. I am not sure why juniors wouldn’t care, although I suppose it depends on what kind of software they’re interested in writing. It’s hard to get away with not caring, for a lot of things. Regarding education, I am self-taught, FWIW.

                                              2. 1

                                                Map, reduce and filter are easily implemented in Go. Managing memory manually, while keeping the GC running, is fully possible. Turning off the GC is also possible. Soft realtime is achievable, depending on your definition of soft realtime.

                                                1. 1

                                                  Map, reduce and filter are easily implemented in Go

                                                  How? Type safe versions of these, that is, without interface{} and hacky codegen solutions?

                                                  1. 1

                                                    Here are typesafe examples for Map, Filter etc: https://gobyexample.com/collection-functions

                                                    Implementing one Map function per type is often good enough. There is some duplication of code, but the required functionality is present. There are many theoretical needs that don’t always show up in practice.

                                                    Also, using go generate (which comes with the compiler), generic versions are achievable too. For example like this: https://github.com/kulshekhar/fungen

                                                    1. 9

                                                      When people say “type safe map/filter/reduce/fold” or “map, reduce, filter, and generics” they are generally referring to the ability to define those functions in a way that is polymorphic, type safe, transparently handled by the compiler and doesn’t sacrifice runtime overhead compared to their monomorphic analogs.

                                                      Whether you believe such facilities are useful or not is a completely different and orthogonal question. But no, they are certainly not achievable in Go and this is not a controversial claim. It is by design.

                                                      1. 1

                                                        Yes, I agree, Go does not have the combination of type safety and generics, unless you consider code generation.

                                                        The implementation of generics in C++ also works by generating the code per required type.

                                                        1. 5

                                                          The implementation of generics in C++ also works by generating the code per required type.

                                                          But they are not really comparable. In C++, when a library defines a generic type or function, it will work with any conforming data type. Since the Go compiler does not know about generics, with go generate one can only generate ‘monomorphized’ types for a set of predefined data types that are defined an upstream package. If you want different monomorphized types, you have to import the generic definitions and run go generate for your specific types.

                                                          unless you consider code generation

                                                          By that definition, any language is a generic language, there’s always Bourne shell/make/sed for code generation ;).

                                                          1. 1

                                                            That is true, and I agree that go does not have support for proper generics and that this can be a problem when creating libraries.

                                                          2. 3

                                                            That’s why I said “transparently handled by the compiler.” ;-)

                                                            1. 0

                                                              I see your point, but “go generate” is provided by the go compiler, by default. I guess it doesn’t qualify as transparent since you have to type “go generate” or place that command in a build file of some sort?

                                                              1. 1

                                                                Yes. And for the reasons mentioned by @iswrong.

                                                                My larger point here really isn’t a technicality. My point is that communication is hard and not everyone spells out every point is precise detail, but it’s usually possible to infer the meaning based on context.

                                                                1. -1

                                                                  I think the even larger point is that for a wide range of applications, “proper” and “transparent” generics might not even be needed in the first place. It would help, yes, but the Go community currently thrives without it, with no lack of results to show for.

                                                                  1. 1

                                                                    I mean, I’ve written Go code nearly daily since before it was 1.0. I don’t need to argue with you about whether generics are “needed,” which is a pretty slimy way to phrase this.

                                                                    Seems to me like you’re trying to pick a fight. I already said upthread that the description of generics is different from the desire for them.

                                                                    1. -2

                                                                      You were the first to change the subject to you and me instead of sticking to the topic at hand. Downvoting as troll.

                                                2. 1

                                                  By superior, I guess you meant shorter?

                                                  1. 2

                                                    Compiling a very large go project with a cold cache might take a minute (sub-second once the cache is warm).

                                                    Compiling a fairly small rust app with a warm cache has taken me over a minute (I think it’s a little better than that now).

                                                    1. 1

                                                      Yes, and superior to Rust in that regard. Also the strict requirement to not have unused dependencies contributes to counteract dependency rot, for larger projects.

                                                1. 3

                                                  In the industry, you can optimize througput over latency because you produce the same thing over and over again. But in software development, you usually develop something new. If the software you need already exists, you just use it. You need an agile process because you develop something new, and you cannot plan everything ahead of time. Some issues are discovered along the road. Because of this, I don’t think the latency versus throughput trade-off is really relevant here.

                                                  1. 2

                                                    And yet, we do reinvent the wheel very often in software development. Sure, nobody writes the same program a million times but there are plenty of programmers who pump out CRUD web apps.

                                                    1. 3

                                                      But those CRUD apps are customized, and the customer paying for them may change the requirements. “Develop something new” doesn’t have to mean “develop something revolutionary” or even “develop something novel”, it just means something that doesn’t already exist.

                                                      To the extent that most CRUD apps share certain design characteristics, that’s why we’ve got LEGO programming or whatever people are calling it now. But even if you use a bunch of off-the-shelf components the customer can make decisions that result in the need to put the pieces together somewhat differently.

                                                  1. 1

                                                    Do you think it’s correct to claim that hashids are not reversible, if the attacker is unable to choose the plain text to be encoded? [1]

                                                    [1] In other words, the sequential IDs are generated server-side and the attacker has not control over them.

                                                    1. 17

                                                      I’ve heard the “binary logs are evil!!!” mantra chanted against systemd so many times that it wasn’t funny anymore. It’s a terrible argument. With so many big players putting their logs into databases, the popularity of the ELK stack, it is pretty clear that storing logs in non-plaintext format works. Way back in 2015, I wrote two blog posts about the topic.

                                                      The gist of it is that binary logs can be awesome, if put to good use. That the journald is not the best tool is another matter, but journald being buggy doesn’t mean binary logs are bad. It just means that the journald is possibly not the most carefully engineered thing out there. There are many things to criticize about systemd and the journal, and they both have their fair share of issues, but binary storage of logs is not one of them.

                                                      1. 10

                                                        Okay, so can we just assume all complaints about “binary logs” are just about these binary logs and get on with things?

                                                        The journald/systemd people don’t act like they have any clue what’s going on in the real world: people can’t use the tools they used to, and these tools evidently suck; Plain text sucked less, so what’s the plan to get anything better?

                                                        1. 8

                                                          I don’t think that’s entirely reasonable. It’s converting a complaint about principle (“don’t do binary logs”) into a complaint about practice, and that makes a big difference. If journald is a bad implementation of an ok idea, that requires very different steps to fix than if it’s a fundamentally bad idea.

                                                          What you’re describing makes sense for people on the systemd project to say (“woah, people hate our binary logs, maybe we should work on them”[0]), but not for the rest of us trying to understand things.

                                                          [0] I fear they’re not saying that, as they seem somewhat impervious to feedback

                                                          1. 2

                                                            I feel like @geocar is against binary logs as a source format, but not as an intermediate or analytics format. Even if your application uses structured logging, it can still be stored in a text file, for example as JSON, at the source. It can be converted to a binary log later in the chain, for example on a centralized logging server, using ELK, SQL, MongoDB, Splunk or whatever. The benefit is that you keep a lot of flexibility at the source (in terms of supporting multiple formats depending on the source application) and are still able to go back to the plain text log if you encounter a problem.

                                                            1. 4

                                                              I’m not even against binary logs “as a source format.”

                                                              Firstly: I recognise that “complaints about binary logs” is directed at journald and isn’t the same thing about complaints about logs in some non-text format.

                                                              I think getting systemd in deep forced sysadmins to retool on top of journald and that hurt a lot for so very little gain (if there was any gain at all- and for most workflows I suspect there wasn’t). This has almost certainly put people off of binary logs, and has almost certainly got people complaining about binary logs.

                                                              To that end: I don’t think those feelings around binary logs are misplaced.

                                                              Some humility is [going to be] required when trying to win people over with binary logs, but appropriating the term “binary logs” to include tools the sysadmin chooses is like pulling the rug out from under somebody, and that’s not helping.

                                                              1. 2

                                                                Thank you very much for clarifying. I agree that forcing sysadmin “to retool on top of journald” hurts.

                                                            2. 2

                                                              No, it’s recognising that when enough people are complaining about “the wrong thing”, telling them it’s the wrong thing doesn’t help them. It just causes them to dig in.

                                                              What’s the right thing?

                                                              I think that’s the point of the bug…

                                                            3. 1

                                                              Okay, so can we just assume all complaints about “binary logs” are just about these binary logs and get on with things?

                                                              As soon as the complaints start to be about journald and not “binary logs”, and the distinction is made explicit, yeah, we can. It’s been four years, so I’m not going to hold my breath.

                                                              and these tools evidently suck

                                                              For a lot of use cases, they do not suck. For many, they are a vast improvement over text logs.

                                                              what’s the plan to get anything better?

                                                              Stop logging unstructured text to syslog or stdout, and either log to files or to a database directly. Pretty much what you’ve been (or should have been) doing the past few decades, because both syslog and stdout are terrible interfaces for logs.

                                                              1. 9

                                                                As soon as the complaints start to be about journald and not “binary logs”, and the distinction is made explicit, yeah, we can. It’s been four years, so I’m not going to hold my breath.

                                                                People complain about things that hurt, and between Windows and journald it should not be a surprise that “binary logs” is getting the flak. journald has a lot of outreach work to do if they want to fix it.

                                                                For a lot of use cases, [the tools] do not suck. For many, they are a vast improvement over text logs.

                                                                And yet when programmers make mistakes implementing them, the sysadmin are left cleaning up after them.

                                                                Text logs have the massive material advantage that the sysadmin can do something with them. Binary logs need tools to do things, and the journald implementation has a lot of work to do.

                                                                Most of the “big players” use a transparent structuring layer rather than making binary logs their golden source of knowledge. This allows people to get a lot of the advantages of binary logs with few disadvantages (and given how cheap disk is, the price is basically zero).

                                                                Stop logging unstructured text to syslog or stdout, and either log to files or to a database directly. Pretty much what you’ve been (or should have been) doing the past few decades, because both syslog and stdout are terrible interfaces for logs.

                                                                These are directions to developers, not to sysadmins. Sysadmins are the ones complaining.

                                                                Are we really to interpret this as refuse to install any software that doesn’t follow this rule?

                                                                I’m willing to whack some perl together to get the text log data queryable for my business, but you give me a binary turd I need tools and documentation and advice.

                                                                1. 4

                                                                  Most of the “big players” use a transparent structuring layer rather than making binary logs their golden source of knowledge.

                                                                  What do you mean by a “transparent structuring layer”?

                                                                  1. 2

                                                                    Something to structure the plain text logs into some tagged format (like JSON or protocol buffers).

                                                                    Splunk e.g. lets users create a bunch of regular expressions to create these tags.

                                                                    1. 2

                                                                      Got it now. Thanks for clarifying!

                                                                  2. 0

                                                                    Text logs have the massive material advantage that the sysadmin can do something with them. Binary logs need tools to do things, and the journald implementation has a lot of work to do.

                                                                    For some values of “can do”, yes. Most traditional text logs are terrible to work with (see my linked blog posts, not going to repeat them here, again). Besides, as long as your journal files aren’t corrupt (which happens less and less often these days, I’m told), you can just use journalctl to dump the entire thing, and grep in the logs, just like you grep in text files. Or filter them first, or dump in JSON and use jq, and so on. Plenty of options there.

                                                                    Most of the “big players” use a transparent structuring layer rather than making binary logs their golden source of knowledge.

                                                                    Clearly our experience differs. Most syslog-ng PE customers (and customers of related products) made binary logs (either PE’s LogStore, or an SQL database) their golden source of knowledge. A lot of startups - and bigger businesses - outsourced their logging to services like loggly, which are a black box like binary logs.

                                                                    These are directions to developers, not to sysadmins. Sysadmins are the ones complaining.

                                                                    These are directions to sysadmins too. The majority of daemons support logging to files, or use a logging framework where you can set them up to log directly to a central collector, or to a database directly. For a huge list of applications, bypassing syslog has been there since day one. Apache, Nginx, pretty much any Java application can all do this, just to name a few things. There are some notable exceptions such as postfix which will always use syslog, but there are ways around that too.

                                                                    You can bypass the journal with most applications, some support that easily, some require a bit more work, but it has been doable by sysadmins all these years. I know, because I’ve done it without modifying any code.

                                                                    I’m willing to whack some perl together to get the text log data queryable for my business, but you give me a binary turd I need tools and documentation and advice.

                                                                    With the journal, you have journalctl, which is quite well documented.

                                                                    1. 2

                                                                      Clearly our experience differs. Most syslog-ng PE customers…

                                                                      Do you believe that syslog-ng has even significant market share of users responsible for logging? Even excluding SMB/VSMB?

                                                                      outsourced their logging to services like loggly, which are a black box like binary logs.

                                                                      I would be surprised to find that most people that use loggly don’t keep any local syslog files.

                                                                      What exactly are you arguing here?

                                                                      Plenty of options there.

                                                                      And?

                                                                      You can bypass the journal with most applications, some support that easily, some require a bit more work, but it has been doable by sysadmins all these years. I know, because I’ve done it without modifying any code.

                                                                      Right, and the goal is to get people using journald right?

                                                                      If journald doesn’t want to be used, what it’s reason for existing?

                                                                      1. 0

                                                                        Do you believe that syslog-ng has even significant market share of users responsible for logging? Even excluding SMB/VSMB?

                                                                        Yes.

                                                                        I would be surprised to find that most people that use loggly don’t keep any local syslog files.

                                                                        Most I’ve seen only keep local logs because they’re too lazy to clean them up, and just leave them to the default logrotate. In the past… six or so years, all loggly (& similar) users I worked with, never looked at their text logs, if they had any to begin with.

                                                                        Right, and the goal is to get people using journald right?

                                                                        For systemd developers, perhaps. I’m not one of them. I don’t mind the journal, because it’s been working fine for my needs. The goal is to show that you can bypass it, if you don’t trust it. That you can get to a state where your logs are processed and stored efficiently, in a way that is easy to work with - easier than plain text files. Without using the journal. But with it, it may be slightly easier to get there, because you can skip the whole getting around it dance for those applications that insist on using syslog or stdout for logging.

                                                                        1. 2

                                                                          Do you believe that syslog-ng has even significant market share of users responsible for logging? Even excluding SMB/VSMB?

                                                                          Yes.

                                                                          I think you’re completely wrong.

                                                                          There are a lot of Debian/RHEL/Ubuntu/*BSD (let alone Windows) machines out there, and they’re definitely not using syslog-ng by default…

                                                                          Debian publishes install information: syslog-ng verus rsyslogd. It’s no contest.

                                                                          A big bank I’m working with has zero: all rsyslogd or Windows.

                                                                          Also, the world is moving to journald…

                                                                          So, why exactly do you believe this?

                                                                          In the past… six or so years, all loggly (& similar) users I worked with, never looked at their text logs, if they had any to begin with.

                                                                          Most I’ve seen only keep local logs because they’re too lazy to clean them up, and just leave them to the default logrotate.

                                                                          Okay, but why do you think this contradicts what I say?

                                                                          You’re talking about people who have built a custom (text based!) logging system, streaming via the syslog protocol. The golden source was text files.

                                                                          The goal is to show that you can bypass it, if you don’t trust it.

                                                                          Ah well, this is a very different topic than what I’m replying to.

                                                                          I can obviously bypass it by not using it.

                                                                          I was simply trying to explain why people who complain about binary logging aren’t ignorant/crackpots, and are complaining about something important to them.

                                                                          1. 1

                                                                            I think you’re completely wrong.

                                                                            I think I know better how many syslog-ng PE customers there are out there (FTR, I work at BalaBit, who make syslog-ng). It has a significant market share. Significant enough to be profitable (and growing), in an already crowded market.

                                                                            A big bank I’m working with has zero: all rsyslogd or Windows.

                                                                            …and we have big banks who run syslog-ng PE exclusively, and plenty of other customers, big and small.

                                                                            Also, the world is moving to journald…

                                                                            …and syslog-ng plays nicely with it, as does rsyslog. They nicely extend each other.

                                                                            You’re talking about people who have built a custom (text based!) logging system, streaming via the syslog protocol. The golden source was text files.

                                                                            I think we’re misunderstanding each other… What I consider the golden source may be very different from what you consider. For me, the golden source is what people use when they work with the logs. It may or may not be the original source of it.

                                                                            I don’t care much about the original source (unless it is also what people query), because that’s just a technical detail. I don’t care much how logs get from one point to another (though I prefer protocols that can represent structured data better than the syslog protocol). I care about how logs are stored, and how they are queried. Everything else is there to serve this end goal.

                                                                            Thus, if an application writes its logs to a text file, which I then process and ship to a data warehouse, I consider that to be binary logs, because that’s how it will ultimately end up as. Since this warehouse is the interface, the original source can be safely discarded, once it shipped. As such, I can’t consider those the golden source.

                                                                            If we restricted “binary logs” to stuff that originated as binary from the application, then we should not consider the Journal to use binary logs either, because most of its sources (stdout and syslog) are text-based. If the Journal uses binary logs, then anything that stores logs as binary data should be treated the same. Therefore, everything that ends up in a database, ultimately makes use of binary logs. Even if their original form, or the transports they arrived there, were text.

                                                                            (Transport and storage are two very different things, by the way.)

                                                                            I was simply trying to explain why people who complain about binary logging aren’t ignorant/crackpots, and are complaining about something important to them.

                                                                            I never said they are. All I said is that storing logs in binary is not inherently evil, linked to blog posts where I explain pretty much the same thing, and give examples for how binary storage of logs can improve one’s life. (Ok, I also asserted that syslog and stdout are terrible interfaces for logs, and I maintain that. This has nothing to do with text vs binary though - it is about free-form text being awful to work with; see the linked blog posts for a few examples why.)

                                                                            1. 1

                                                                              I think I know better how many syslog-ng PE customers there are out there

                                                                              Or we just have different definitions of significant.

                                                                              Significant enough to be profitable (and growing), in an already crowded market.

                                                                              Look, I have an advertising business that makes enough money to be profitable, and is growing, but I’m not going to say I have a “significant” market share of the digital advertising business.

                                                                              But whatever.

                                                                              All I said is that storing logs in binary is not inherently evil

                                                                              And I didn’t disagree with that.

                                                                              If you try and re-read my comments knowing that, maybe it’ll be more clear what I’m actually pointing to.

                                                                              At this point, we’re just talking past each other, and there’s no point in that.

                                                              2. 2

                                                                Thanks for linking to the blog posts, they were most informative.

                                                              1. 1

                                                                There seems to be some unspoken assumption here that SQL is simpler/easier/cheaper than ML/AI. That’s not my experience: everything you can do with modern tools is possible in SQL, sure, but the modern tools make it much easier and eliminate a lot of the pitfalls.

                                                                1. 5

                                                                  I don’t follow: what kind of modern ML/AI tools are you thinking of (as simple/easy/cheap as SQL)?

                                                                  1. 1

                                                                    Mainly Spark. Hardware performance/cost is probably worse unless you’re on a really big dataset, but for me it more than made up for it in programmer cost: I found it so much easier to answer questions based on our data when I could use a shell in a normal programming language with access to our data as normal values and just call e.g. .aggregateByKey and pass code to do what I wanted.

                                                                    1. 5

                                                                      Calling aggregateByKey isn’t using ML, it’s using straightforward querying of a dataset. Spark isn’t an ML solution so much as it’s a non-relational data store that’s queried differently than SQL-based data stores.

                                                                      You can build ML solutions on top of Spark, much the same as you can on top of SQL.

                                                                      The big difference between the two is instead the declarative vs. imperative nature of querying. With SQL, you describe what you want, whereas with Spark (and a number of other big data and nosql stores), you describe how to get it. The latter is more familiar to many imperative/OO programmers, but the former is generally more approachable to non-programmers, and tends to deal with changes to data more smoothly.

                                                                      In fact, the declarative approach is useful enough that Spark SQL exists and is widely used.

                                                                      1. 1

                                                                        So maybe: “you don’t necessarily need ML/AI: you should consider just using SQL if you already know it.“?

                                                                        1. 1

                                                                          I still don’t follow: as far as I know, Spark is not a ML/AI tool.

                                                                    1. 9

                                                                      The article makes some good points; of course there’s not much technically interesting or novel in Docker.

                                                                      But the alternatives he suggests would be non-starters at every place I’ve worked, simply because some people insist on using Macs, which don’t support them. Docker let mac users finally join the containerization party (even if Docker for Mac secretly uses Linux behind the scenes anyway, ssshhh; don’t tell anyone.) I still think the story of “how a technically-inferior/uninteresting solution dominated the market by demonstrating that social/perception issues are usually the main thing that drives uptake” is more interesting than “Docker is bad, m’kay” takes anyway, and there’s a lot to learn there.

                                                                      1. 2

                                                                        Thanks for thinking of us, poor Mac users :-) I switched from Ubuntu to macOs a few years ago, and one of the things I missed was not being able to use Docker natively (without lauching a VM myself). The advent of Docker for Mac made things a lot easier (and standardized) and made me switch my main project from Vagrant to Docker.

                                                                      1. 9

                                                                        While the syntax is quite different, I think people really underappreciate how simple the Erlang syntax is. It takes no more than a few hours to learn everything about Erlang’s syntax. It has so few things about it that even if they are someone odd, it’s still so much smaller than almost any other language I’ve used. I think people tend to get way too caught up on its oddness rather than this strength of simplicity.

                                                                        1. 1

                                                                          I actually agree with you about stuff like that. Most programmers don’t, though. Being C-like helped C++ and Java win converts more easily. As in, you always get more adoption giving a crowd what they’re used to starting out.

                                                                          1. 1

                                                                            Also, the syntax is really unsurprising if you’ve toyed with Prolog in the past.

                                                                            1. 1

                                                                              The author explicitly mentions that as a negative given it’s a weird language hardly any mainstream coders would’ve messed with. Instantly creates an obstacle to adoption for the masses.

                                                                            2. 1

                                                                              Yes, I played a bit with Erlang some time ago, and I was really surprised by how beautiful and simple it is.

                                                                            1. 4

                                                                              What’s the meaning of the last line, “I showed up for him”?

                                                                              1. 13

                                                                                As used here, “him” implicitly means more than just “Steve Jobs.” It alludes to the essence of Jobs, the things that make him who he is. Carmack is saying he dropped everything he was working on when Jobs asked for him because of his admiration and respect for Jobs’ as a person, not just for Jobs’ fame or influence. Sentences like these are frequently written with “him” italicized, and spoken with strong emphasis on “him.”

                                                                                That’s how I read it anyway.

                                                                                Sorry if my explanation seems patronizing, I just went ahead and assumed you’re a non-native English speaker.

                                                                                1. 6

                                                                                  Thanks – not patronizing at all, even though I am in fact a native speaker :) Just not familiar with that phrase and unsure if I should take it literally.

                                                                                  1. 8

                                                                                    As a non-native speaker, that’s quite reassuring to know that English subtleties are deep, even for a native speaker :-)

                                                                                  2. 6

                                                                                    I read it as John showing up for Jobs’ funeral :-)

                                                                                1. 21

                                                                                  Gosh, I couldn’t make it very far into this article without skimming. It goes on and on asking the same ‘why’ but mentally answering it in the opposite direction of the quoted comments.

                                                                                  Docker is easy, standard isolation. If it falls, something will replace it. We’re not going in the opposite direction.

                                                                                  The article doesn’t explain to me what other ways I have of running 9 instances of an app without making a big mess of listening ports and configuration.

                                                                                  Or running many different PHP apps without creating a big mess of PHP installs and PHP-FPM configs. (We still deal with hosting setups that share the same install for all apps, then want to upgrade PHP.)

                                                                                  Or how to make your production setup easy to replicate (roughly) for developers who actually work on the codebase. (Perhaps on macOS or Windows, while you deploy on Linux.)

                                                                                  We’re not even doing the orchestration dance yet, these are individual servers that run Docker with a bunch of shell scripts to provision the machine and manage containers.

                                                                                  But even if we only use 1% of the functionality in Docker, I don’t know how to do that stuff without it. Nevermind that I’d probably have to create a Vagrantbox or something to get anyone to use it in dev. (I’ve come to dislike Vagrant, sorry to say.)

                                                                                  Besides work, I privately manage a little cloud server and my own Raspberry Pi, and sure they don’t run Docker, but they don’t have these requirements. It’s fine to not use Docker in some instances. And even then, Docker can be useful as a build environment, to document / eliminate any awkward dependencies on the environment. Makes your project that much easier to pick up when you return to it months later.

                                                                                  Finally, I’m sorry to say that my experiences with Ansible, Chef and Puppet have only ever been bad. It seems to me like the most fragile aspect of these tools is all the checks of what’s what in the current environment, then act on it. I’m super interested in trying NixOS sometime, because from what I gather, the model is somewhat similar to what Docker does: simply layering stuff like we’ve always done on software.

                                                                                  1. 1

                                                                                    For the php part it’s not that complex. Install the required versions (Debian and Ubuntu both have 5.6 through 7.2 “major” releases available side by side that’s to Ondrej Sury’s repo. Then just setup a pool per-app (which you should do anyway) and point to the apps specific Unix domain socket for php-fpm in the vhost’s proxy_fcgi config line.

                                                                                    I’ve used this same setup to bring an app from php5.4 (using mod_php) up through the versions as it was tested/fixed too.

                                                                                    Is there some config/system setup required? You betcha. Ops/sysadmins is part of running a site that requires more than shared hosting.

                                                                                    What are you gonna do with docker, have each developer just randomly writing whatever the fuck seems like a good idea and pushing their monolithic images to prod with no ops supporting it?

                                                                                    1. 12

                                                                                      What are you gonna do with docker, have each developer just randomly writing whatever the fuck seems like a good idea and pushing their monolithic images to prod with no ops supporting it?

                                                                                      Yes. The whole point of “DevOps”/docker is to deploy softwares certified by “Works on My Machine” certification program. This eliminates coordination time with separate Ops team.

                                                                                      1. 2

                                                                                        Is this sarcasm, or are you actually in favour of the definition “DevOps = Developers [trying to] do Ops” ?

                                                                                        1. 7

                                                                                          Descriptively, that’s what DevOps is. I am prescriptively against such DevOps, but describing what’s currently happening with docker is unrelated to whether I am in favor of it.

                                                                                          1. 3

                                                                                            I don’t disagree that it’s a definition used by a lot of places (whether they call it devops or not). But I believe a lot of people who wax poetic about “DevOps” don’t share this same view - they view it as Operations using ‘development’ practices: i.e. writing scripts/declarative state files/etc to have reproducible infrastructure, rather than a “bible” of manual steps to go through to setup an environment.

                                                                                            I’m in favour of the approach those people like, but I’m against the term simply because it’s misleading - like “the cloud” or “server less”.

                                                                                      2. 2

                                                                                        I don’t understand your last point, that’s exactly what developers do all day.

                                                                                        In Docker, the PHP version the app depends on is set in code. It doesn’t even take any configuration changes when the app switches to a new PHP version.

                                                                                        But if there’s one gripe I have with the Docker way of things, baking everything into an image, it’s security. There are no shared libraries in any way, upgrading a dependency minor version requires baking a new image.

                                                                                        I kinda wish we had a middle road, somewhere between Debian packages and Docker images.

                                                                                        1. 3

                                                                                          the PHP version the app depends on is set in code

                                                                                          And of course we all know Docker is the only way to define dependencies for software packages.

                                                                                          1. 4

                                                                                            Did anyone say it was? Docker is just one of the easiest ways to define the state of the whole running environment and have it defined in a text file which you can easily review to see what has been done.

                                                                                          2. 1

                                                                                            You can share libraries with Docker by making services share the same Docker image. You can actually replicate Debian level of sharing by having a single Docker image.

                                                                                            1. 2

                                                                                              Well, I guess this is just sharing in terms of memory usage? But what I meant with security is that I’d like if it were possible to have, for example, a single layer in the image with just OpenSSL, that you can then swap out with a newer version (with, say, a security fix.)

                                                                                              Right now, an OpenSSL upgrade means rebuilding the app. The current advantage managing your app ‘traditionally’ without Docker is that a sysadmin can do this upgrade for you. (Same with PHP patch versions, in the earlier example.)

                                                                                              1. 4

                                                                                                And this is exactly why I don’t buy into the whole “single-use” container shit show.

                                                                                                Want to use LXC/LXD for lightweight “VM’s”? Sure, I’m all for it. So long as ops can manage the infra, it’s all good.

                                                                                                Want to have developers having the last say on every detail of how an app actually runs in production? Not so much.

                                                                                                What you want is a simpler way to deploy your php app to a server and define that it needs a given version of PHP, an Apache/Nginx config, etc.

                                                                                                You could literally do all of that by just having your app packaged as a .deb, have it define dependencies on php-{fpm,moduleX,moduleY,moduleZ} and include a vhost.conf and pool.conf file. A minimal (i.e. non-debian repo quality but works for private installs) package means you’ll need maybe half a dozen files extra.

                                                                                                And then your ops/sysadmin team can upgrade openssl, or php, or apache, or redis or whatever other thing you use.

                                                                                                1. 2

                                                                                                  I actually do think this is a really good idea. But what’s currently there requires a lot more polish for it to be accessible to devs and small teams.

                                                                                                  Debian packaging is quite a pain (though you could probably skip a lot of standards). RPM is somewhat easier. But in both cases, the packages typically bundle default app configuration and systemd unit files, which is a model that sort of assumes things only have 1 instance.

                                                                                                  You could then go the LXC route, and have an admin manage each instance in a Debian container. That’s great, but we don’t have the resources to set up and manage all of this, and I expect that is the case for quite a lot of small teams out there.

                                                                                                  Maybe it’s less complicated than I think it is? If so, Docker marketing got something very right, and it’d help if there was a start-to-finish guide that explains things the other way.

                                                                                                  Also remember that Docker for Mac/Windows makes stuff really accessible for devs that are not on Linux natively. Not having to actually manage your VM is a blessing, because that’s exactly my gripe with Vagrant. At some point things inside the VM get hairy, because of organic growth.

                                                                                                  1. 3

                                                                                                    But in both cases, the packages typically bundle default app configuration and systemd unit files, which is a model that sort of assumes things only have 1 instance.

                                                                                                    In the case of the context - it is one instance. Either you build your packages with different names for different stages (e.g. acme-corp-foo-app-test, acme-corp-foo-app-staging, acme-corp-foo-app-prod) or use separate environments for test/stage/prod - either via VMs, LXC/LXD, whatever.

                                                                                                    Nothing is a silver bullet, Docker included. It’s just that Docker has a marketing team with a vested interest in glossing over it’s deficiencies.

                                                                                                    If you want to talk about how to use the above concept for an actual project, I’m happy to talk outside the thread.

                                                                                                    1. 2

                                                                                                      Also remember that Docker for Mac/Windows makes stuff really accessible for devs that are not on Linux natively. Not having to actually manage your VM is a blessing, because that’s exactly my gripe with Vagrant. At some point things inside the VM get hairy, because of organic growth.

                                                                                                      This is exactly why at work we started to use Docker (and got rid of Vagrant).

                                                                                                      1. 1

                                                                                                        At some point things inside the VM get hairy, because of organic growth.

                                                                                                        Can you define “hairy”?

                                                                                                        1. 2

                                                                                                          The VM becomes a second workstation, because you often SSH in to run some commands (test migrations and the like). So people install things in the VM, and change system configuration in the VM. And then people revive months old VMs, because it’s easier than vagrant up, which can take a good 20 minutes. There’s no reasoning about the state of Vagrant VMs in practice.

                                                                                                          1. 3

                                                                                                            So people install things in the VM, and change system configuration in the VM

                                                                                                            So your problem isn’t vagrant then, but people. Either the same people are doing the same thing with Docker, or not all things are equal?

                                                                                                            because it’s easier than vagrant up, which can take a good 20 minutes

                                                                                                            What. 20 MINUTES? What on earth are you doing that causes it to take 20 minutes to bring up a VM and provision it?

                                                                                                            There’s no reasoning about the state of Vagrant VMs in practice.

                                                                                                            You know the version of the box that it’s based on, what provisioning steps are configured to run, and whether they’ve run or not.

                                                                                                            Based on everything you’ve said, this sounds like blaming the guy who built a concrete wall, when your hammer and nails won’t go into it.

                                                                                                            1. 1

                                                                                                              I suppose the main difference is that we don’t build images for Vagrant, but instead provision the machine from a stock Ubuntu image using Ansible. It takes a good 3 minutes just to get the VirtualBox VM up, more if you have to download the Ubuntu image. From there, it’s mostly adding repos, installing deps, creating configuration. Ansible itself is rather sluggish too.

                                                                                                              Compare that to a 15 second run to get a dev environment up in Docker, provided you have the base images available.

                                                                                                              A people problem is a real problem. It doesn’t sound like you’ve used Docker for Mac/Windows, but the tool doesn’t give you a shell in the VM. And you don’t normally shell into containers.

                                                                                                              1. 1

                                                                                                                That’s interesting that it takes you 20 minutes to get to something usable. I never had that experience back when I used VMware and VirtualBox. I can’t remember having it anyway. I decided to see what getting Ubuntu up on my box takes with the new version for comparison to your experience. I did this experiment on my backup laptop: a 1.5GHz Celeron with plenty of RAM and older HD. It’s garbage far as performance goes. Running Ubuntu 16-17 (one of them…), VirtualBox, and Ubuntu 18.04 as guest in the a 1GB VM. That is, the LiveCD of Ubuntu 18.04 that it’s booting from.

                                                                                                                1. From power on to first Ubuntu screen: 5.7 seconds.

                                                                                                                2. To get to the Try or Install screen: 1 min 47 seconds.

                                                                                                                3. Usable desktop: 4 min 26 seconds.

                                                                                                                So, it’s up in under 5 minutes on the slowest-loading method (LiveCD) on some of the slowest hardware (Celeron) you can get. That tells me you could probably get even better startup time than me if you install and provision your stuff into a VirtualBox VM that becomes a base image. You use it as read-only, snapshot it, whatever the feature was. I rarely use VirtualBox these days so can’t remember. I know fully-loaded Ubuntu boots up in about a minute on this same box with the VirtualBox adding 5.7s to get to that bootloader. Your setup should just take 1-2 minutes to boot if doing it right.

                                                                                                                1. 0

                                                                                                                  It takes a good 3 minutes just to get the VirtualBox VM up

                                                                                                                  What? Seriously? Are your physical machines running on spinning rust or with only 1 or 2 GB of RAM or something? That is an inordinate amount of time to boot a VM, even in the POS that is Virtualbox.

                                                                                                                  but the tool doesn’t give you a shell in the VM.

                                                                                                                  What, so docker attach or docker exec /bin/bash are just figments of my imagination?

                                                                                                                  you don’t normally shell into containers

                                                                                                                  You don’t normally just change system settings willy nilly in a pre-configured environment if you don’t know what you’re doing, but apparently you work with some people who don’t do what’s “normal”.

                                                                                                                  1. 2

                                                                                                                    Physical machines are whatever workstation the developer uses. Typically a Macbook Pro in our case. Up until Vagrant has SSH access to the machine, I’m not holding my breath.

                                                                                                                    You’re confusing shell access to the VM with shell access to containers. The Docker commands you reference are for container access.

                                                                                                                    People do regularly make changes to vhost configuration, or installed packages in VMs when testing new features, instead of changing the provisioning configuration. Again, because it takes way longer to iterate on these things with VMs. And because people do these things from a shell inside the VM, spending time there, they start customizing as well.

                                                                                                                    And people do these things in Docker too, and that’s fine. But we’re way more comfortable throwing away containers than VMs, because of the difference in time. In turn, it’s become much easier to iterate on provisioning config changes.

                                                                                                                    1. 2

                                                                                                                      If time was a problem, sounds like the Docker developers should’ve just made VM’s faster in existing stacks. The L4Linux VM’s in Dresden’s demo loaded up about one a second on old hardware. Recently, LightVM got it down to 2.3 milliseconds on a Xen variant. Doing stuff like that also gives the fault-isolation and security assurances that only come with simple implementations which Docker-based platforms probably won’t have.

                                                                                                                      Docker seems like it went backwards on those properties vs just improving speed or usability of virtualization platforms.

                                                                                                                      1. 1

                                                                                                                        You’re confusing shell access to the VM with shell access to containers. The Docker commands you reference are for container access.

                                                                                                                        No. Your complaint is that people change configuration inside the provisioned environment. The provisioned environment with Docker isn’t a VM - that’s only there because it requires a Linux kernel to work. The provisioned environment is the container, which you’ve just said people are still fucking around with.

                                                                                                                        So your complaint still boils down to “virtualbox is slow”, and I still cannot imagine what you are doing to take twenty fucking minutes to provision a machine.

                                                                                                                        That’s closer to the time to build a base box from nothing than the time to bring up an instance and provision it.

                                                                                                                        1. 2

                                                                                                                          Look, this is getting silly. You can keep belittling every experience I’ve had, as if we’ve made these choices based on a couple of tiny bad aspects in the entire system, but that’s just not the case, and that’s not a productive discussion.

                                                                                                                          I did acknowledge that in practice Docker images a lot more things, which factors into a lot of the slowness of provisioning in the Vagrant case for us. There’s just a lot more provisioning has to do compared to Docker.

                                                                                                                          And while we could’ve gone another route, I doubt we would’ve been as happy, considering where we all are now as an industry. Docker gets a lot of support, and has a healthy ecosystem.

                                                                                                                          I see plenty of issues with Docker, and I can grumble about it all day. The IPv6 support is terrible, the process management is limited, the Docker for Mac/Windows filesystem integrations leave a lot to be desired, the security issue I mentioned in this very thread. But it still has given us a lot more positives than negatives, in terms of developer productiveness and managing our servers.

                                                                                                                          1. 1

                                                                                                                            You can keep belittling every experience I’ve had Every ‘issue’ you raised boils down to ‘vagrant+virtualbox took took to long to bring up/reprovision’. At 20 minutes, that’s not normal operation, it’s a sign of a problem. Instead of fixing that, you just threw the whole lot out.

                                                                                                                            This is like saying “I can’t work out why apache keeps crashing under load on Debian. Fuck it, I’m moving everything to Windows Server”.

                                                                                                                            But it still has given us a lot more positives than negatives The linked article seems to debunk this myth.

                                                                                                                          2. 2

                                                                                                                            I have the same experience as @stephank with VirtualBox. Every time I want to restart with a clean environment, I restart with a standard Debian base box and I run my Ansible playbooks on it. This is slow because my playbooks have to reinstall everything (I try to keep a cache of the downloaded packages in a volume on the host, shared with the guest). Docker makes this a lot easier and quicker thanks to the layer mechanism. What do you suggest to keep using Vagrant and avoid the slow installation (building a custom image I guess)?

                                                                                                                            1. 2

                                                                                                                              Please tell me “the same experience” isn’t 20 minutes for a machine to come up from nothing?

                                                                                                                              I’d first be looking to see how old the base box you’re using is. I’m guessing part of the process is an apt-get update && apt-get upgrade - some base boxes are woefully out of date, and are often hard-coded to use e.g. a US based mirror, which will hurt your update times if you’re elsewhere in the world.

                                                                                                                              If you have a lot of stuff to install, then yes I’d recommend making your own base-box.

                                                                                                                              What base-box are you using, out of interest? Can you share your playbooks?

                                                                                                                              1. 2

                                                                                                                                Creating a new VM with Vagrant just takes a few seconds, provided that the base box image is already available locally.

                                                                                                                                Provisioning (using Ansible in my case) is what takes time (installing all the services and dependencies required by my app). To be clear, in my case, it’s just a few minutes instead of 20 minutes, but it’s slow enough to be inconvenient.

                                                                                                                                I refresh the base box regularly, I use mirrors close to me, and I’ve already checked that apt-get update/upgrade terminates quickly.

                                                                                                                                My base box is debian/jessie64.

                                                                                                                                I install the usual stuff (nginx, Python, Go, Node, MySQL, Redis, certbot, some utils, etc.).

                                                                                                                                1. 2

                                                                                                                                  Reading all yours comments, you seem deeply interested by convincing people that VMs are solving all the problems people think Docker is solving. Instead of debating endlessly on comments here, I’d be (truly) interested to read about your work-flow as a an ops and as a dev. I’ve finished my studies using Docker and never had to use VMs that much on my machines, so I’m not an expert and would be really interested to have a good article/post/… that I could learn from on the subject on how VM would be better than Docker.

                                                                                                        2. 1

                                                                                                          I think the point is to use something like ansible, so you put some ansible config in a git repo then you pull the repo, build the docker image, install apps, apply the config and run, all via ansible.

                                                                                                        3. 2

                                                                                                          How do you manage easily 3 different versions of PHP with 3 different version of MariaDB? I mean, this is something that Docker solves VERY easily.

                                                                                                          1. 4

                                                                                                            Maybe if your team requires 3 versions of a database and language runtime they’ve goofed…

                                                                                                            1. 8

                                                                                                              It’s always amusing to have answers pointing the legacy and saying “it shouldn’t exist”. I mean, yes it’s weird, annoying but it exists now and will exists later.

                                                                                                              1. 6

                                                                                                                it exists now and will exists later.

                                                                                                                It doesn’t have to exist at all–like, literally, the cycles spent wrapping the mudballs in containers could be spent just…you know…cleaning up the mudballs.

                                                                                                                There are cases (usually involving icky third-party integrations) where maintaining multiple versions of runtimes is necessary, but outside of those it’s just plan sloppy engineering not to try and cleanup and standardize things.

                                                                                                                (And no, having the same container interface for a dozen different snowflakes is not standardization.)

                                                                                                                1. 2

                                                                                                                  I see it more like, the application runs fine, the team that was working on it doesn’t exist anymore, instead of spending time to upgrade it (because I’m no java 6 developer), and I still want to benefit from bin packing, re-scheduling, … (and not only for this app, but for ALL the apps in the enterprise) I just spend time to put it in a container, and voila. I still can deploy it in several different cloud and orchestrator without asking for a team to spend time on a project that already does the job correctly.

                                                                                                                  To be honest, I understand that containers are not the solution to everything, but I keep wondering why people don’t accept that it has some utility.

                                                                                                                2. 2

                                                                                                                  I think the point is that there is often little cost/benefit analysis done. Is moving one’s entire infrastructure to Docker/Kubernetes less work than getting all one’s code to run against the same version of a database? I’m sure sometimes it is, but my experience is that these questions are rarely asked. There is a status-quo bias toward solutions that allow existing complexity to be maintained, even when the solutions cost more than reducing that complexity.

                                                                                                                  1. 4

                                                                                                                    Totally agreed, but I’m also skeptical on the reaction of always blaming containers to add complexity. From my point of view, many things that I do with containers is way easier than if I had to do it another way (I also agree that some things would be easier without them too).

                                                                                                              2. 2

                                                                                                                Debian solves three different versions of php with Ondrej’s packages (or ppa on Ubuntu).

                                                                                                                In anything but dev or the tiniest of sites you’ll have you database server on a seperate machine anyway - what possible reason is there to have three different versions of a database server on the same host for a production environment?

                                                                                                                If you need it for testing, use lx{c,d} or vms.

                                                                                                                1. 3

                                                                                                                  Especially MySQL has broken apps in the past, going from 5.5 -> 5.6, or 5.6 -> 5.7. Having a single database server means having to upgrade all apps that run on top of it in sync. So in practice, we’ve been running a separate database server per version.

                                                                                                                  Can’t speak for other systems, though.

                                                                                                                  1. 1

                                                                                                                    As you said, testing is a good example of such use case. Then why using VMs when I can bin-pack containers on 1 (or many) machine, using less resources?

                                                                                                                    1. 1

                                                                                                                      That still isn’t a reason to use it in prod, and it isn’t that different from using LXC/LXD style containers.

                                                                                                                      1. 1

                                                                                                                        Do you have rational arguments to be against Docker which is using LXC? For now I don’t see any good reason not too. It’s like saying that you don’t want to use a solution because you can use the technologies it uses underneath.

                                                                                                                        1. 6

                                                                                                                          It’s like saying that you don’t want to use a solution because you can use the technologies it uses underneath.

                                                                                                                          That’s a reasonable position though. There are people who have good reasons to prefer git CLI to Github Desktop, MySql console to PHPMyAdmin, and so forth. Abstractions aren’t free.

                                                                                                                          1. 1

                                                                                                                            Exactly! But I don’t see such hatred for people using Github Desktop or PHPmyadmin. It’s not because you don’t want to use it that it doesn’t fit the usecase of someone.

                                                                                                                            1. 1

                                                                                                                              As someone who usually ends up having to ‘cleanup’ or ‘fix’ things after someone has used something like a GUI git client or PHPMyAdmin, I wouldn’t use the word hatred, but I’m not particularly happy if someone I work with is using them.

                                                                                                                              1. 1

                                                                                                                                I can do interactive staging on the CLI, but I really prefer a GUI (and if I find a good one, would probably also use a GUI for rebasing before sending a pull request).

                                                                                                                          2. 2

                                                                                                                            If I want a lightweight machine, LXC provides that. Docker inherently is designed to run literally a single process. How many people use it that way? No, they install supervisord or whatever - at which point, what’s the fucking point?

                                                                                                                            You’re creating your own ‘mini distribution’ of bullshit so you can call yourself devops. Sorry, I don’t drink the koolaid.

                                                                                                                            1. 1

                                                                                                                              Your argument is purely flawed. You justify the initially of Docker by generalizing what a (narrow) subset of users is doing. Like I said, I’m ready to hear rational arguments.

                                                                                                                              1. 2

                                                                                                                                generalizing what a (narrow) subset of users is doing

                                                                                                                                I found you 34K examples in about 30 seconds: https://github.com/search?l=&q=supervisord+language%3ADockerfile&type=Code

                                                                                                                                1. 1

                                                                                                                                  Hummm okay you got me on this one! Still, I really think there is some real utility for such a solution, even if yes it can be done in many other ways.

                                                                                                              1. 2

                                                                                                                On my Mac, I use MacPass (compatible with KeePass .kbd files) and I sync with Dropbox. On Android, I use KeePassDroid and Dropbox. I also use the password manager built-in in Chrome to sync passwords between my Mac and Android (to login in sites like Lobsters).

                                                                                                                1. 3

                                                                                                                  If you ever let your computer unlocked, it takes me 4 second to retrieve a passwords from chrome built-in password manager:

                                                                                                                  3-dot option button > settings > search for “pass” > click on the eye to see the password.

                                                                                                                  I did steal password this way, and anyone who can use a GUI can do it.

                                                                                                                  https://i.imgur.com/y7TkAy0.png

                                                                                                                  1. 3

                                                                                                                    Yes, I’m aware of this threat.

                                                                                                                    1. 1

                                                                                                                      Then I’m ok with it, carry on if you like.

                                                                                                                      1. 2

                                                                                                                        Just to make it clear:

                                                                                                                        • I don’t let my computer unlocked.
                                                                                                                        • Chrome for Mac, in Settings > Manage passwords, asks my system password before showing a website’s password.

                                                                                                                        According to your screenshot, it looks like you are not using a Mac, but at least, it looks like the threat you mentioned doesn’t exist on Mac.

                                                                                                                        1. 1

                                                                                                                          As you knew the vulnerability, you could block the threat.

                                                                                                                          I’ll recommend this way of doing for people storing passwords on chrome and Mac. It looks like they chose great defaults for this.

                                                                                                                          1. 2

                                                                                                                            Thanks!

                                                                                                                1. 3

                                                                                                                  Yes. 95% of all usages of K8S are plain silly. Like hunting mosquitos with machine guns.

                                                                                                                  https://www.youtube.com/watch?v=cZvT3MHpffk

                                                                                                                  “There’s nothing more dangerous than a half-configured Kubernetes cluster”

                                                                                                                  Nomad is way better (like one commenter already mentioned), but sometimes I dream of openbsd installations so I wouldn’t have to worry about any of this crap.

                                                                                                                  1. 1

                                                                                                                    but sometimes I dream of openbsd installations so I wouldn’t have to worry about any of this crap

                                                                                                                    How OpenBSD would change anything on this topic?

                                                                                                                    1. 1

                                                                                                                      Mainly that containers are non-existent.

                                                                                                                      1. 1

                                                                                                                        I get it now ;-)

                                                                                                                        Then how do you isolate from each other programs living on the same server, especially when they use different versions of the same dependencies? Do you use something similar to Solaris’ Zones, FreeBSD’s jails, or Nix?

                                                                                                                        1. 2

                                                                                                                          Yes, those are pretty much the questions I ask myself as well, and the the dreaming stops ;) OpenBSD has techniques that hover around these topics (like vmm and tons of admittedly cool low-level security features like pledge), but nothing really for containerisation (except chroot).

                                                                                                                          1. 2

                                                                                                                            I am sad. I hoped you had found something I haven’t to avoid Docker complexities ;-)

                                                                                                                  1. [Comment removed by author]

                                                                                                                    1. 6

                                                                                                                      The only thing you’ve done in this thread is fanned the flames. You might consider re-reading @pushcx’s comment above: https://lobste.rs/s/nf3xgg/i_am_leaving_llvm#c_pwiove

                                                                                                                      1. 3

                                                                                                                        Yep. The point of codes of conduct is to legitimize and give power to particular kinds of discriminatory thoughts and behaviors, and it’s working wonders, in this case by pushing away one core contributor.

                                                                                                                        1. 1

                                                                                                                          I’m trying to understand. What is the specific wording of llvm’s CoC that legitimizes particular kinds of discrimatory thoughts and behaviors?

                                                                                                                          1. [Comment removed by author]

                                                                                                                            1. 7

                                                                                                                              If you read the follow up messages it seems like a great deal was lost for the LLVM project.

                                                                                                                        1. 7

                                                                                                                          Massive kudos to this guy for not putting up with this SJW madness. I wish him all the best!

                                                                                                                          We at suckless are heavily opposed to code of conducts and discriminatory organizations of any shape or form.

                                                                                                                          1. 11

                                                                                                                            Suckless takes a similarly principled stand against runtime config files.

                                                                                                                            1. 8

                                                                                                                              How does suckless oppose discrimination?

                                                                                                                              1. 13

                                                                                                                                It’s very simple. Any non-technological matters during software development move the software away from its ideal form. Thus, to make your software suck less, you only take the best developers no matter what race, gender, heritage, etc. these persons have.

                                                                                                                                We do not believe in equal status (i.e. e.g. forcibly obtaining a 50/50 gender ratio), as this immediately leads to discrimination. We do however strongly believe in equal rights, naturally. You also naturally cannot have both.

                                                                                                                                1. 94

                                                                                                                                  Any non-technological matters during software development move the software away from its ideal form.

                                                                                                                                  Suckless makes a window manager: a part of a computer that human beings, with all their rich and varying abilities and perspectives, interact with constantly. Your choices of defaults and customization options have direct impact on those humans.

                                                                                                                                  For example, color schemes determine whether color-blind people are able to quickly scan active vs inactive options and understand information hierarchy. Font sizes and contrast ratios can make the interface readable, difficult, or completely unusable for visually impaired people. The sizes of click targets, double-click timeouts, and drag thresholds impact usability for those with motor difficulties. Default choices of interface, configuration, and documentation language embed the project in a particular English-speaking context, and the extent to which your team supports internationalization can limit, or expand, your user base.

                                                                                                                                  With limited time and resources, you will have to make tradeoffs in your code, documentation, and community about which people your software is supportive and hostile towards. These are inherently political decisions which cannot be avoided. This is not to say that your particular choices are wrong. It’s just you are already engaged in “non-technical”, political work, because you, like everyone else here, are making a tool for human beings. The choice to minimize the thought you put into those decisions does not erase the decisions themselves.

                                                                                                                                  At the community development level, your intentional and forced choices around language, schedule, pronouns, and even technical terminology can make contributors from varying backgrounds feel welcome or unwelcome, or render the community inaccessible entirely. These too are political choices. Your post above is one of them.

                                                                                                                                  There is, unfortunately, no such thing as a truly neutral stance on inclusion. Consider: you wish to take only the best developers, and yet your post has already discouraged good engineers from working on your project. Doubtless it has encouraged other engineers (who may be quite skilled!) with a similar political view to your own; those who believe, for instance, that current minority representation in tech is justified, representing the best engineers available, and that efforts to change those ratios are inherently discriminatory and unjust.

                                                                                                                                  Policies have impact. Consider yours.

                                                                                                                                  1. 7

                                                                                                                                    I don’t know if that was your goal, but this is one of the best arguments for positive discrimination I’ve read. Thanks for posting it, and also thanks for noting that all decisions have some inherent politics whether we like it or not.

                                                                                                                                    Unfortunately there is simply no solution: positive discrimination is opposed to meritocracy. Forced ratios are definitely an unethical tool, as they are a form of discrimination. However, this unethical tool brings us to a greater good, which is a final product that incorporates diversity on its design and accommodates more users, which is a desirable goal on itself, for the reasons you explained.

                                                                                                                                    1. 4

                                                                                                                                      color schemes determine whether color-blind people are able to quickly scan active vs inactive options and understand information hierarchy. Font sizes and contrast ratios can make the interface readable, difficult, or completely unusable for visually impaired people. The sizes of click targets, double-click timeouts, and drag thresholds

                                                                                                                                      Let me see if I understand what you’re saying. Are you claiming that when color schemes, font sizes and drag thresholds are chosen that that is a political decision? I think that many people would find that quite a remarkable claim.

                                                                                                                                      1. 3

                                                                                                                                        It’s impossible to not be political. You can be “the status quo is great and I don’t want to discuss it”, but that’s political. The open source “movement” started off political - with a strong point of view on how software economics should be changed. In particular, if you say a CoC that bans people from being abusive is unacceptable, you are making a political statement and a moral statement.

                                                                                                                                        1. 3

                                                                                                                                          It’s impossible to not be political

                                                                                                                                          Could I ask you to clarify in what sense you are using the word “political”?

                                                                                                                                          Merriam-Webster (for example) suggests several different meanings that capture ranges of activity of quite different sizes. For example, I’m sure it’s possible to act in a way which does not impinge upon “the art or science of government” but perhaps every (public) action impinges upon “the total complex of relations between people living in society”.

                                                                                                                                          In what sense did you use that term?

                                                                                                                                          1. 4

                                                                                                                                            Let’s start off with a note about honesty. FRIGN begins by telling us “We do not believe in equal status (i.e. e.g. forcibly obtaining a 50/50 gender ratio)” as if someone was proposing the use of force to produce a 50/50 gender ratio - and we all know that wasn’t proposed by anyone. There’s no way to discuss this properly if people are going to raise false issues like that. What comment’s like FRIGN’s indicate is an unwillingness to have an open and honest conversation. The same bogus rhetoric is at the heart of Damore’s memo: he claims to be in favor of equal rights and just against mythical demand for 50/50 gender equality so that he can oppose obviously ineffective affirmative action programs at Google where 80% of technical staff are male (Damore’s misappropriation of science is similarly based on an objection to a position that nobody ever argued.).

                                                                                                                                            The next point is that some people are objecting that a CoC and a minority outreach program are “political”. That’s true, but it involves the use of the more general meaning of “political” which the Collins dictionary provides as “the complex or aggregate of relationships of people in society, esp those relationships involving authority or power”. If we are using that definition, of course a CoC and a minority outreach program are political, but opposition to a CoC and a minority outreach program fits the definition as well. If you have an opinion one way or another, your opinion is political. You can’t sensibly use this wide definition of political to label the effort to adopt a CoC and to recruit more minorities and then turn around and claim your opposition to those is somehow not political. So that’s what I mean by “it is impossible to not be political”. The question is a political question and those who try to claim the high ground of being objective, disinterested, non-political for their side of the question are not being straightforward (perhaps it’s just that they are not being straightforward with themselves).

                                                                                                                                            1. 3

                                                                                                                                              I agree that a CoC, a minority outreach program, and opposition to a CoC all impinge upon “the complex or aggregate of relationships of people in society, esp those relationships involving authority or power”.

                                                                                                                                              Would you also agree that there is a popular ideological political movement in favour of CoCs (some combination of the feminist, civil rights and social justice movements)? Perhaps there is also a popular ideological movement against CoCs (some combination of MRAs and the alt right). Are you also claiming that if one claims a “neutral” stance on CoCs one is de facto supporting one of these ideologies?

                                                                                                                                              1. 3

                                                                                                                                                I’m not sure it is possible to have a neutral stance. In fact, I doubt it.

                                                                                                                                                1. 1

                                                                                                                                                  Interesting! Do you also doubt it is possible to take any action that is neutral with regard to a political ideology?

                                                                                                                                                  1. 3

                                                                                                                                                    You are introducing something different. I don’t think you have to line up with one “side” or another, but you can’t avoid being a participant.

                                                                                                                                                    1. 1

                                                                                                                                                      You said “It’s impossible to not be political” so I’m trying to understand what you mean by that. So far I’m not clear whether you think every action is political. I’d appreciate it if you’d clarify your position.

                                                                                                                                                      1. 2

                                                                                                                                                        I’m making a very concrete assertion, which I sense does not fit into your schema. My assertion is that there is no neutrality on workplace equality and inclusion for anyone involved in the workplace. Anyone who, for example, participates in an open source development effort has a position on whether efforts should be made to make it more inclusive even if that position is “this is not important enough for me to express an opinion.”

                                                                                                                                                        1. 1

                                                                                                                                                          Thank you for clarifying. When you originally said “It’s impossible to not be political” I got the wrong impression.

                                                                                                                                                          Do you also hold the same point of view when it comes to roughly comparable statements in other spheres? For example ‘Anyone who eats has a position on vegetarianism even if that position is “this is not important enough for me to express an opinion.”’?

                                                                                                                                      2. 1

                                                                                                                                        You’ve been quoted by LWN: https://lwn.net/Articles/753709/

                                                                                                                                      3. 11

                                                                                                                                        AKA shut up and hack? :)

                                                                                                                                        1. 1

                                                                                                                                          The suckless development process has no non-technical discussions?

                                                                                                                                          How are the best developers identified?

                                                                                                                                          1. 8

                                                                                                                                            just curious, why would you need to identify the best developers? Wouldn’t the quality of their code speak for that?

                                                                                                                                            1. 5

                                                                                                                                              I also fail to see what the reasoning is. Just send your code, get the non technical discussions out.

                                                                                                                                              1. -1

                                                                                                                                                Apparently, quoting @FRIGN from above, “to make your software suck less.”

                                                                                                                                              2. 8

                                                                                                                                                How are the best developers identified?

                                                                                                                                                I think this is a totally reasonable question, and one I’d like to see the answer too–if for no other reason than it might help those of us on other projects find more objective metrics to help track progress with.

                                                                                                                                                Do you all at suckless use something like:

                                                                                                                                                • defect rate
                                                                                                                                                • lines of code/feature shipped
                                                                                                                                                • execution time
                                                                                                                                                • space in memory, space in storage

                                                                                                                                                Like, what metrics do you use?

                                                                                                                                                1. 7

                                                                                                                                                  You know, suckless is not a big company and the metrics that can be applied are more of a heuristic. A good developer is somebody who e.g. supplies a patch with a bug report, provides feedback to commits, makes contributions to the projects, thinks his commits through and doesn’t break stuff too often and does not personally identify with their code (i.e. is not butthurt when it’s not merged).

                                                                                                                                                  What needs to be stressed here is that the metric “lines of code” is completely off. There are horrible programmers who spit out lots of code and excellent ones who over time drop more lines than they add. Especially the latter group is very present among us and thus the LOC-metric will only give false results. Same with execution time, you find that when not enough time is spent on a problem you end up solving it wrong, in the worst case having to start all over.

                                                                                                                                            2. 5

                                                                                                                                              By being very diverse and doing fackelmärsche of course. https://suckless.org/conferences/2017/

                                                                                                                                              1. 3

                                                                                                                                                @FRIGN What’s the purpose of this “torchlight hike” in the context of producing code that sucks less? Don’t you see that the activities you choose to have during your conferences are a cultural stance, and because of that, can be perceived as exclusive by programmers that don’t recognize themselves in these activities?

                                                                                                                                                1. 0

                                                                                                                                                  I get your point, but must honestly say that your argument sadly aligns with the ever-excluding and self-segregating destructful nature of cultural marxism. By eating food together at the conferences, do we exclude anorexics that might otherwise be willing to attend such a conference? I don’t drink any alcohol and never have. Still, it was not a problem when we went to a local Braukeller and some people drank alcohol and others like myself didn’t.

                                                                                                                                                  The fundamental point I think is that one can never fully and analytically claim that a certain process is completely unaffected by something else. If we dive down into these details we would then move on and say that the different choice of clothings, hairstyle, means of travel and means of accomodation all affect the coding process at suckless. This can be taken further and further with no limit, as we all know about the butterfly effect. At some point it is just not measurable any more.

                                                                                                                                                  If you ask me, this is a gross overstretching of what I said. There are quite a lot of people who do not attend the conferences but still work together with us on projects during that time. What really matters is that we e.g. do not ignore patches from these people or give them less relevance than those of others. To pick the example up: The torchlight hike did not affect any coding decision in a direct way, but it really bonded the team further together and was a very nice memory of this conference that I and the others are very fond of from what I’ve heard. On top of that, during the hike we were able to philosophize about some new projects of which some have become a reality. The net-gain of this event thus was positive.

                                                                                                                                                  In classical philosophy, there are two main trains of thought when it comes to evaluating actions: Deontology and Teleology. Deontology measures the action itself and its ethical value, completely ignoring the higher goal in the process. Teleology is the opposite, evaluating actions only by their means to reach a goal, completely ignoring the value of the action itself. The best approach obviously should be inbetween. However, there is a much more important lesson that can be taken from here: When evaluating a decision, one needs to realize what they are measuring and what is unimportant for a decision. What I meant is that to reach the goal of software perfection, the gender and other factors of the submitters do not matter. So even though we here at suckless have a goal, we are not teleologists, as we just ignore the factors that do not matter for coding.

                                                                                                                                                  It is an ethical question which norms you apply to a decision.

                                                                                                                                                  If we look at organizations like Outreachy, one might be mistaken to think that they are deontologists, striving to improve processes. However, after closer inspection it becomes clear that this is not the case and they are actually working towards a certain goal, increasing the number of trans and minority people in such communities. No matter how you think about this goal, it makes one thing clear: When you are working towards such a goal and also do not ignore irrelevant factors in your norms (and they in fact do by not ignoring e.g. race and gender), you quickly end up discriminating against people.

                                                                                                                                                  I hope this clears this up a bit, but as a short sentence, what can be taken from here is: When discussing ethical matters, it’s always important to make clear which norms are applied.

                                                                                                                                                  1. 2

                                                                                                                                                    fackelmärsche

                                                                                                                                                    I’m not going to wade into anything else on this, but I’d like to just take a second and let you know that, while you may not mean it in this way the phrase “cultural marxism” is very, very often used as a stand in for “jews”. Some links for the record:

                                                                                                                                                    https://www.splcenter.org/fighting-hate/intelligence-report/2003/cultural-marxism-catching

                                                                                                                                                    https://newrepublic.com/article/144317/trumps-racism-myth-cultural-marxism https://www.smh.com.au/world/cultural-marxism--the-ultimate-postfactual-dog-whistle-20171102-gzd7lq.html

                                                                                                                                                    1. 3

                                                                                                                                                      It’s not my fault that some idiots don’t understand this term or it’s critical analysis. Cultural marxism, as the term implies, is the classical theory of marxism applied to culture. It has nothing to do with jews directly, it’s just an idea. If you know any better term to describe it, please let me know.

                                                                                                                                                      Anyway, in the philosophical realms it’s known as ‘Critical Theory’, which originated in the Frankfurt School. However, nobody knows this term.

                                                                                                                                                      Unless a better term is found, I disregard your argument and won’t accept your attempt to limit language of perfectly acceptable words to describe an idea. At the end of the day, terminology must be found that adequately describes what a certain idea is, and I see no reason why this should be wrong.

                                                                                                                                                      Regarding the torch hike: Yes, marching with torches was abused by the NSDAP as a means of political rallying. However, at least in Germany, it is a much older and deeper-reaching tradition that dates back hundreds of years.

                                                                                                                                                      1. 0

                                                                                                                                                        You have amply demonstrated that you don’t know anything about the topic. You could start with the decent Wikipedia article. https://en.wikipedia.org/wiki/Frankfurt_School

                                                                                                                                                      2. 2

                                                                                                                                                        wow, uh, kind of a weird red flag that pointing this out is getting seriously downvoted. I picked these links pretty quickly, and anybody who comes behind and reads this and wonders how serious this is, do yourself a favor and image search and see how many memes have the star of david, greedy merchant, world strangling octopus or any of a number of openly anti-semitic imagery. Its not hidden, its not coy. If you’re tossing “cultural marxism” around you’re either willfully ignoring this or blatantly playing along. Its not a thing in the world. There are no leftists (at all) who call themselves “cultural marxists”, and in fact there is a sizeable faction of marxists who are openly disdainful of any marxism that eschews political struggle. The new republic article linked above goes into this, Perry Andersons “Considerations on Western Marxism”, a well known, well regarded text across a number of marxist subsects, is explicitly based on this. Anyway, enjoy contributing to a climate of increasing hostility toward jews. good stuff.

                                                                                                                                                        edit: have some fun with this https://www.google.com/search?q=cultural+marxism&client=firefox-b&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjz2tWrhvnaAhUJ7YMKHVgcCccQ_AUIDCgD&biw=1247&bih=510#imgrc=_

                                                                                                                                                        1. 1

                                                                                                                                                          The term ‘Cultural Marxism’ describes very well what it is, and not all leftists are cultural marxists. The classical theory of marxism, roughly spoken, is to think of society as being split in two camps, the Proletariat and the Bourgeoisie, eternally involved in a struggle, where the former is discriminated against and oppresed by the latter.

                                                                                                                                                          Cultural Marxism applies these ideas to society. In the Frankfurt School it was called ‘Critical Theory’, calling people out to question everything that was deemed a cultural norm. What is essentially lead to was to find oppressors and oppressed, and we reached the point where e.g. the patriarchy oppressed against women, white people against minorities, christians against muslims and other religions and so forth. You get the idea. Before you go again rallying about how I target jews or something please take a note that up to this point in this comment, I have just described what cultural marxism is and have not evaluated or criticized it in any way, because this here is the wrong platform for that.

                                                                                                                                                          What you should keep in mind is that the nature of cultural marxism is to never be in a stable position. There will always be the hunt for the next oppressor and oppressed, which in the long run will destroy this entire movement from the inside. It was a friendly advice from my side to you not to endulge in this separatory logic, but of course I understand your reasoning to the fullest.

                                                                                                                                                          Just as a side note: I did not see you getting ‘seriously’ downvoted. What do you mean?

                                                                                                                                                          1. 2

                                                                                                                                                            It’s uncommon to find such a well-put explanation; thanks for that.

                                                                                                                                                            There will always be the hunt for the next oppressor and oppressed, which in the long run will destroy this entire movement from the inside.

                                                                                                                                                            If the movement runs out of good targets (and falls apart because they can’t agree on new ones), wouldn’t that imply that it will self-destruct only after it succeeds in its goals? That doesn’t sound like a bad thing.

                                                                                                                                                            1. 1

                                                                                                                                                              I’m glad you liked my explanation. :)

                                                                                                                                                              That is a very interesting idea, thanks for bringing this thought up! It’s a matter dependent on many different factors, I suppose. It might fall apart due to not being able to agree on new targets or when everybody has become a target, but it is a very theoretical question which one of these outcomes applies here.

                                                                                                                                                            2. 1

                                                                                                                                                              Did you actually read any of the links I posted? Specifically the New Republic and SPLC links? I don’t know how else to say this and you pretty much side stepped what I said the first time so I’ll try to reiterate it: There is no such thing as “Cultural Marxism”. At all. Its not a descriptive category that any marxist actually self applies or applies to other marxists. I’m fully aware of the Frankfurt School, Adorno, Horkheimer, etc. I’ve read some of them and many, many of their contemporaries from Germany, people like Karl Mannheim. I read marxist publications everyday, from here in the states and from Europe. I’m a member of an explicitly marxist political party here in the states. I can’t emphasize this enough, “cultural marxism” isn’t real and is roughly on par with “FEMA camps”, “HARRP rays” and shape shifting lizard jews, meaning; its a far far right wing paranoid fantasy used to wall off people from other people and an actual understanding of the material conditions of their world. I also didn’t say, specifically in fact pointing out that I wasn’t saying this, that you were “targeting jews”. That being said, if you use a phrase that has its origins in anti-semitic polemics, is used explicitly and over-whelmingly by anti-semites, than that is on you. (Did you take a look at the linked image search? Does that sort of thing not give you pause?) To say that you “just described what cultural marxism is” is also inaccurate, you absolutely used it in a descriptive way

                                                                                                                                                              I get your point, but must honestly say that your argument sadly aligns with the ever-excluding and self->segregating destructful nature of cultural marxism.

                                                                                                                                                              White supremacist organizing is experiencing an enormous upsurge, not only here in the states but in Europe as well. From Le Pen to AfD to SVO in Austria and on and on. These people are not interested in polite conversation and they’re not using “cultural marxism” as a category to illuminate political opponents, its meant to denigrate and isolate, ironically given thats exactly what Neo Nazis and white supremacists here in the states accuse left wingers and “SJWs” of doing.

                                                                                                                                                              I appreciate that you’re discussing this peacefully but I’m going to bow out of this thread unless you’re interested enough to take some time and read the links

                                                                                                                                                              FWIW these also dismantle the trope and point out pretty much exactly what I’m saying around anti-semitism: https://www.vice.com/en_us/article/78mnny/unwrapping-the-conspiracy-theory-that-drives-the-alt-right https://www.theguardian.com/commentisfree/2016/feb/22/chris-uhlmann-should-mind-his-language-on-cultural-marxism

                                                                                                                                                              1. 2

                                                                                                                                                                I took some more time to read it up and from what I could see, I found that indeed cultural marxism has become more of a political slogan rather than a normal theoretical term in the USA.

                                                                                                                                                                Here in Germany the term “Kulturmarxismus” is much less politically charged from what I can see and thus I was surprised to get this response after I just had “translated” this term into English. It might be a lesson to first get some background on how this might be perceived internationally, however, it is a gigantic task for every term that might come around to you.

                                                                                                                                                                So to reiterate my question, what term could be better used instead? :)

                                                                                                                                                                1. 1

                                                                                                                                                                  interesting that it has a different grounding/connotation in Germany, but then again I’m not surprised since thats where its supposed to have originated from. I’ll reread your other posts and come up with a response thats fair. Thanks for taking the time to read those links.

                                                                                                                                                              2. 1

                                                                                                                                                                Generally people who use “cultural marxism” as a pejorative are sloganeering. The idea of an “eternal struggle” is completely foreign to any kind of marxism which is based on a theory that classes come out of the historical process and disappear due the historical process. Marxism claims that the proletariat and bourgeosie are temporary divisions that arise from a certain type of economic organization. Whatever one thinks of that idea, your characterization of Marxism is like describing baseball as a game involving pucks and ice. Your summary of “cultural marxism” is even worse. Maybe take a class or read a decent book.

                                                                                                                                                  2. 17

                                                                                                                                                    I’m not going to remove this because you’re making a public statement for suckless, but please don’t characterize positions you disagree with as madness. That kind of hyperbole generally just leads to unproductive fights.

                                                                                                                                                    1. 9

                                                                                                                                                      Please don’t remove anything unless it’s particularly vulgar…

                                                                                                                                                      1. [Comment removed by author]

                                                                                                                                                        1. 3

                                                                                                                                                          hey that’s my account you’re talking about!

                                                                                                                                                      2. -1

                                                                                                                                                        Removing differing viewpoints? It is precisely this kind of behavior that maddens people who complain about SJW, who (the SJW) seem unable to take any discussion beyond calling their opponent’s position “evil”, “alt-right”, “neo-nazi”, or, if they are exceptionally well-spoken, “mad”.

                                                                                                                                                        1. 14

                                                                                                                                                          No, removing abuse and hyperbole that acts as flamebait regardless of the political opinions expressed. So far I’ve removed one post and hope not to remove more.

                                                                                                                                                          1. 2

                                                                                                                                                            It’s hard for me to see a reason to remove things when we have the voting system in place, neither are perfect but one is at your sole discretion whereas the other is the aggregate opinion of the users.

                                                                                                                                                            1. 21

                                                                                                                                                              Voting isn’t a replacement of moderation. It helps highlight and reward good comments and it can punish bad comments, but it’s not sufficient for running a community. I’m trying to head off places where people give up on argument and just try to hurt or tar the people they disagree with because it doesn’t lead to a good community. Lobsters is a very good place for discussing computing and I haven’t seen that in communities this size with hands-off moderation (but I’d love counter-examples to learn from!) From a quick query, we’ve had comments from 727 unique users in the last 30 days and there’s around 15k unique IPs in the logs per weekday, so people are constantly interacting with the others who don’t know their background, don’t share history, can’t recognize in-jokes, simply don’t have reason to trust when messages are ambiguous, let alone provocative. Friendly teasing like “ah yeah, you would think that” or “lol php sucks” that’s rewarding bonding in a small, familiar group hurts in a big one because even if the recipient gets the joke and laughs along or brushes it off as harmless, it’s read by thousands of people who don’t or can’t.

                                                                                                                                                              1. 2

                                                                                                                                                                Lobsters is a very good place for discussing computing and I haven’t seen that in communities this size with hands-off moderation

                                                                                                                                                                I support your position on sub-topic but even my Trial you linked to shows a bit otherwise on just this point. This site has more flexible, hands-off moderation than many I’ve seen with this much political dispute. Even in that link, we saw an amount of honest, civility, and compromise I don’t usually see. There’s been quite a bit better results in this thread than usual elsewhere. There seems to be enough community closeness despite our size that people are recognizing each others positions a bit. Instead of comments, you can actually see it by what’s not said more since it’s prior ground we’ve covered. The others are learning as discussion furthers. Then, there’s the stuff we don’t want which seems to be basically what those individuals are intending in a way that has nothing to do with site’s size.

                                                                                                                                                                So, I support you getting rid of just pure abuse, trolling, sockpuppeting, etc. I don’t think we’ve hit the full weaknesses and limited vision of large sites yet despite our increase in comments and views. We’re still doing a lot better than average. We’re still doing it with minimal intervention on things like politics relative to what I’ve seen elsewhere. I think we can keep at current moderation strategy for now because of that. For now.

                                                                                                                                                                Just wanted to say that in the middle of all this.

                                                                                                                                                                1. 0

                                                                                                                                                                  Voting isn’t a replacement of moderation. It helps highlight and reward good comments and it can punish bad comments, but it’s not sufficient for running a community.

                                                                                                                                                                  I’m not sure if I see why it’s not a good replacement. To me, I see voting as distributed moderation and the “real” moderation is automatically hiding (not removing) comments when they fall below a threshold.

                                                                                                                                                                  I’m trying to head off places where people give up on argument and just try to hurt or tar the people they disagree with because it doesn’t lead to a good community.

                                                                                                                                                                  I think this method relies on an accurate crystal ball where you can foresee people’s actions and to an extent, the reactions of the people reading the comments.

                                                                                                                                                                  I’d have to question what you mean by “a good community”, it seems like it’s just a place where everyone agrees with what you agree with and those that disagree aren’t heard because it risks offending those that do agree.

                                                                                                                                                                  I think the best discussions on here are because we have many people with wide and varied opinions and backgrounds. The good comes from understanding what someone else is saying, not excluding them from the discussion. The only places I see that warranted is where someone has said something purposely and undeniably vile.

                                                                                                                                                                  1. 8

                                                                                                                                                                    The automatic hiding of low-scoring comments is also a “sole discretion” thing; jcs added it and I tweaked it a few months ago. The codebase enforces a lot of one moderator’s ideas of what’s good for a community in a hands-off way and the desire to do that motivated its creation.

                                                                                                                                                                    I strongly agree that a community where everyone agrees with the moderator would be bad one, even if I am that moderator. It’s tremendously rewarding to understand why other people see things differently, if for no other reason than the selfish reason that one can’t correct learn or correct mistakes if one never sees things one doesn’t already agree with.

                                                                                                                                                                    I think the crystal ball for foreseeing problems is experience, from many years of reading and participating in communities as they thrive or fail. I think it’s possible to recognize and intervene earlier than the really vile stuff because I’ve seen it work and I’ve seen its absence fail. I keep asking for examples of excellent large communities without active moderators because I haven’t seen those, and after a couple decades and a few hundred communities I see the anthropic principle at work: they don’t exist because they self-destruct, sink into constant vileness, or add moderation. At best they have maintain with signal-to-noise ratios far below that of Lobsters where the thoughtful commentary is crowded out by trolling, running jokes, ignorance, and plan low-quality comments because it doesn’t seem worth anyone’s while to care when posting.

                                                                                                                                                                    But moderation is not a panacea in and of itself. Without good experience, judgment, and temper a bad moderator swiftly destroys a community, and this is a very common way communities fail. If it helps any, the author of the comment I removed agrees that it wasn’t done to suppress their opinion.

                                                                                                                                                                    1. 1

                                                                                                                                                                      The benefit I see from moderation being part of the codebase is that it’s public, predictable and repeatable (it terms of reliability). When you take moderation decisions into your own discretion many of these virtues are lost.

                                                                                                                                                                      As for experience, I think that’s tricky because it can easily lead you to making the same mistake twice. It’s also made of your personal experiences and you’re using that to curate the discussion of other people, I would caution that it’s another method of controlling dialog (perhaps subconsciously) to what you find acceptable, not necessarily what’s best for everyone.

                                                                                                                                                                      1. 3

                                                                                                                                                                        The benefit I see from moderation being part of the codebase is that it’s public, predictable and repeatable (it terms of reliability). When you take moderation decisions into your own discretion many of these virtues are lost.

                                                                                                                                                                        Most of them go into the Moderation Log. I’ve been watching it since the jcs days since it’s what folks are supposed to do in a transparent, accountable system. Gotta put effort in. I haven’t seen much of anything that bothered me. The bans and deletes I’ve been able to follow @pushcx doing were trolling, alleged sockpuppeting, and vicious flamewaring. Some I couldn’t see where I’d rather the resource go off the front page rather getting deleted so someone looking at logs could see it for whatever it was. Nonetheless, his actions in the thread about me, the general admining, and what I’ve seen in moderation have been mostly good. A few really good like highlighting the best examples of good character on the site. I think he’s the only one I’ve seen do that on a forum in a while.

                                                                                                                                                                        You have little to worry about with him in my opinion at the moment. Do keep an eye on the comments and log if you’re concerned. Scrape them into version storage if concerned about deletions. What goes on here is pretty public. Relax or worry as much as you want. I’m more relaxed than worried. :)

                                                                                                                                                                        1. 3

                                                                                                                                                                          Yeah, I agree on the pitfalls of experience. As SeanTAllen noted in a separate branch of this thread a minute ago, there’s “but you didn’t say” and other wiggle room; I think that’s where automatic moderation falls down and human judgment is required. Voting has its own downsides like fads, groupthink, using them to disagree (which is all over this thread), in-jokes, a drifting definition of topicality, all the parallels to the behaviors of political rhetoric, etc. Lobsters has never been voting only and I don’t see a compelling reason to change that. jcs’s involvement in the site was steadily declining so I’m certainly more actively moderating, but I don’t see that as a change in character. I guess what it comes down to is that I agree with you about what successful communities do and don’t look like, but I haven’t seen one that works on the model you’ve outlined and I don’t see that kind of fundamental change as a risk worth taking.

                                                                                                                                                              2. 1

                                                                                                                                                                So FRIGN writes to oppose “SWJ madness”, and you chime in to complain that “SWJ” calls opponents “mad”. Are you calling FRIGN “SWJ” or what? It’s kind of hard to discern your point in that cloud of grievance.

                                                                                                                                                                1. 1

                                                                                                                                                                  “SJW” for “social justice warrior.”

                                                                                                                                                                  @COCK is sarcastically non-replying because you typo’ed.

                                                                                                                                                                  1. 2

                                                                                                                                                                    Not exactly, I was sarcastically non-replying because I assumed he was intentionally misunderstanding me. I assumed this because I didn’t see any ambiguity in my answer. On later inspection I noticed the ambiguity so I gave an actual reply:

                                                                                                                                                                    https://lobste.rs/s/nf3xgg/i_am_leaving_llvm#c_yzwuux

                                                                                                                                                                    1. 1

                                                                                                                                                                      The interesting thing is how people agreeing with Mr. cock pile on the insults against the people who they complain are insulting them by forcing them to sign on to codes of conduct which prohibit insults. It’s almost as if there was a good reason for those codes.

                                                                                                                                                                      1. 1

                                                                                                                                                                        I doubt the irony is lost on anyone supporting a CoC.

                                                                                                                                                                    2. -1

                                                                                                                                                                      Yes, I’m calling FRIGN a “SWJ”.

                                                                                                                                                                      1. -1

                                                                                                                                                                        Yes, well, one sympathizes with your plight.

                                                                                                                                                                        1. 2

                                                                                                                                                                          Ah now I see the ambiguity: “people who complain about SJW, who…” the “who” referred to the “SJW”, not the “people”

                                                                                                                                                                    3. 1

                                                                                                                                                                      The only comment that was removed was against FRIGN point of view. Nobody is removing differing point of view, just enforcing civil discussion.

                                                                                                                                                                  2. [Comment removed by author]

                                                                                                                                                                    1. 4

                                                                                                                                                                      “We at suckless are heavily opposed to code of conducts and discriminatory organizations of any shape or form.”

                                                                                                                                                                    2. 4

                                                                                                                                                                      It’s responses like yours that really make the case for codes of conduct.

                                                                                                                                                                      1. 2

                                                                                                                                                                        Are you speaking for the group or is that your own opinion? Knowing that the group aligns itself with that position would certainly make me not interested in working with it or contributing.

                                                                                                                                                                        1. 6

                                                                                                                                                                          To be fair, suckless is not well-organised enough to be a group that can have a single opinion to be spoken for.

                                                                                                                                                                          That said, FRIGN is a prominent contributor and I from what I’ve seen most contributors are heavily on the side of “the code will speak for itself”.

                                                                                                                                                                      1. 39

                                                                                                                                                                        I don’t understand the author’s objection to Outreachy. As far as I can tell, they want to fund some interns from marginalized groups so that they can work on open-source. They are not preventing the author from working on open-source. They are not preventing the author from funding interns he approves of from working on open-source. What is the problem?

                                                                                                                                                                        1. 23

                                                                                                                                                                          Outreachy funds members of specific minority groups and would not fund a cisgender white guy’s internship. He decries this as discrimination.

                                                                                                                                                                          On this topic, the term discrimination has differing interpretations and it’s very easy for folks to talk past each other when it comes up. It sounds he’s using it in a way that means disfavoring people based on the sex or race they belong to. Another popular definition is that it only applies to actions taken against groups that have been historically discriminated against. This use gets really strong pushback from people who disagree with the aims or means of projects like Outreachy as begging the question, making an assumption that precludes meaningful discussion of related issues.

                                                                                                                                                                          1. 4

                                                                                                                                                                            It’s not only that Outreachy would not fund a cisgender white guy’s internship. Outreachy also would not fund Asian minority’s internship. Asian minority is a group that has been historically discriminated against. Outreachy is discriminating against specific minority. In summary, Outreachy is simply discriminating, it is not using alternative definition of discrimination.

                                                                                                                                                                            (Might be relevant: I am Asian.)

                                                                                                                                                                            1. 7

                                                                                                                                                                              I asked Karen Sandler. This is the reason for the selection of groups:

                                                                                                                                                                              <karenesq> JordiGH: I saw the lobsters thread. the expansion within the US to the non-gender related criteria was based on the publication by multiple tech companies of their own diversity statistics. We just expanded our criteria to the groups who were by far the least represented.

                                                                                                                                                                              1. 2

                                                                                                                                                                                Thanks a lot for clarifying this with Karen Sandler!

                                                                                                                                                                                I think this proves beyond any shade of doubt that Outreachy is concerned with not historical injustice, but present disparity.

                                                                                                                                                                              2. 3

                                                                                                                                                                                He had a pretty fair description of where the disputes were coming from. Far as what you’re saying on Outreachy, the Asian part still fits into it as even cultural diversity classes I’ve seen say the stereotypes around Asians are positive for stuff like being smart or educated. Overly positive to the point that suicide due to pressure to achieve was a bit higher according to those sources. There’s lots of Asians brought into tech sector due to a mix of stereotypes and H1-B. The commonness of white males and Asians in software development might be why they were excluded with the white males. That makes sense to me if I look at it through the view they likely have of who is privileged in tech.

                                                                                                                                                                                1. 3

                                                                                                                                                                                  Yes, it makes sense that way, but it does not make sense in “historical discrimination” sense pushcx argued. I believe this is an evidence that these organizations are concerned with the present disparity, not with the history. Therefore, I believe they should cease to (dishonestly, I think) argue history argument.

                                                                                                                                                                                2. 2

                                                                                                                                                                                  Well, if you were a woman or identified as one they would accept you, regardless if you were Asian or not. I do wonder why they picked to outreach to the particular groups they picked.

                                                                                                                                                                                  And you have to pick some groups. If you pick none/all, then you’re not doing anything different than GSoC, and there already is a GSoC, so there would be no point for Outreachy.

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    You can pick groups that have been historically discriminated against, as pushcx suggested. Outreachy chose otherwise.

                                                                                                                                                                                    1. 2

                                                                                                                                                                                      To nitpick, I was talking about the term “discrimination” because I’ve seen it as a source of people talking past each other, not advocating for an action or even a particular definition of the term. Advocating my politics would’ve compromised my ability to effectively moderate, though incorrect assumptions were still made about the politics of the post I removed and that I did so out of disagreement, so… shrug

                                                                                                                                                                              3. 49

                                                                                                                                                                                For those who are used to privilege, equality feels like discrimination.

                                                                                                                                                                                1. 18

                                                                                                                                                                                  I think the author’s point is that offering an internship for only specific groups is discrimination. From a certain point of view, I understand how people see it that way. I also understand how it’s seen as fair. Whether that’s really discrimination or not is up for debate.

                                                                                                                                                                                  What’s not up for debate is that companies or people should be able to give their money however they feel like it. It’s their money. If a company wants to only give their money to Black Africans from Phuthaditjhaba, that’s their choice! Fine by me!

                                                                                                                                                                                  Edit: trying to make it clear I don’t want to debate, but make the money point.

                                                                                                                                                                                  1. 19

                                                                                                                                                                                    It is discrimination, that’s what discrimination means. But that doesn’t automatically make it unfair or net wrong.

                                                                                                                                                                                    1. 12

                                                                                                                                                                                      The alternative is inclusive supply plus random selection. You identify the various groups that exist. Go out of your way to bring in potential candidates of a certain number in each one. The selection process is blind. Whoever is selected gets the help. Maybe auditable process on top of that. This is a fair process that boosts minorities on average to whatever ratio you’re doing the invite. It helps whites and males, too.

                                                                                                                                                                                      That’s the kind of thing I push. Plus, different ways to improve the blindness of the evaluation processes. That is worth a lot of research given how much politics factors into performance evaluations in workplaces. It affects everyone but minority members even more per the data. Those methods, an equal pull among various categories, and blind select are about as fair as it gets. Although I don’t know exact methods, I did see GapJumpers describing something that sounds closer to this with positive results. So, the less-discriminating way of correcting imbalances still achieves that goal. The others aren’t strictly necessary.

                                                                                                                                                                                      The next scenario is specific categories getting pulled in more than everyone with organizations helping people in the other ones exclusively to boost them. That’s what’s going on here. Given the circumstances, I’m not going to knock them even if not as fair as other method. They’re still helping. It looks less discriminatory if one views it at a high level where each group addresses those they’re biased for. I did want to show the alternative since it rarely gets mentioned, though.

                                                                                                                                                                                      1. 13

                                                                                                                                                                                        I really agree with this. I was with a company who did a teenage code academy. I have a masters, and did a lot of work tutoring undergrads and really want to get back into teaching/academia.

                                                                                                                                                                                        I wanted to teach, but was actually pushed down the list because they wanted to give teaching positions to female staff first. I was told I could take a support role. The company also did a lot of promotion specifically to all girls schools and to try to pull women in. They had males in the classes too, but the promotion was pretty bias.

                                                                                                                                                                                        Also I want to point out that I had a stronger teaching background/qualifications than some of the other people put in those positions.

                                                                                                                                                                                        I’m for fairness and giving people opportunity, but I feel as if efforts to stop discrimination just lead to more discrimination. The thing is, we’re scientists and engineers. We know the maths. We can come up with better ways to pull in good random distributions of minorities/non-minorities and don’t have to resort to workshops that promote just another equal but opposite mono-culture. If anything you do potential developers a disservice by having workshops that are only women instead of half-and-half. You get a really one sided narrative.

                                                                                                                                                                                        1. 9

                                                                                                                                                                                          I appreciate you sharing that example. It mirrors some that have happened to me. Your case is a good example of sexism against a man that might be more qualified than a women being hired based on gender. I’ll also note that so-called “token hires” are often treated poorly once they get in. I’ve seen small organizations where that’s not true since the leadership just really believed in being good to people and bringing in different folks. They’re rare. Most seem to be environments people won’t want to be in since conflict or resentment increases.

                                                                                                                                                                                          In your case and most of those, random + blind selection might have solved the problem over time without further discrimination or resentment. If process is auditable, everyone knows the race or gender part gave everyone a fair shot. From there, it was performance. That’s a meaningful improvement to me in reducing the negative effects that can kick in when correcting imbalances. What I will say, though, is I don’t think we can always do this since performance in some jobs is highly face-to-face, based on how groups perceive the performer, etc. I’m still uncertain if something other than quotas can help with those.

                                                                                                                                                                                          Most jobs I see people apply for can be measured, though. If it can be measured, it can sometimes already be blinded or may be measured blindly if we develop techniques for that.

                                                                                                                                                                                          1. 3

                                                                                                                                                                                            I agree with these comments, plus, thanks for sharing a real life example. We are definitely fighting discrimination with more discrimination doing things the current way. For a bit I’ve thought that a blind evaluation process would be best. It may not be perfect, but it seems like a step in a better direction. It’s encouraging to see other people talking about it.

                                                                                                                                                                                            One other thought- I think we as society are handling race, gender, age, etc problems wrong. Often, it’s how a certain group ‘A’ has persecuted another group ‘B’. However, this isn’t really fair for the people in group ‘A’ that having nothing to do with what the other people are doing. Because they share the same gender/race/whatever, they are lumped in. Part of this seems to be human nature, and it’s not always wrong. But maybe fighting these battles in more specific cases would help.

                                                                                                                                                                                          2. 5

                                                                                                                                                                                            I think the problem here is that whites and males don’t need extra help. They already get enough help from their position in society. Sure, equal distribution sounds great, but adding an equal amount to everyone doesn’t make them equal; it doesn’t nullify the discrepancy that was there before. Is it good to do so? Yes, of course, but it would be better served and better for society to focus on helping those without built-in privilege to counteract the advantage that white males have.

                                                                                                                                                                                            1. 9

                                                                                                                                                                                              There are lots of people in bad situations who are white and male. Saying someones race and gender determines how much help someone has had in life seems both racist and sexist.

                                                                                                                                                                                              1. 2

                                                                                                                                                                                                I’m not saying that it applies in all circumstances. But I am saying that they have a much larger support structure available to them, even if they didn’t get started on the same footing as other examples.

                                                                                                                                                                                                It’s not directly because of their race and sex, it’s because of their privilege. That’s the fundamental difference.

                                                                                                                                                                                                1. 6

                                                                                                                                                                                                  I don’t even know how much it matters if it was true. Especially in rural or poor areas of white people. Their support structure is usually some close friends, family, people they live with, and so on. Often food stamps, too. Their transportation or Internet might be unreliable. Few jobs close to them. They have to pack up and leave putting themselves or their family into the unknown with about no money to save for both the move and higher cost of living many areas with more jobs will entail. Lots of drug abuse and suicide among these groups relative to whites in general. Most just hope they get a decent job where management isn’t too abusive and the lowish wages cover the bills. Then, you talk about how they have “a much larger support structure available to them” “because of their privilege.” They’d just stare at you blinking wondering what you’re talking about.

                                                                                                                                                                                                  Put Your Solutions Where Your Ideology Is

                                                                                                                                                                                                  Since you talk about advantages of privilege and support structures, I’m curious what you’d recommend to a few laypeople in my white family who will work, have basic to good people skills, and are non-technical. They each have a job in area where there aren’t lots of good jobs. They make enough money to make rent. I often have trouble contacting them because they “have no minutes” on their phones. The areas they’re in have no wired Internet directly to renters (i.e. pay extra for crap), satellite, spotty connections, or they can’t afford it. Some have transportation, others lost theirs as it died with four digit repairs eclipsing 1-2 digits of surplus money. All their bosses exploit them to whatever extent possible. All the bosses underschedule them where the work couldn’t get done then try to work them to death to do it. The schedules they demand are horrible with at least two of us having schedules that shift anywhere from morning to evening to graveyard shift in mid-week. It kills people slowly over time. Meanwhile, mentally drains them in a way that prevents them learning deep stuff that could get them in good jobs. Most of them and their friends feel like zombies due to scheduling with them just watching TV, chilling with friends/family, or something otherwise comfortable on off days. This is more prevalent as companies like Khronos push their optimizations into big businesses with smaller ones following suit. Although not among current family now, many of them in the past worked 2-3 jobs with about no time to sleep or have fun just to survive. Gets worse when they have an infant or kids.

                                                                                                                                                                                                  This is the kind of stuff common among poor and working classes throughout America, including white people. Is this the average situation of you, your friends, and/or most white males or females you know of? These people “don’t need help?” I’m stretching my brain to try to figure out how what you’re saying fits their situation. In my view, they don’t have help so much as an endless supply of obstacles ranging from not affording bills to their evil bosses whose references they may depend on to police or government punishing them with utility bill-sized tickets for being poor. What is your specific recommendation for white people without any surplus of money, spotty Internet, unreliable transportation, and heavily-disrupted sleep?

                                                                                                                                                                                                  Think quickly, too, because white people in these situations aren’t allowed much time to think between their stressful jobs (often multiple) and families to attend to. Gotta come up with solutions about on instinct. Just take the few minutes of clarity a poor, white person might have to solve a problem while in the bathroom or waiting in line at a store. It’s gotta work with almost no thought, energy, savings, or credit score. What you got? I’ll pass it on to see if they think it’s hopeful or contributes to the entertainment for the day. Hope and entertainment is about the most I can give to the person I’m visiting Saturday since their “privilege” hasn’t brought them much of anything else.

                                                                                                                                                                                                  1. 2

                                                                                                                                                                                                    I’m not saying that it’s applicable in every situation; I am specifically talking about the tech industry. I don’t think it’s about prejudice in this case. I think it’s about fixing the tech culture, which white males have an advantage in, regardless of their economic background. White males don’t always have privilege, that would be a preposterous claim. But it’s pretty lopsided in their favor.

                                                                                                                                                                                                    1. 2

                                                                                                                                                                                                      I am specifically talking about the tech industry.

                                                                                                                                                                                                      It’s probably true if narrowed to tech industry. It seems to favor white and Asian males at least in bottom roles. Gets whiter as it goes up. Unfortunately, they also discriminate more heavily on age, background, etc. They want us in there for the lower-paying stuff but block us from there in a lot of areas. It’s why I recommend young people considering tech avoid it if they’re worried about age discrimination or try to move into management at some point. Seems to reduce the risk a bit.

                                                                                                                                                                                                    2. 2

                                                                                                                                                                                                      Your comment is a great illustration of the danger of generalizing things on the basis of racis or gender, mistakenly classifying a lot of people as “privileged”. Ideally, the goal of a charity should be to help unprivileged people in general, for whatever reason they are unprivileged, not because of their race or gender.

                                                                                                                                                                                                    3. 4

                                                                                                                                                                                                      “It’s not directly because of their race and sex, it’s because of their privilege. That’s the fundamental difference.”

                                                                                                                                                                                                      But that’s not a difference to other racist/sexist/discriminatory thinking at all. Racists generally don’t dislike black people because they’re black. They think they’re on average less intelligent, undisciplined, whatever, and that this justifies discriminating against the entirety of black people, treating individuals primarily as a product of their group membership.

                                                                                                                                                                                                      You’re doing the exact same thing, only you think “white people are privileged, they don’t need extra help” instead of “black people are dumb, they shouldn’t get good jobs”. In both cases the vast individual differences are ignored in favor of the superficial criteria of group membership. That is exactly what discrimination is.

                                                                                                                                                                                                      1. 2

                                                                                                                                                                                                        You’re right in that I did assume most white males are well off, and it is a good point that they need help too. However, I still think that the ideas of diversifying the tech industry are a worthy goal, and I think that having a dedicated organization that focuses on only the underrepresented groups is valuable. I just don’t think that white males have the same kind of cultural bias against them in participating in this industry that the demographics that Outreachy have, and counteracting that is Outreachy’s goal. Yes, they are excluding groups, but trying to help a demographic or collection of demographics necessarily excludes the other demographic. How could it work otherwise?

                                                                                                                                                                                                  2. 1

                                                                                                                                                                                                    Why exclude Asians then? Do Asians also already get enough help from their position in society?

                                                                                                                                                                                                    1. 5

                                                                                                                                                                                                      Asians are heavily overrepresented in tech. To be fair, the reason we are overrepresented in tech (as in medicine) is likely because software development (like medicine) is an endeavour that requires expertise in challenging technical knowledge to be successful, which means that (unlike Hollywood) you can’t just stick with white people because there simply aren’t enough of them available to do all the work. So Asians who were shut out of other industries (like theatre) flocked to Tech. Black men are similarly overrepresented in the NBA but unfortunately the market for pro basketball players is a bit smaller than the market for software developers.

                                                                                                                                                                                                      1. 2

                                                                                                                                                                                                        Do they exclude Asians? I must have missed that one. I don’t think excluding that demographic is justified.

                                                                                                                                                                                                        1. 2

                                                                                                                                                                                                          Do they exclude Asians?

                                                                                                                                                                                                          Yes they do. Quoting Outreachy Eligibility Rules:

                                                                                                                                                                                                          You live in the United States or you are a U.S. national or permanent resident living aboard, AND you are a person of any gender who is Black/African American, Hispanic/Latin@, Native American/American Indian, Alaska Native, Native Hawaiian, or Pacific Islander

                                                                                                                                                                                                          In my opinion, this is carefully worded to exclude Asians without mentioning Asians, even going so far as mentioning Pacific Islander.

                                                                                                                                                                                                  3. 4

                                                                                                                                                                                                    It’s a simple calculus of opprotunity. Allowing those who already have ample opprotunity (i.e. white, cis, males) into Outreachy’s funding defeats the point of specifically targeting those who don’t have as much opprotunity. It wouldn’t do anything to help balance the amount of opprotunity in the world, which is Outreachy’s end goal here.

                                                                                                                                                                                                    It’s the author’s idea that they deserve opprotunity which is the problem. It’s very entitled, and it betrays that the author can’t understand that they are in a priviledged position that prevents them from receiving aid. It’s the same reason the wealthy don’t need tax cuts.

                                                                                                                                                                                                    1. 1

                                                                                                                                                                                                      Outreachy’s end goal seems to be balancing the amount of opportunity in the world for all, except for Asian minority.

                                                                                                                                                                                                      1. 4

                                                                                                                                                                                                        Each of us gets to choose between doing good and doing best. The x is the enemy of the y. If Outreachy settles for acting against the worst imbalance (in its view) and leaving the rest that’s just their choosing good over best.

                                                                                                                                                                                                        You’re also confusing their present action with their end goals. Those who choose “best” work directly towards their end goal, but Outreachy is in the “good” camp. By picking a worst part of the problem and working on that part, they implicitly say that their current work might be done and there’ll still be work to do before reaching the end goal.

                                                                                                                                                                                                    2. 4

                                                                                                                                                                                                      What’s not up for debate is that companies or people should be able to give their money however they feel like it.

                                                                                                                                                                                                      That is debatable. But, I too think Outreachy is well within their rights.

                                                                                                                                                                                                    3. 6

                                                                                                                                                                                                      I’m not going to complain about discrimination in that organization since they’re a focused group helping people. It’s debatable whether it should be done differently. I’m glad they’re helping people. I will note that what you just said applies to minority members, too. Quick example.

                                                                                                                                                                                                      While doing mass-market, customer service (First World slavery), I ran an experiment treating everyone in a slightly-positive way with no differences in speech or action based on common events instead of treating them way better than they deserved like we normally did. I operated off a script rotating lines so it wasn’t obvious what I was doing. I did this with different customers in new environment for months. Rather than appreciation, I got more claims of racism, sexism, and ageism then than I ever did at that company. It was clear they didn’t know what equal treatment or meritocracy felt like. So many individuals or companies must have spoiled them that experiencing equality once made them “know” people they interacted with were racist, sexist, etc. There were irritated people among white males but they just demanded better service based on brand. This happened with coworkers in some environments, too, when I came in not being overly selfless. The whites and males just considered me slightly selfish trading favors where a number of non-whites or women suspected it was because they were (insert category here). They stopped thinking that after I started treating them better than other people did and doing more of the work myself. So, it was only “equal” when the white male was doing more of the work, giving more service in one-way relationships, etc.

                                                                                                                                                                                                      I’d love to see a larger study done on that kind of thing to remove any personal or local biases that might have been going on. My current guess is that their beliefs about what racism or sexism are shifted their perceptions to mis-label the events. Unlike me, they clearly don’t go out of their way to look for more possibilities for such things. I can tell you they often did in the general case for other topics. They were smart or open-minded people. Enter politics or religion, the mind becomes more narrow showing people what they want to see. I spent most of my life in that same mental trap. It’s a constant fight to re-examine those beliefs looking at life experiences in different ways.

                                                                                                                                                                                                      So, I’m skeptical when minority members tell me something was about their status because I’ve personally witnessed them miscategorizing so many situations. They did it by default actually any time they encountered provable equality or meritocracy. Truth told, though, most things do mix forms of politics and merit leaning toward politics. I saw them react to a lot of that, too. I’m still skeptical since those situations usually have more political biases going on than just race or gender. I can’t tell without being there or seeing some data eliminating variables what caused whatever they tell me.

                                                                                                                                                                                                      1. 17

                                                                                                                                                                                                        So, in your anecdotal experience, other people’s anecdotal experience is unreliable? 😘

                                                                                                                                                                                                        1. 5

                                                                                                                                                                                                          You got jokes lol. :) More like I’m collecting this data on many views from each group to test my hypotheses whereas many of my opponents are suppressing alternative views in data collection, in interpretation, and in enforcement. Actually, it seems to be default on all sides to do something like that. Any moderate listening closely to those that disagree looking for evidence of their points is an outlier. Something wrong with that at a fundamental level.

                                                                                                                                                                                                          So, I then brought in my anecdotes to illustrate it given I never see them in opponents’ data or models. They might be wrong with their anecdotes right. I just think their model should include the dissent in their arguments along with reasons it does or doesn’t matter. The existence of dissent by non-haters in minority categories should be a real thing that’s considered.

                                                                                                                                                                                                        2. 3

                                                                                                                                                                                                          I think that the information asymmetry that you had with your anecdotes affected some of the reactions you got. For one, if someone considers your actions negative in some way, they are conditioned by society to assume that you were being prejudiced. If your workplace was one that had more of a negative connotation (perhaps a debt collection service or what have you) that goes double. That’s a reason for the percieved negativity that your white male colleagues didn’t even have to consider, and they concluded that you were just being moderately nice. Notice that you didn’t have to be specifically discriminatory, nor was it necessarily fair. It’s just one more negative thing that happens because prejudice does exist. I would imagine that you would not have so many negative reactions if you explained exactly what you were doing vis-a-vis the randomization of greetings and such. I think I would discount percieved discrimination if someone did that to me.

                                                                                                                                                                                                      2. 14

                                                                                                                                                                                                        Yes, it’s a ludicrous hissy fit. Especially considering that LLVM began at UIUC which, like many (most? all?) universities, has scholarships which are only awarded to members of underrepresented groups–so he’d have never joined the project in the first place if this were truly a principled stand and not just an excuse to whine about “the social injustice movement.” (I bet this guy thinks it’s really clever to spell Microsoft with a $, too.)

                                                                                                                                                                                                        1. 7

                                                                                                                                                                                                          That jab “Microsoft with a $” was really uncalled for. You have no evidnece of this. Please stop.

                                                                                                                                                                                                          1. 6

                                                                                                                                                                                                            The point is a bit bluntly made, but it’s for a reason. There’s a certain kind of internet posting style which uses techniques like changing “social justice movement” to “social injustice movement” to frame the author’s point of view. Once upon a time “Micro$oft” was common in this posting style.

                                                                                                                                                                                                            For extreme cases of this, see RMS’ writing (Kindle=Swindle, etc).

                                                                                                                                                                                                            (The problem with these techniques, IMO, is that they’re never as clever and convincing as the person writing them thinks that they are. Maybe they appeal to some people who already agree with that point of view, but they can turn off anyone else…)

                                                                                                                                                                                                            1. 2

                                                                                                                                                                                                              I think there is a difference here. “Microsoft” is not framing any point of view. “social justice movement”, on the other hand, is already framing certain point of view. I think “social injustice movement” is an acceptable alternative to “so-called social justice movement”, because prefixing “so-called” every time is inconvenient.

                                                                                                                                                                                                        2. 0

                                                                                                                                                                                                          Without more info it seems persecution complex.

                                                                                                                                                                                                        1. 11

                                                                                                                                                                                                          “Domain fronting” wouldn’t exist as a misfeature if SNI was encrypted.

                                                                                                                                                                                                          If SNI was encrypted, Google or Amazon would probably still be unhappy about Signal’s use of their IP addresses. But, the conversation would be “Google / Amazon kicks Signal off the cloud” as opposed to “Signal fraudulently uses Google / Amazon’s domain name to combat censorship.”

                                                                                                                                                                                                          Food for thought wrt. the continued existence of unencrypted SNI, even post-TLS 1.3.

                                                                                                                                                                                                          1. 4

                                                                                                                                                                                                            How could SNI be encrypted? SNI determines which certificate should be used to encrypt and decrypt traffic. Maybe I’m missing something, but it looks like an egg and chicken problem.

                                                                                                                                                                                                            Update: Here is some slides about “encrypted SNI”. It makes sense only if the DNS query is encrypted too, and if the server IP cannot be used to indirectly identify the HTTP host.

                                                                                                                                                                                                            1. 9

                                                                                                                                                                                                              The SNI Encryption in TLS RFC draft summarizes the issue far better than I can.

                                                                                                                                                                                                              Update: “make sense only if…” is letting perfect be the enemy of good. Work is being done to encrypt SNI. Work is being done to encrypt DNS. Like peanut butter and jelly, both are good in and of themselves. And they’re even better together!

                                                                                                                                                                                                          1. 6

                                                                                                                                                                                                            Can anyone speak to who this sort of things is targeted at? The way I see it:

                                                                                                                                                                                                            • If uptime is really important, you’re going to need your app running across multiple machines anyways, in which case doing zero downtime deployments is a pretty standard pattern of shifting traffic to one machine, upgrading the other, and then letting traffic come back.
                                                                                                                                                                                                            • If uptime is not so important to your application, then 10 to 20 seconds of downtime doesn’t seem like a big loss. You could say “hey it’s just a better experience” which is roughly what the article says but I struggle squaring that with the running on one machine thing.
                                                                                                                                                                                                            1. 3

                                                                                                                                                                                                              I agree, but the technique described in the article is similar in a multiple machines setup. Instead of having 2 machines, they have 2 processes (the old one and the new one) running on the same machine, with nginx acting as a load balancer.