1. 4

    I once looked at the Python VM to see how hard it would be to convert the code to use Smalltalk-style smallints or use float packing (which would limit it to systems using IEEE floating point with pointers that fit into 48 bits, which is true of all currently supported platforms) to avoid heap-allocated numbers.

    It would not be worth the effort, IMHO. It would break just about every extension module too, and I don’t know how much of a speed up you’d actually get.

    1. 3

      Is that kinda like tagged pointers, where you borrow a few bits in the pointer to store type info?

      1. 6

        So NaN boxing (the floating point method I was talking about) is where you take as advantage of the fact that IEEE NaN values have an enormous amount of representations but only one or a few are ever used in most common situations, so you can use the others to represent anything, like pointers or whatever (so long as your pointers fit into 52-bits).

        A brief write up on the technique is https://brionv.com/log/2018/05/17/javascript-engine-internals-nan-boxing/

        The other technique I was talking is what classic Smalltalk implementations used: any number under a certain limit is an integer, everything above that threshold is an index into the object memory. This is equivalent to saying some bits on the top are used as a tag.

        A lot of the ML systems use a single bit at the top to distinguish between garbage collected and value types, IIRC. And something similar was done in a lot of LISP implementations as well.

      2. 2

        There’s numba, which does JIT on numeric code: https://numba.pydata.org/

      1. 16

        Thanks for drafting these and trying to get at a pattern. (For context, this came out of a twitter conversation.) It’s been particularly frustrating for me that meta conversations (even tag proposals) rarely propose a practical definition of where they’d draw a line, or make a list of stories/comments they’d put on either side of it so we’re actually sure we’re not talking past each other. We very often are. And most proposals are leave the important bits vague, which gives mods huge power to play favorites… and huge workload to carefully investigate and infer repeatable principles for ambiguous situations.

        For tone policing: Would you restrict this to political topics? We regularly get meta comments like “There’s a better way you could’ve said this” or when someone vents about their story/comment being flagged, explaining what happened. Would, and how would you draw a line for something like “I actually think that he’s too harsh on Objective-C”?

        How about meta comments like “I flagged this story because…” which sometimes is followed by a reason? I think it’s useless flamebait when there’s no reason because the comments inevitably come off like sounding like “My opinion is so popular I don’t have to explain it” or “You’re not worth explaining things to”. The codebase has a lot of little UI touches (not showing scores at first, not showing flags until there’s a couple, not showing replies in flagged conversations) that try to damper meta conversation outside of the meta tag, maybe there’s a broader rule worth writing/enforcing.

        Systemic issues: I think a useful test here is: would even the person whose comment is being deleted agree that they’ve violated the rule? They don’t have to like the rule or agree on the larger political topic, but they need to recognize while writing that they’re about to break a rule* so that they can avoid it, and so that every enforcement of the rule doesn’t turn into whataboutism or plausible allegations that it’s a mod enforcing their personal politics. So I see where you’re going with this and why, but I think we’d need to get more specific or find a different angle to have a practical tool here.

        • And yeah, people try flimsy excuses like “oops I didn’t know calling someone a fucking moron was an insult” but I’m not worried about that as a failure mode because it doesn’t matter to predictable enforcement of the rule.

        Maybe a theme in your proposals is “dismissiveness”? Though I’m not sure how much you’re saying you don’t want to see it at all, or in relation to specific political topics, or if recognizing it as a theme will lead towards a better draft of a policy.

        1. 12

          Maybe a theme in your proposals is “dismissiveness”?

          +1. After perusing the thread linked by the OP, I had arrived at the same word. And dismissiveness cuts across ideological divides; I see both ‘progressives’ and ‘libertarians’ (for lack of better terms) resort to it, here and elsewhere.

          A similar tactic I see especially often here is the one used in the comment the above thread was in reply to, I.e. “Why are there no women on this list?” I’m sure there’s a name for this: Responding with a short question that attempts to invalidate the entire statement without giving context or proposing any improvement. Kind of like Socratic dialog, implying the other person needs to be educated by re-considering their statement. I have seen this go on for three or more rounds here (sorry, I don’t remember where exactly.) There’s a strong implication of superiority: “I am so much more intelligent/woke/cynical then you that I shall simply pose koans that you might educate yourself by answering.”

          “Dismissive” may or may not be specific enough to use in a moderation policy, but it seems a good starting point.

          1. 4

            Going to pull in a reply to @dsc that I posted over in the thread that’s linked from this story because it’s an extension of this conversation:

            We don’t have temporary bans as a feature. It’s possible to add, of course, though I’m unsure of the value. Right now the mod tools in this area are pretty much only ‘delete comment’ and ‘ban user’. A temporary ban might be a useful series of escalating punishments. I do think may be a benefit to the mod tools being the fairly harsh ‘deletion of your work’ and then jumping way up to ‘death penalty’ in that people will self-police to avoid them… but maybe that’s just not working.

            Recently I added a “standing” page that’s linked from the warning about being heavily flagged (and PMd all users seeing the warning - add /standing to your profile URL if you’re curious) on the theory that part of the dysfunction is a failure to contextualize and take seriously the community/moderator feedback. Now that it’s had a month to guide actions, and now that its added feedback eliminates excuses about ignorance/non-seriousness, I should probably reach for user bans more.

            And there’s another big reason those mod actions may not be working. I’m recently back on a working schedule after being badly disrupted by the pandemic, so I haven’t been very active since ~February. Probably a reason feedback is not working is lack of enforcement, which is my fault as the admin, either for not doing or not delegating it. I’ve been thinking about posting to recruit more moderators; I’ll do so in a week or two. I think it’d be helpful to finish this meta conversation and otherwise improve the explicitness of mod activity before bringing more on. (And on the technical side, create a reddit-like work queue system for mods; the current dashboard doesn’t effectively prevent things from slipping through the cracks.)

            1. 2

              Worth noting that this comment is seemingly inspired by a simple question trying to incentivize people to think about how women are treated in open source which led to dozens of people deciding that I was trying to attack them instead of taking a big picture look at the question.

              This also has led to me having a very exciting warning in my reply inbox which seems excessive because a few people have fragile egos and think I’m attacking them because they need to assume negative intent. Maybe the site shouldn’t be giving me a slap on the wrist for that in the first place.

              People are now suggesting temporary bans, so now we are here where someone is trying to incentivize that we are the kind of community who punishes bad behavior such as trying to get people to think about how people that aren’t like them are cast out of their industry.

              This is the largest bunch of BS. People are trying to use a thread about not tone-policing as a way to promote a heavier ban process as a way to directly tone-police me because they assumed my intentions and can’t have a regular discussion about it.

              https://cdn.monokro.me/images/why_are_there_no_women_on_this_list.png

              First thing that we see here is that 42 people in this situation agree with tone policing, because that’s the basis for people leaving negative flags on this post. Specifically citing in multiple cases that I’m being “aggressive” because they see a direct attack and don’t recognize that I’m trying to point out something that is representative of the bigger picture. I also can’t help but wonder if my gender plays into the responses having seen similar responses to women in the workplace and open source environments in the past, but I’ll digress from that as it is no better than a guess…

              I can agree that maybe I could have been a bit more verbose to prevent people from assuming, but preventing people from assuming is also not really my job. Surely people should be smart enough to realize that their assumed intent isn’t always the true intention. Come on, are y’all for real right now?

              Even with +29 upvotes, I still end up flagged for this comment because of 28 people flagging this without any good reason. To make that clear, most of these flags don’t even make sense. Let’s break down a large trail of intentional ignorance, shall we?

              • There are 17 who think I’m trolling? Trolling shouldn’t even be a flag, because it’s so easily abused to mean anything that doesn’t fit the rest. This is obviously defensive men being angry and down-voting for no reason and generally acting like this is Reddit.

              • There are 10 “off topic” when I’m providing commentary regarding the nature of patterns exhibited obviously in the list made in the previous comment.

              • One is “unkind” because they they assumed a meaning and flagged me? At least if you take the assumed meaning, this is the single one that makes the most sense - even though it does rely on a big and wrong assumption to get there.

              This is a mob acting mindlessly and that’s the real problem here. My guess is they are also probably a majority of people who got here after the site lost it’s invite-only nature, which is really something that should be reconsidered because the entire site is going downhill fast without the invitation pipeline in place.

              So, it seems that - at least in this case - even when someone tries to defend the culture of how women are treated by making a simple observation, nearly half of the people who respond to that comment can’t reason about it and have to resort to attacking because they seem to feel like having women recognized is somehow taking something away from them. In reality nothing is being taken, but we see a bunch of defensive men.

              Just to clarify so someone doesn’t think I’m being sexist here, this is of course among many men who are thoughtful and agreed with my point - as can be seen in the comments.

              Maybe making votes public would be a good thing, because it’s worth being able to look further into who are doing this. I can’t substantiate my assumptions without that information, so I’ll just sit back and see if anyone else figures it out.

              This will be seen as my most aggressive comment. Mostly because my comments are rarely aggressive. However, this comment is intended to defend against a pattern of poor behavior that I’ve seen consistently on this site since it stopped being invite-only. My point in this post is to offer reasoning on why we should be invite-only and why we should publicize people’s voting history. All statements made are in attempt to realize this situation, and I am only defending what has turned into my profile showing a giant warning about “too many flags’ because some people are being defensive when there isn’t anyone attacking them.

              TLDR; A stronger ban policy is an immature, reasonless solution. The only way that stronger ban policies can exist for content is if we as a community are able to honestly flag content for what it is instead of bringing personal emotions into the mix and not meeting them half-way with logical reasoning. If you can’t define “good” and “bad” behavior objectively, then you end up with the worst people in your community having the most power because they have no empathy or means of reasoning about other people. You create a system that brings the worst to the top. If we wanted that, we could go to Reddit or Twitter.

              This is likely the result of a culture problem. My guess it that it’s from when the site was invite-only this was rarely a problem. Instead of inventing new solutions for fixing the problem, maybe we can address that getting rid of the invite-only nature of this platform negatively affects it. The culture here has gone downhill steadily after this occurred and seems to not be self-adjusting. When people can just sign up for a new account without recourse whenever they feel like it, they are less likely to be thoughtful about what they say and how their account looks. More people who are just casually browsing and don’t intend to be part of the community show up. It’s not good.

              1. 14

                For my part, I flagged it as off-topic because someone just listed four of their favourite Open Source authors and trying to shoe-horn a particular social issue in to that is just, well, off-topic. I might be more forgiving if there was some substance as that would provide value in spite of being off-topic, but as it stands it just comes off as a challenge to person you replied to, at least in my reading of it. I appreciate this was not your intent, but that doesn’t change it.

                I most certainly do think it’s your job to ensure your intent is conveyed clearly. There’s been plenty of times where I’ve been misunderstood, and this has always almost been a failing on my part in not being clear enough. I don’t think it’s the reader’s job to guess or assume your intent: it’s the author’s job to clearly convey it.

                If half the community has a problem with your post then there is probably something wrong with your post, and not with half of the community. I appreciate that you feel attacked and that this is a shitty position to be in, but I wish you’d spend some more effort in trying to understand why people have a problem with your challenging question. I find it ungenerous to dismiss that as a “a mob acting mindlessly”, “fragile egos”, “defensive men being angry”, “angry babies”, and so forth. It’s funny you complain about “assuming negative intent”, because there’s a whole lot of that in your post here.

                My guess is they are also probably a majority of people who got here after the site lost it’s invite-only nature

                When people can just sign up for a new account without recourse whenever they feel like it

                I’m confused, because as far as I know Lobsters is still operating on an invite-bases, just as it always has?

                1. 0

                  Lobsters was open registration for a while. If you don’t think that the cultural issues in open source that have lead to this list aren’t on topic when talking about a list of people in open source then oh well, I guess. It’s absolutely on topic and worth taking a moment to think about.

                  1. 6

                    It’s absolutely on topic

                    You’re now just contradicting arp242 without providing any reason.

                    Not every thing that is related to some subject is necessarily ‘on topic’. If you would scour the internet for people posting all-male lists of their favorite somethings and ask that same question every time, would you then always be ‘on topic’? All the time, for everyone, not matter who the audience of the list?

                    I would say your comment would have been appropriate and ‘on topic’ on an article with a top 4 of most important open source people as voted by a large group of open source people. Then it would be a clear demonstration of cultural issues in open source and would be worth pointing out. On a comment by someone musing about something done by 4 personal heroes: no, not so much. And doing it in the form of a passive-aggressive rhetorical question didn’t help.

                    1. 5

                      I never said it’s not worth thinking about. I’m also not injecting my veganism every time someone mentions something vaguely related to meat, and on the extremely rare cases that I do I make sure it’s something of substance (which, hopefully, is helpful in explaining my viewpoint) rather than just a challenge. I don’t like these kind of challenges on any topic as the main reason of a discussion/conversation is to better understand a point of view (and maybe, perhaps, also adjust your own). Challenges like this are not conducive to that.

                      I would also not be impressed if someone injects “Free Software vs. Open Source” debates on every story that mentions “Open Source” instead of Free Software.

                      And so forth, and so forth.

                      As mentioned, I probably wouldn’t have flagged it it was actually something interesting; I’m not especially pedantic about “off-topic discussions” because interesting conversations are interesting conversations. The off-topic flag is a calculation of offtopic + substance.

                      1. -2

                        Unfortunately for you, the community is not not wholly focused on or interested in only following the behaviors that impress you.

                        Your responses show that you think it’s not worth thinking about, regardless of how you want to defend that statement. Your assumption that there’s no substance to it shows that you care little about thinking about these things, and that’s revealing enough.

                        Furthermore, your reply right here has less substance than even my initial comment.

                        And so forth, and so forth.

                        1. 10

                          I don’t even know what to say to that … If you’re hell-bent on making assumptions about anyone who doesn’t agree with on what are actually comparatively minor issues then there’s not much else I can say 🤷‍♂️

                          As I have mentioned elsewhere, the only thing that has turned me off from social justice is the last few years have been some of its proponents and not its detractors. You can’t fight a toxic culture with toxicity. I strongly suggest you reflect on how your words are being perceived, because you’re not just being ineffective at improving the position of woman in Open Source, but are actively harming this cause. This entire thread has left me exasperated and frustrated, and I don’t think that exasperating and frustrating people is an effective way to promote a cause (quite the opposite), and the only thing that this thread has done for me is ensuring that I don’t want to touch this topic with a ten-foot pole, rather than actively trying to spend effort in improving it (which, in spite of your assumptions, is something I do care about).

                          virtue signal all day about how you gave money to girls who code

                          You are confusing me with burntsushi.

                          1. -4

                            the only thing that has turned me off from social justice is the last few years

                            Just to be clear, you have now agreed to not care about this any more.

                            I actually edited that out and didn’t mean to submit with it because it wasn’t productive, but either way I’m uninterested in hearing what you have to say because you are only seeking confirmation of your own biases against this topic. I’m tired of it. 🥱

                            Have fun running around providing unsolicited advice to everyone you talk to. I’m done with it, though, and hoping that you are as well. I will not respond to any replies, though. Waste of both our time to convince you of the biases that you seem to not want to be made aware of.

                  2. 7

                    For more reasoning here, we can look at my recent flags:

                    https://cdn.monokro.me/images/flags_on_lobsters.png

                    The only one here that is actually accurate is where I said “Seems about time” in reference to the Perl language not existing - which feels pretty troll-like in retrospect - although my intention was more based on there not really being a lot of movement in the Perl world any more. The language has been barely thriving for nearly 20 years at this point with few companies adopting it and many throwing it away, but I digress…

                    The rest are all someone saying “I disagree” by flagging “incorrect” or similar childish behavior and behaving like we’re on Reddit. Even though Reddit even has a global site-wide rule against that, they can’t really control the behavior unless they can read your mind.

                    It seems pretty unnecessary to toss a giant warning in my inbox and worry me when I wake up to it in the morning when the reasoning behind the warning is so very based on subjective nonsense.

                    https://cdn.monokro.me/images/2spooky4me.png

                    Is this really how we want to treat nonsense? A giant warning that only serves to shut people up and preserve the insecurity of those who say your posts are “incorrect” because they disagree or don’t want to hear it but also don’t want to have a civilized discussion about it? No thanks.

                    I haven’t caused any problems in 6 years here. What is going on?

                    1. 3

                      It seems pretty unnecessary to toss a giant warning in my inbox and worry me when I wake up to it in the morning when the reasoning behind the warning is so very based on subjective nonsense.

                      I currently have that same warning on my inbox, and I can empathize with being disturbed by it. I don’t think that getting that warning because of a bunch of politically-motivated downvotes necessarily makes you or me a bad person or a bad contributor. I do think that most of those downvotes you’ve shown in that picture strike me as unfair. Even the perl comment, while not exactly the most productive comment imaginable, is not what I would call trolling.

                    2. 7

                      Worth noting that this comment is seemingly inspired by a simple question trying to incentivize people to think about how women are treated in open source which led to dozens of people deciding that I was trying to attack them instead of taking a big picture look at the question.

                      I engaged your post with good faith. I thought you were trying to start a fight and wrote my response to mitigate it. The difference between trolling and good faith is often writing a single sentence vs writing four. If you wanted to incentivize people to think about women in open source, you could have written more than just one sentence.

                      My guess is they are also probably a majority of people who got here after the site lost it’s invite-only nature, which is really something that should be reconsidered because the entire site is going downhill fast without the invitation pipeline in place.

                      The site’s still invite-only.

                      1. 1

                        If you wanted to incentivize people to think about women in open source, you could have written more than just one sentence

                        I agree and even said this in another post somewhere :)

                        At some point, it wasn’t invite-only, right? That was for a while, I think…

                        1. 4

                          IIRC that was only during the april fools gag a couple years back. I remember @pushcx being pretty miffed because a couple of spamming rings joined then. There might have been another no-invite period, but that’s the only one I definitely remember happening!

                      2. 10

                        This is obviously defensive men being angry and down-voting for no reason

                        I voted your post as “trolling” because it appeared to be a content-free attack on the prior poster for simply listing a few developers they liked. Your comment did not even vaguely suggest you were making some bigger point: it’s a line of text, approaching it like that leaves innumerable possible questions embedded in those 8 words. It was unnecessary and kinda lame. Your gender has nothing to do with it.

                        1. 4

                          Can whoever downvoted this as “troll” tell me why?

                          1. 1

                            Seems like it’s not downvoted any more

                            1. 2

                              Now it has two unexplained troll votes, and one unkind vote. I can’t see how the comment is either of those.

                              1. 2

                                Weird, it doesn’t show for others I guess?

                                1. 2

                                  Yep, it shows to authors and moderators, though there are some caveats for authors to try to prevent discussions from veering into meta.

                                  1. 1

                                    Flags and flaggers should be public.

                          2. 0

                            You are still the one who assumed the worst, though.

                          3. 6

                            Worth noting that this comment is seemingly inspired by a simple question trying to incentivize people to think about how women are treated in open source which led to dozens of people deciding that I was trying to attack them instead of taking a big picture look at the question.

                            Here was your mistake, I think–having done it myself. If you have come out and said something like, “Hey, I notice there aren’t any women on your list–I don’t think you’re sexist, but can you think of any that might make it on there or alternately how could we do better?” then you would have both been clear to people who wanted to engage and you would’ve set a good tone for a starting point for discussion. People who responded like assholes to that would’ve in fact been assholes.

                            By asking a pointed, leading question and not making clear your intentions, though, I think you both signaled that you were trying to trick lobsters into thinking a particular way (even if that way is reasonable introspection) and that you didn’t actually care enough to put effort into crafting a comment that could’ve take into account context and been positive. I’ve done the same thing, and have only gotten away with it on occasion by the charity of the audience.

                            Don’t ask leading, pointy questions if you can’t read a room or play off the response.

                            One is “unkind” because they they assumed a meaning and flagged me? At least if you take the assumed meaning, this is the single one that makes the most sense - even though it does rely on a big and wrong assumption to get there.

                            They who live by the sword….

                            This is obviously defensive men being angry and down-voting for no reason and generally acting like this is Reddit.

                            This is a mob acting mindlessly and that’s the real problem here.

                            This appears to me to be the same category of error as people assuming that, say, downvotes of shilling for Urbit or something is the result of mean old antifa. You have dehumanized the people who disagree with your actions, whether or not they disagree with your aims and whether or not you yourself actually did the correct thing.

                            If you can’t define “good” and “bad” behavior objectively, then you end up with the worst people in your community having the most power because they have no empathy or means of reasoning about other people.

                            Again, please refrain from dehumanizing your fellow users. It may be the case that specifically because they have means of reasoning about other people they recognize objective claims of good and bad behavior are a poor fit when real people are involved. It may be because of their empathy they are willing to give people a little more room in decorum than would otherwise be afforded.

                            When people can just sign up for a new account without recourse whenever they feel like it, they are less likely to be thoughtful about what they say and how their account looks.

                            We briefly flirted with open registration, but as far as I know it’s been invite-only for quite some time.

                            1. -1

                              I shouldn’t be getting a ton of unsolicited advice for this, but apparently that’s where we’re at now.

                              “Hey, I notice there aren’t any women on your list–I don’t think you’re sexist, but can you think of any that might make it on there or alternately how could we do better?”

                              This recommendation is completely against my intentions. My intentions were to point out a cultural issue instead of pointing at the person posting the list. Your suggested response is even closer to pointing at them than I expected the intentions to be. There is no expectation for the person who made the list to include women, but I was simply trying to point out that open source culture as a whole seems to exclude women. Not any specific individual.

                              They who live by the sword….

                              I got the irony of me having added the commit for the unkind option to the site, but honestly I think this - just like the rest of your reply - is more harmful than it is helpful.

                              Having an “unkind” flag is still valuable, and people would just call it “troll” or some other in-genuine thing if we didn’t have it. Having not added an unkind flag doesn’t stop the community from using the downvote as a “I don’t agree” instead of what it is intended for.

                              You have dehumanized the people who disagree with your action Again, please refrain from dehumanizing your fellow users

                              No, I haven’t dehumanized anyone. This is a human behavior that they are exhibiting and I am pointing it out. I’m only speaking of those who have already flagged for reasons that are complete nonsense and based on unfair assumptions. People who behave like that are the worst people, and the fact is that bad people tend to end up in positions of power. I even got a private message from someone based on my recent posts from someone who said that they don’t come here any more because of these problems and said thanks for being human on a site that has a culture of hostility.

                              Maybe someone should consider all of us who actually care about the human side are leaving, because ignoring that obvious fact is exactly how we end up with people flagging posts due to assumed negative intentions, giving unsolicited advice, and being generally hostile towards others.

                              This community has been slowing dying for 2-3 years now and it has been descending extremely fast for the last 9 months. I’m sure that someone’s going to say “that’s only because people have been stressed a lot this year” but that’s honestly not a good excuse.

                              Anyway, are we done with the unsolicited advice now? If not, could the advice at least be useful and not repeats of things that I’ve already personally called out next time? Also, are you a mod on this site? If this is the behavior that comes from mods then I’ll probably just drop the site completely. I’ve already removed it from my homepage. Seems like anyone who gives a shit is leaving, so maybe someone should start recognizing it.

                              If I wanted to be on Reddit, I’d just go to Reddit.

                              1. 7

                                Also, are you a mod on this site?

                                I am not a mod! I don’t think anybody would abide by that–and more generally the people who want to be mods shouldn’t be mods.

                                1. 4

                                  This community has been slowing dying for 2-3 years now and it has been descending extremely fast for the last 9 months.

                                  Could you elaborate on what you mean by dying and descending? What do you see happening, and what feels especially different about the last nine months?

                              2. 3

                                Lobsters has never had open signups and still requires an invite; users can’t trivially sign up for an account. The last couple significant changes in the invitations and signups controllers were in Feb 2020 to restrict new users’ ability to send invites, in May 2016 to allow admins to disable a user’s ability to send invites, and in Feb 2015 to disable the invite queue system.

                                Edit: Oops, @hwayne reminded that the 2018 April Fools’ Day gag had open signups and, yes, it strongly reiterated the anti-spam value of the invite system. That spammer later wheedled fresh invites out of Lobsters users on Twitter to re-establish his voting ring twice.

                                1. 2

                                  Worth noting that this comment is seemingly inspired by a simple question trying to incentivize people to think about how women are treated in open source which led to dozens of people deciding that I was trying to attack them instead of taking a big picture look at the question.

                                  Implying that there’s something wrong with a group of men working in a group on some kind of project without going out of their way to include a woman is an attack on men. I don’t think you incentivized people to think about how women are treated in open source, but rather, how men are. I will say that I don’t think your post should’ve been moderated away, but neither do I think that about the various people who responded to your post disagreeing with it, and those response posts are what this meta-thread is about.

                                  Even with +29 upvotes, I still end up flagged for this comment because of 28 people flagging this without any good reason. To make that clear, most of these flags don’t even make sense. Let’s break down a large trail of intentional ignorance, shall we?

                                  I agree that the vast majority of the downvotes on your post were surely submitted in a spirit of political disagreement with it, rather than out of a sincere desire to mark content that is genuinely trolling, off-topic, etc. I think this is true of a lot of politically-controversial posts on lobsters, as well as on other sites with some kind of downvote system. It’s pretty easy for a motivated reasoner to convince themselves that some post they have a strong political reaction to is incorrect or trolling or otherwise downvote-worthy.

                                  Trolling shouldn’t even be a flag, because it’s so easily abused to mean anything that doesn’t fit the rest.

                                  Agreed. I don’t think your post constituted actual trolling. I often don’t think other posts on lobsters I see with troll downvotes constitute actual trolling either.

                                  First thing that we see here is that 42 people in this situation agree with tone policing, because that’s the basis for people leaving negative flags on this post. Specifically citing in multiple cases that I’m being “aggressive” because they see a direct attack and don’t recognize that I’m trying to point out something that is representative of the bigger picture. I also can’t help but wonder if my gender plays into the responses having seen similar responses to women in the workplace and open source environments in the past, but I’ll digress from that as it is no better than a guess…

                                  I doubt it - I assumed you were male until I read this paragraph, and it would be easy for other people to have done that as well. Certainly there are plenty of people who are (well-known to be) male who make arguments similar to yours, and receive pushback in a similar way to how you have.

                                  1. 2

                                    I agree that the vast majority of the downvotes on your post were surely submitted in a spirit of political disagreement with it, rather than out of a sincere desire to mark content that is genuinely trolling, off-topic, etc. I think this is true of a lot of politically-controversial posts on lobsters, as well as on other sites with some kind of downvote system. It’s pretty easy for a motivated reasoner to convince themselves that some post they have a strong political reaction to is incorrect or trolling or otherwise downvote-worthy.

                                    I always find myself questioning (but not really weighing in one way or the other) the need for an upvote/downvote system for comments. I understand using the functionality for topics, and I understand moderators specifically taking actions against posts, but I feel that if a comment feels inane, snarky, or something you disagree with (or on the flip side, you find yourself angry and reply with a short, pithy acerbic reply rather than a well thought out response that you would have produced under better emotional circumstances), then the comment chain can just end there. Of course I also go back to the other side of thinking of the destructive behavior that can be wrought without a user-based form of QC on posts, and that makes me question my original question.

                                    1. 2

                                      Implying that there’s something wrong with a group of men working in a group on some kind of project without going out of their way to include a woman is an attack on men

                                      Sure, but I never said that. I simply made a statement suggesting that there’s probably a stronger reason why men are the only ones who tend to be in these list or even more so who even tend to get popular in open source as a whole. A lot of this isn’t because someone only included men, and more about why women have such a hard time “rising in the ranks” as it were. You are making assumptions in order to come to this conclusion, and they are defensive assumptions based on another assumption of negative intent on my part - which is absolutely not the case and never was.

                                      I agree that the vast majority of the downvotes on your post were surely submitted in a spirit of political disagreement with it, rather than out of a sincere desire to mark content that is genuinely trolling, off-topic, etc

                                      That’s my main point here. We can’t be banning people or doing anything too over-the-top (I think even the warning I got is too much tbh) until we have established an ability to objectively weigh the content of people’s posts, which is unlikely to happen any time soon.

                                      Agreed. I don’t think your post constituted actual trolling.

                                      Neither did I, although I do counter my own argument there by providing an example of a previous post of mine that I think feels like trolling even if it wasn’t my intention - so maybe my argument isn’t too strong here 🤷 lol

                                      I doubt it

                                      I agree with everything that you said here, although what I was getting at was more of the reaction to this kind of behavior than the initial response itself. Like, if I see that half of the people who voted on my post don’t care about women in tech then that’s pretty disconcerting. After 6 years of lobste.rs being my homepage and never really immediately after seeing an official banner in my replies warning me about these flags. Even if it wasn’t meant to be used this way (I hope), it’s still a slap on the wrist over someone else being defensive about their misinterpretations.

                                      I decided to put it back, though, because I’d rather just defend myself than give the angry babies what they want.

                                      1. 13

                                        Like, if I see that half of the people who voted on my post don’t care about women in tech then that’s pretty disconcerting.

                                        I am a reoccurring donator to Rails Girls. I encourage others to do the same. I participated in the first Increasing Rust’s Reach program. My wife is a programmer, to whom I lend my unconditional support. To say that I “don’t care about women in tech” is to not know me at all.

                                        And yet, I downvoted your comment for trolling. Because I knew it was going to lead to a spiraling and pretty unproductive discussion. And because, despite your intentions, it did come across as an underhanded and unwarranted attack to me. My downvote of your comment was me expressing, “this is not the kind of discussion I want to see at lobste.rs.”

                                        If you assume that everyone who disagrees with you is acting in bad faith and can’t be anything more than an “angry baby,” then it’s too easy to turn it into an Us vs Them issue where the people who disagree with you are totally unreasonable people, and therefore, shouldn’t be listened to at all. I am exposing myself as one of your downvoters with the hope to counteract that perception. I am not against you because of your gender. I am not someone who doesn’t care about women in tech. I am not against you because I’m angry. I am against your comment because of the kind of discourse it promotes, which I think ultimately hinders the cause of progress and makes it impossible to have good faith discussions.

                                        1. 0

                                          “Trolling” assumes intention. I thought this community to be a more positive space than it actually is, and mentioned this because I thought it was something worth thinking about. Apparently y’all disagree.

                                          I had and still have no intent to troll anyone.

                                          1. 6

                                            and mentioned this because I thought it was something worth thinking about. Apparently y’all disagree.

                                            That’s very clearly not what I said. I told you precisely why I downvoted your comment, and it had nothing to do with my interest in the topic. You’re making my point for me. You’re going out of your way to make this Us vs Them by trying to draw a bright dividing line between Good and Evil. My whole point of exposing myself as one of your downvoters was to show you that there’s more to it than that. There’s nuance.

                                            And if your definition of trolling requires intent, then it’s almost impossible to flag anyone for trolling because people almost never say, “my intent is to troll.” Without that, you can’t know intentions.

                                            Intentions count for something, but they aren’t everything.

                                2. 3

                                  “Political” has a bunch of meanings. One of them is “how power is exercised and distributed.” In that particular sense (as opposed to say, party politics) moderation is inherently political, and moderation decisions will inherently have to pick one or the other side of certain issues.

                                  1. 3
                                    Tone policing

                                    “Tone policing” means focusing solely on the tone, instead of the content. The “why are there no women on this list?” was mostly responded to with comments saying it was “aggressive”. I’m sure there are plenty of other comments that are aggressive on lobsters that don’t get that response. And it’s pretty clear why.

                                    If someone says “This is technology is a flaming pile of useless garbage”—a response saying “why are you being so aggressive” (however unlikely it is) doesn’t add much. A response saying “technology is about tradeoffs, it’s rare that something is that useless” or “well, once you understand the organizational context” is useful.

                                    I flagged this story because

                                    friendlysock specifically seems to have the modus operandi of saying NO POLITICS and then injecting his politics everywhere (see below—you can’t not be political. I disagree with his politics, but the relevant point here is that I think his stated “NO POLITICS” is disingenous).

                                    Not sure how general of a problem this sort of comment is.

                                    Systemic issues

                                    “Would even the person whose comment is being deleted agree that they’ve violated the rule?” as motivated by “plausible allegations that it’s a mod enforcing their personal politics”—I think this is highly misguided:

                                    1. People who genuinely don’t realize they’re hurting other people are given a free pass.
                                    2. Bad actors will of course give you “whataboutism” and rules lawyering. That’s how it goes.
                                    3. You have to enforce a political point of view. There is no way around it.

                                    That last point is the fundamental one I keep trying to get across.

                                    As an example: either you allow sexism, and then you get less women participating (not to mention all the men I know who very deliberately aren’t participating in lobste.rs because they also consider it toxic.) If you disallow sexism you will lose some sexist people.

                                    On some issues you have to pick a side, and you are going to have to exclude someone. There is no way around it, no way to avoid politics. If you say “I don’t want to exclude anyone” then you’re excluding lots of people by default. (See also “torelance is a peace treaty”: https://extranewsfeed.com/tolerance-is-not-a-moral-precept-1af7007d6376).

                                    1. 5

                                      friendlysock specifically seems to have the modus operandi of saying NO POLITICS and then injecting his politics everywhere (see below—you can’t not be political. I disagree with his politics, but the relevant point here is that I think his stated “NO POLITICS” is disingenous).

                                      I would be as fascinated to know what you think my politics are as I would be certain it would be off-topic for this site. DMs are open.

                                      EDIT:

                                      I missed something that bears pointing out.

                                      You have to enforce a political point of view. There is no way around it.

                                      This is an axiom of your belief system. There are a great many of us here that clearly do not share this axiom. Without stating whether or not the axiom itself is correct (it’s an axiom, so that doesn’t really matter) we can state there will be inevitable and continuous conflict between two belief systems (effectively, realities) that differ by construction.

                                      It would probably be better were you to spin up your own instance so you wouldn’t have to keep dealing with this. It might well even become more popular!

                                      1. -2

                                        I would be as fascinated to know what you think my politics are as I would be certain it would be off-topic for this site.

                                        Your stated position is that it’d would be totally cool to have people who want to murder me on lobste.rs so I’m going to say “misguided.”

                                      2. 2

                                        As an example: either you allow sexism, and then you get less women participating (not to mention all the men I know who very deliberately aren’t participating in lobste.rs because they also consider it toxic.) If you disallow sexism you will lose some sexist people.

                                        The original post was sexist against men (or at least it was reasonable to interpret it as such; I’ll grant that that might not have been the post-writer’s intention), and none of the posts arguing against it were sexist. You are falsely claiming that they were sexist because you adhere to a political ideology that doesn’t treat sexism against men as morally relevant, but still want to use the existing societal understanding that sexism is bad and can be legitimately socially punished as a tool to enforce your personal politics.

                                        On some issues you have to pick a side, and you are going to have to exclude someone. There is no way around it, no way to avoid politics. If you say “I don’t want to exclude anyone” then you’re excluding lots of people by default.

                                        Communities that intend to be broadly inclusive should always pick the side of the people who aren’t demanding that existing members be excluded as a condition of their own membership.

                                        1. 2

                                          The original post was sexist against men (or at least it was reasonable to interpret it as such; I’ll grant that that might not have been the post-writer’s intention)…

                                          The question “why aren’t there women on this list?” is not sexist, nor is it reasonable to interpret it as such. It’s clear that many others felt it was brusque or dismissive and that it could have been framed with extra clarity so as to avoid those appearances, but it’s factually dishonest to refer to that question as sexist. If you are genuinely concerned about “sexism against men” then I suggest that you don’t harm your credibility by mischaracterizing instances of sexism. However, I’m struggling to imagine that you are concerned with sexism as a social problem, because…

                                          …and none of the posts arguing against it were sexist.

                                          You yourself wrote “Because none of them has come out as trans yet.”. Which is genuinely one of the most vitriolic things I’ve ever read on this website. I’ve seen dogwhistling and whataboutism; I’ve seen users with biases against social justice due to bad experiences and lack of opportunities for learning. We can work on that. On the other hand I can not recall having actually seen content so explicitly demeaning here. The OP of this story calls for “No pretending systemic racism, sexism, and bias aren’t a thing” which I agree with, but it appears that lobste.rs will have to tie its shoes before it can walk. I’ll call on @pushcx to clarify that lobste.rs in fact does not stand for forms of hatred and bigotry that include racism, sexism, transphobia, ableism, and anti-semitism. And I’ll call on the moderators of this community to ban users such as yourself that make engage in overtly violent forms of communication such as this.

                                          I think that frankly enough is enough. This is a question of basic human needs. If the desire is to welcome new members and cultivate diverse discussions on a range of deep technical topics, we need to first make sure that this website is not a platform for hate and bigotry.

                                      3. 1

                                        How about meta comments like “I flagged this story because…” which sometimes is followed by a reason? I think it’s useless flamebait when there’s no reason because the comments inevitably come off like sounding like “My opinion is so popular I don’t have to explain it” or “You’re not worth explaining things to”. The codebase has a lot of little UI touches (not showing scores at first, not showing flags until there’s a couple, not showing replies in flagged conversations) that try to damper meta conversation outside of the meta tag, maybe there’s a broader rule worth writing/enforcing.

                                        I would personally love this addition.

                                        Very often when my comments are flagged as something like Incorrect or Troll if I don’t understand why they did that I’ll try to reach out and ask politely. It’s resulted in some very useful feedback and a better shared understanding on both sides.

                                        I think it would also help discourage off the cuff flags that actually reduce to “I disagree”.

                                        1. 4

                                          On tildes.net you can “label” posts as “Exemplary”, “Off-topic”, “Joke”, “Noise”, and “Malice”. For the “Exemplary” and “Malice” labels you can leave a reason which is presented to the author anonymously.

                                          There are no downvotes like Reddit/HN, this is how Tildes does “flagging”.

                                          What I like about this is that you can leave feedback anonymously, without getting embroiled in a discussion about it. The author of the post that you flagged can then take it or leave it.

                                          The downside of this is that people can leave insults in the feedback (“this is fucking stupid”, or whatnot), so there is some potential for abuse and would probably require some moderation. AFAIK this isn’t a problem on tildes as the community is very small, but it may be on Lobsters.

                                          1. 2

                                            The downside of this is that people can leave insults in the feedback (“this is fucking stupid”, or whatnot), so there is some potential for abuse and would probably require some moderation. AFAIK this isn’t a problem on tildes as the community is very small, but it may be on Lobsters.

                                            I’d argue that anyone who put an insult in a space for feedback has shown themselves to not be interested in contributing to any meaningful community.

                                            1. 4

                                              Indeed. My main concern is that people will see those insults, and if people leave real nasty comments it might be bordering on abusive. It’s just one more thing that would require some amount of mod involvement (flag the flag? meh)

                                      1. 25

                                        The general pattern in which I’ve seen “tone policing” used is usually something like this:

                                        • “Something offensive, abrasive, insulting, or otherwise unpleasant”
                                        • “Hey, that was pretty offensive, abrasive, insulting, or otherwise unpleasant!”
                                        • “Stop tone policing me you troll!”

                                        I am not a fan.

                                        A better solution would be to just not say something offensive, abrasive, insulting, or otherwise unpleasant. Or if you do (we all do sometimes) then just either own up to it or have a conversation about it, instead of going “tone policing!” full defence mode.

                                        As a general point, I’ve found that almost every time I see someone use “that’s [fallacy X]” they’re not really interested in having a meaningful conversation. It’s just trying to “win the argument”. Checkmate atheists, or something.


                                        As for your second point … sometimes people disagree on complex social issues 🤷‍♂️ If I look at this comment you linked then the good-faith interpretation of it is that they actually agree on the basic premise that there are problems with disadvantaged groups, but they just disagree on the solution. I don’t think being wrong[1] is automatically “sexist”, and certainly not a “hostile sexist troll” as you said on Twitter. Why not just engage in conversation?

                                        It seems to me that making any conversations or scepticism verboten would be far more harmful for your cause. How do you know you’re 100% right in your analysis of the situation and proposed problems? I’m all for calling out sexism or whatnot, but this is really just a disagreement.


                                        [1]: although I do think there is a kernel of truth in it, based on my own personal experience (which this margin is too narrow to contain); but I think they’re focusing on small details rather than the larger far more important issues.

                                        1. 3

                                          All you to have to do is actually, like, talk to some women in tech and hear the shit they go through. When told “it’s just the label that is screwing you over, just get over it” most people who have better things to do with their life will just go somewhere else, not waste time debating. So you lose people.

                                          I can’t say that Hillel trying to debate this person made any difference.

                                          “You should debate people who say you are less worth/less human/just making stuff up” is not how you make a welcoming community.

                                          1. 12

                                            I’m not going to go in to the matter itself here. My comment was not intended to take any position on the matter as such, mostly because typing out an entire essay here with the appropriate nuance and such which will take several hours to get right, and I have other things to do as well.

                                            But I will reply to this:

                                            people who have better things to do with their life will just go somewhere else, not waste time debating

                                            Who started the debate? That entire story had no bearing on the position of women in open source until someone rather forcefully and a abrasively injected it (and yes, the question comes off as a pretty abrasive challenging, almost an accusation).

                                            people who say you are less worth/less human/just making stuff up

                                            No one said anything like that. Do you really think those people think that woman are “less human”? I mean … like … what?

                                            1. 1

                                              No one said anything like that. Do you really think those people think that woman are “less human”? I mean … like … what?

                                              I didn’t mean that someone said that in this particular context. Rather, that is an explanation for why I think the second proposed moderation policy is useful, rather than saying “You should just debate those people.”

                                              1. 8

                                                Well, we’re discussion specific examples, so if you say something like that I think it’s reasonable to assume it’s about those specific examples we’re discussing.

                                                I think very few – if any – people here would be against banning overt sexism or the like, but the examples you’ve posted don’t really seem to exhibit that. As I said in my top-level comment: being incorrect is not the same as being sexist.

                                                1. 4

                                                  I think you and @itamarst may have gotten at a core disagreement, that he sees more expressions as overt sexism.

                                                  1. 10

                                                    Indeed; perhaps there’s an even more fundamental disagreement: whether or not these people are acting in good faith or not.

                                                    Generally speaking I tend to assume people are acting in good faith unless there’s a decent indication that they’re not, even if they say something stupid, harmful, or otherwise spectacularly wrong. This is perhaps a bit naïve because clearly there are bad-faith trolls – and I’m not afraid to call them out if there’s a good indication that they are – but I’d rather be a bit naïve than write people off as trolls so quickly.

                                                    This seems to be a general issue I’ve seen many times, both here and elsewhere.

                                                    I hate to use the term because it’s so often used by right-wing trolls (I can’t think of a better term), but this is probably what distinguishes a “Social Justice Advocate” from a “SJW”.

                                                    1. -2

                                                      Basically what you seem to be saying is that “sexism and racism is OK so long as it’s in good faith.” If someone is sincere in their beliefs, and not particularly trying to get a rise out of anyone, you are saying that is reasonable discourse that is acceptable in a community you’re part of.

                                                      From my perspective it’s not about good faith or bad faith, it’s about how certain positions inherently force you to choose which people are welcome. And I would argue that “don’t make sexism or racism allowable” is the better choice.

                                                      1. 5

                                                        That’s an extremely ungenerous reading of what I said. To quote myself from the top comment:

                                                        Sometimes people disagree on complex social issues 🤷‍♂️ If I look at this comment you linked then the good-faith interpretation of it is that they actually agree on the basic premise that there are problems with disadvantaged groups, but they just disagree on the solution. I don’t think being wrong is automatically “sexist”.

                                                        As I mentioned I always do my best to engage in good faith, but you’re giving me a hard time here…

                                                        1. 0

                                                          There’s two questions:

                                                          1. Should sexist or racist comments be allowed if they’re in good faith?
                                                          2. What counts as sexist or racist.

                                                          I am willing to discuss #2 if you are willing to say to #1 “should be not be allowed”, but otherwise there’s no point.

                                                          (Edited, original version got the negative wrong.)

                                                          1. 10

                                                            Should sexist or racist comments be allowed if they’re in good faith?

                                                            I literally answered this question already in my top comment: “I’m all for calling out sexism or whatnot, but this is really just a disagreement”. What more do you want? A bloody “I am not sexist” club membership card? Do I need to take a test? Is there certification involved? Can I put it on my CV?

                                                            I am retiring from this discussion as I’ve grown weary of it. I don’t need to prove a damn thing to you, especially not if your implied accusations are based on bad-faith readings of posts I carefully spent some time writing. Talking to you seems like playing a game of “avoid being called sexist”. There is no value for me here at all, and only frustration and accusations.

                                                            Somehow you managed to misconstrue everything I said completely different from the first reply with your ridiculous “people who say you are less worth/less human/just making stuff up” comment which is apparently about hypothetical other people or something, rather than the people and comments we’re discussing. What kind of debating tactic is that? Sjeez…

                                                            So yeah, I’m done with it. The only thing that has turned me off from social justice is the last few years have been its proponents, not its detractors. Ya’ll impossible to engage with.

                                                            1. 0

                                                              At no point did I say you were sexist.

                                                            2. 4

                                                              If I read this correctly, you’re willing to “discuss” one point if the other party is willing to cede the other without question?

                                                              I don’t believe I could agree to such terms, and I’m pretty sure nobody seriously interested in engaging in genuine discourse could, either.

                                                              (And I say this as somebody who believes that sexist or racist comments categorically don’t belong here on lobsters, good faith or not, because they are not relevant to the subject matter.)

                                                              1. -2

                                                                They didn’t say that anyone had to agree.

                                                                They said that they are willing to discuss IF that is the case. It is likely meant to be a nice way of saying that if you don’t agree with that then the conversation need not continue and it is likely not someone trying to force you into a decision.

                                                                People are allowed to state boundaries without it being a requirement of some sort to concede.

                                                                Furthermore, this person has already stated that they are “turned off” from “social justice”. Their intentions here are clear as freshly cleaned glass. Nobody else has said it, but they are being sexist if they think that supporting women being treated equally is something that you can be “turned off” from.

                                                                As I have mentioned elsewhere, the only thing that has turned me off from social justice is the last few years have been some of its proponents and not its detractors.

                                                                https://lobste.rs/s/lpvcsm/proposal_for_moderation_policies_no_tone#c_97hpgq

                                                                Interestingly, this person has also joined in the last couple years during the time that I’ve noticed the worst degradation of discussions on this site. Maybe a coincidence since it’s just my experience, but at some point I guess that every community becomes another HackerNews, Reddit, whatever…

                                                                1. 7

                                                                  Furthermore, this person has already stated that they are “turned off” from “social justice”. Their intentions here are clear as freshly cleaned glass. Nobody else has said it, but they are being sexist if they think that supporting women being treated equally is something that you can be “turned off” from.

                                                                  Do you really think it’s impossible for someone to completely burn out because of the people behind the mission and want nothing to do with it in spite of wanting the core of the mission to succeed? That there’s no difference between supporting the methods and arguments for a cause vs supporting the cause?

                                                                  Your paragraph right there is phrased as such and I find that kind of weird.

                                                                  1. 4

                                                                    Furthermore, this person has already stated that they are “turned off” from “social justice”. Their intentions here are clear as freshly cleaned glass. Nobody else has said it, but they are being sexist if they think that supporting women being treated equally is something that you can be “turned off” from.

                                                                    The problem isn’t the cause, it’s having to deal with people like you. But yeah quote me out of context all you want to confirm your witch-hunt.

                                                                    1. -1

                                                                      interestingly, in my view, Lobste.rs is already far worse than HN/Reddit, etc. I think these threads are pretty instructive of why – the balance of active and engaged users along the political spectrum is off. most of the hard left is gone, and the rest of the site is center-right to hard right. and it can be seen easily in which topics are labelled as political and which aren’t.

                                                                      there’s probably also something to how small the lobste.rs community is I think and how that makes the imbalance so much more glaring.

                                                                      EDIT: actually, the issue is quite obvious now that I think about it. a website for whom membership is invite based needs to have a diverse group of “seed users”. I’m fairly sure you could track the rightward slide of the site with a gigantic tree view of users invited on a timeline.

                                                                      1. 3

                                                                        We could test that at a single data point right now, you know. We were invited by the same user (what are the odds, eh?). I’m not sure if it would hold true for the rest of the users though.

                                                                        I would be center left based on wikipedia’s explanation. I don’t keep track of this stuff tbh.

                                                                        I would like to add that the information that I can glean from the comments and stories on this site is greater than what I can gather on Hackernews and Reddit. The conversations are higher quality most times as well.

                                                2. 1

                                                  I don’t think being wrong[1] is automatically “sexist”, and certainly not a “hostile sexist troll” as you said on Twitter

                                                  The Twitter thread, for context: https://twitter.com/itamarst/status/1279366605185613827

                                                1. 74

                                                  Telling people they’re “being aggressive” or “controversial” adds no meaningful information to a conversation.

                                                  It’s an important meta point to make, either signalling to them that they can say something more constructively or signalling to other users “hey this person is probably not going to engage in good faith, head’s up.”

                                                  No pretending systemic racism, sexism, and bias aren’t a thing.

                                                  I don’t think anybody seriously believes that in the abstract those aren’t things, or that concretely in other places they do not exist. However, we aren’t really prepared to talk about those things here with any sort of rigor and clarity, and if we were the discussions would take a whole bunch of space away from the bread and butter of the site. There are many, many places elsewhere that do a better job discussing these things.

                                                  Telling people their experiences aren’t real, or are caused by labels, or pretending that it just so happens that most people in our industry look a certain way just makes the problem worse.

                                                  My lived experience is that there is a concerted effort to politicize technical spaces and also to shutdown conversation in cases where people disagree with those politics. Is my experience not real to you?

                                                  When it comes to “most people in our industry looking a certain way”, this is a whole thing, right:

                                                  Times being what they are, though, I think that you probably meant “why are there so many cis white males in programming?” Not only is your statement loaded, it is reductionist and erases the unique identities of hundreds of thousands, if not millions, of people–something theoretically you are against!

                                                  There is gigantic implicit provincialism in the discussions around race in tech, which is understandable given that the background of the people talking can influence things so much (for example, Germans developers talking about the plight of blacks may be somewhat lacking in first-hand experience). Since Americans do represent a plurality (right now) of the software engineering demographics, it makes sense that the conversation is going to be heavily slanted in favor of the issues of race in the United States–but those issues are very different once you look at any other diaspora. Issues of racism in Japanese technology companies look very different from Oakland.

                                                  One can make similar arguments about perspective for any other problematic partitioning. On sex I’ll only point out that you have to really get deep into the demographics and sociology to make compelling points, and that’s again a lot of space to spend here away from our core content–and space that ultimately won’t fix anything. If you want just more women, fire some men and hire women; don’t post on Lobsters. If you want to be more inclusive, go counterbully the people that are directly bullying women on Twitter instead of freaking out here when somebody says “guys” instead of “folks”.

                                                  ~

                                                  I really dislike this suggestion because not only does it stifle discussion, it establishes a beachhead for politics in this community that isn’t deserved or earned, and perhaps most importantly it doesn’t even fix the problems those groups actually have.

                                                  1. 14

                                                    My only regret is that I have but one upvote to give to this comment. Thank you for putting it in succinct, respectful, terms.

                                                    1. 3

                                                      I don’t think anybody seriously believes that in the abstract those aren’t things, or that concretely in other places they do not exist.

                                                      I have seen people arguing exactly that, yes.

                                                      There are many, many places elsewhere that do a better job discussing these things.

                                                      I agree with this - infinite-nesting discussions are not good at discussing these topics.

                                                      1. 2

                                                        Bit offtopic, but thanks for those links, especially the qz one. I had been searching for such numbers more than once already.

                                                        1. 3

                                                          On sex I’ll only point out that you have to really get deep into the demographics and sociology to make compelling points, and that’s again a lot of space to spend here away from our core content–and space that ultimately won’t fix anything. If you want just more women, fire some men and hire women; don’t post on Lobsters.

                                                          This constitutes illegal gender-based discrimination under current American civil rights law. Talking about whether this kind of gender discrimination is good or bad is inherently political, since it necessarily involves questioning whether a specific body of law is good or bad.

                                                          1. 0

                                                            I am not trying to politicize lobste.rs. I am saying that everything is political (in terms of “how power is distributed”) and so it’s already political in a particular way.

                                                            I do not recall saying anything about the word “guys”, though.

                                                            1. 22

                                                              I am saying that everything is political (in terms of “how power is distributed”)

                                                              I strongly disagree with that idea (eg how “A simple to use Java 8 JWT Library”. is about power?).

                                                              But even if it were true, the power dinamic could only be relevant in your circle and in your city/country, and the world is much much bigger than the USA.

                                                              1. 0

                                                                “A simple to use Java 8 JWT library” is not about power. Saying “this article is OK, this isn’t” is about power.

                                                                1. 18

                                                                  No, it is about focusing the discourse on the target discipline. There are others places where things like the power struggle as described in critical theory can be discussed.

                                                          1. 1

                                                            If I use Spark in my pipeline then I don’t need this is that right?

                                                            1. 1

                                                              I know of one company that wants to put a lot of resources into reducing memory usage in their Spark jobs. So you might want it :)

                                                              In practice, however, I expect it won’t work with Spark.

                                                            1. 1

                                                              Neat idea, interested in seeing where this ends up.

                                                              One big thing that’s missing is issue trackers. A lot of folk knowledge only resides in various issue trackers.

                                                              1. 1

                                                                Atm, “repos” filter includes github, gitlab, sr.ht and sourceforge (https://github.com/freetonik/quickref.dev/blob/master/sites/repositories.txt). Are there other issue trackers I should add?

                                                                1. 2

                                                                  Oops, sorry. I guess the issue is then “it didn’t order search results in way I expected”, then.

                                                                  I maintain somewhat obscure logging library named eliot, but searches for eliot logging python pandas do not return the issue in the Eliot issue tracker covering Pandas support. Plausibly too hard to get this right in a general search engine, so not sure it’s a flaw or just a hard-to-fix edge case.

                                                                  Apache’s JIRA might be useful.

                                                                  1. 1

                                                                    Ah, seeing that it’s driven by Bing, my guess is Bing (and definitely Google) considers individual GitHub issues in less popular repos to be not worth indexing.

                                                              1. 2

                                                                Speaking for myself, I’ve never been promoted or hired based on technical problem solving. It’s always been about being more impactful to the business by working on the human side. It took a while for me to get excited about that, as coding’s what’s drawn me to the job. I think it’s related to what the authors of Peopleware call “the high-tech illusion”.

                                                                That’s based on a short commercial career of ~7 years, but it’s been consistent enough that that’s my mental model of providing value to startups nowadays (which tends to become self-fulfilling.)

                                                                In startups, I’ve never worked with programmers considerably more senior than where I am now (the fact that someone inexperienced as myself gets job titles such as “senior engineer” and “tech lead analyst” disturbs me, but I brush most of it off to title inflation). I’ve always asked myself what kind of company hires programmers at the level the article describes, as I’d love to join such an organisation. Would be great to be able to focus more on the technical.

                                                                1. 2

                                                                  None of this problem solving is necessarily technical. Often it’s a range, from “deep technical problem” to “development process problem” to “organizational problem requiring negotiation”.

                                                                  1. 1

                                                                    It’s always been about being more impactful to the business by working on the human side

                                                                    Can you please elaborate on the “ Human side “ ? Thanks.

                                                                    1. 2

                                                                      “Human side” was a poor way to put it.

                                                                      My point is that getting better jobs correlates with providing better value to the company, and, even in so-called “tech companies”, the value that better technical solutions can provide is marginal. We as developers tend to focus on technical problems because those are to control and, frankly, more fun most of the time, but that’s not addressing the bottleneck.

                                                                      In contrast, there’s much lower-hanging fruit to be had by improving team productivity through better understanding of the model, better frameworks for collaboration, tighter feedback loops, higher team resilience, quality management, etc. Developing those skills has given me a much greater ROI. It tends to be the skills my team hires for, beyond entry-level positions.

                                                                  1. 2
                                                                    1. My new memory profiler for Python is now publicly available as open source (https://pythonspeed.com/products/filmemoryprofiler/), so next up need to do some promotion.
                                                                    2. More research on security scanners for Python+Docker.
                                                                    3. Working on more chapters and related articles about programmer productivity for my book on the topic (https://codewithoutrules.com/secretskills/).
                                                                    1. 4

                                                                      Microoptimizations like “don’t do attribute lookups” are all fine and good, but it’s workarounds for super-slow runtime. PyPy would run this code much faster, numba JIT too. If you care about performance, plain CPython is not how you should be running your numeric code.

                                                                      1. 3

                                                                        Given the answer to every vague software question is “it depends”, you can’t just talk about “Lisp”.

                                                                        I can’t install “Lisp”, I can install Clojure or Racket or SBCL; each has its own different tradeoffs, libraries, and communities. So the question isn’t useful from the get go, because software in the real world is about specific situations and resulting tradeoffs and engineering decisions.

                                                                        I wouldn’t write the flight search engine that powers Google Flights in Python; having worked with the people who wrote it, Common Lisp actually seemed like a good choice. But I wouldn’t use Common Lisp for 99% of the software I’ve written over the past few years. I don’t want to write every tiny utility I might need from scratch because there’s no libraries. And the language itself wouldn’t have added much.

                                                                        And while Clojure definitely and probably Racket are much better ecosystems than CL they still wouldn’t help me much with the job that did scientific computing—I guess Clojure + ImageJ would have worked just fine, but Clojure wouldn’t have added anything, and the organization was already using Python.

                                                                        1. 6

                                                                          To add to what sjamaan says, “comic images, unusual text layout, and a fantastic sense of humor” is missing the point a little. The books’ design was based on a specific set of theories about teaching, covered in Sierra’s book “Badass”. The theories don’t require that particular design, for that matter.

                                                                          There some excellent talks she gave on the subject available online: https://www.youtube.com/watch?v=FKTxC9pl-WM

                                                                          1. 6

                                                                            This has the same problem as referencing a Docker image by hash:

                                                                            RUN apt-get install -y nginx=1.14.2-2+deb10u1
                                                                            

                                                                            There will eventually be another nginx release, and eventually the version you pinned will disappear from the repository. To make this work you’ll need to maintain an apt repository that retains every version you’ve ever pinned.

                                                                            The “starting point” advice at the bottom is good, however.

                                                                            1. 2

                                                                              The “starting point” advice at the bottom is good, however.

                                                                              I’m not sure if it is, tbh. Yes, pinning versions of packages in pip’s requirements.txt is good advice if you want to rebuild something and know it will work. But it’s bad advice if you care about security fixes, bug fixes that newer versions can bring (along with the risk of breaking changes/new bugs).

                                                                              There’s a balance somewhere in there between “never update anything, ever” and “edge or GTFO”, how do you find it?

                                                                              1. 3

                                                                                Finding that balance is definitely tough.

                                                                                I think pinning is good for reproducibility, but it’s important to bump your dependencies to whatever is getting security fixes. Unfortunately not every package has a long-term-support channel.

                                                                                1. 2

                                                                                  Yes, you have to periodically re-pin at the least.

                                                                                  Also, just having a full list of your dependencies is useful. For example, GitHub can generate security notices if you’ve pinned a known-vulnerable version. Dependabot knows how to automatically issue update PRs when new releases occur.

                                                                                  1. 2

                                                                                    That’s why you need tests, and lots of them: then you can safely bump dependencies and have your CI pipelines catch issues that arise (if you are using GitHub, Dependabot’s PRs will get automatically tested).

                                                                                    Of course, some dependencies (those interfacing with outside systems) will be mostly mocked out in tests, so you will still need to do some manual testing if those get updated.

                                                                                  2. 2

                                                                                    The longer I do this, the longer I think I need a way to get notifications from dependencies when they patch a security issue, so I can make sure to apply it.

                                                                                    That’s my path to get off the treadmill of edge while keeping security patches applied.

                                                                                    I’m a fan of the counterintuitive approach proposed by one of the go package managers - installing the oldest version that matches your dependency list instead of the newest.

                                                                                    This reduces the need for a lock file (older versions do not suddenly appear) and tends to give you combinations of software that are compatible with each other.

                                                                                    The downside is you now need a way to figure out what versions you can/should bump, and tooling to apply those changes to your dependency list. However, those have been implemented.

                                                                                    1. 2

                                                                                      GitHub will do this for you if your dependency-spec file (for any of various languages) is checked in.

                                                                                      For Python specifically, there are tools like pyup and requires.io that will notify you of new releases of dependencies (and which ones are security issues). I’m sure there are similar services for other languages.

                                                                                    2. 2

                                                                                      I recommend pinning everything and having automation to update those pins. A tool like https://github.com/renovatebot/renovate can automatically create pull/merge requests and merge them if the tests pass.

                                                                                      1. 1

                                                                                        There are other articles on the site about doing security updates and the like, but it’s true it’s not mentioned there. I’ll try to update it with some links tomorrow. Or, perhaps, that may be a whole new article to write…

                                                                                      2. 1

                                                                                        I like to use https://hub.docker.com/r/debian/snapshot which is based on a specific timestamp of snapshot.debian.org

                                                                                      1. 1

                                                                                        Regarding the security point specifically: it would be nice if there were more rigorous frameworks for reasoning about such things.

                                                                                        E.g. “How do I take into account, model and parameterize all possible attack vectors, including NASA or random hackers”

                                                                                        1. 3

                                                                                          There are books on threat modeling that actually go into a variety of models.

                                                                                        1. 4

                                                                                          This benchmark is probably inaccurate; NumPy is plausibly both faster and slower than the article claims!

                                                                                          The issue is that by default NumPy’s BLAS backend is multi-threaded (see https://pythonspeed.com/articles/parallelism-slower/). So wall clock time isn’t a good measure.

                                                                                          I ran the same Python code, once with normal settings, which is multi-threaded BLAS:

                                                                                          CPU times: user 3.84 s, sys: 517 ms, total: 4.36 s
                                                                                          Wall time: 1.52 s
                                                                                          

                                                                                          And once with export OMP_NUM_THREADS=1 so that it uses single-threaded BLAS.

                                                                                          CPU times: user 2.89 s, sys: 128 ms, total: 3.02 s
                                                                                          Wall time: 3.05 s
                                                                                          

                                                                                          Notice that in single-threaded mode the wall clock time is double but total CPU time is much lower. So NumPy could potentially be both faster and slower than this article’s benchmark says.

                                                                                          (There can also be benchmarking artifacts from hyperthreading, which I have turned off, and turboboost, which I haven’t turned off yet.)

                                                                                          1. 4

                                                                                            The thing I’m always missing in these data-oriented design tutorials is how the reorganized data structure integrates back into the higher level application’s logic. Like in this case, you’ve decomposed the game object into this array of dense structs - how are those FooUpdateIn structs related back to data from the original GameObject when necessary? Is this all typically decomposed into arrays of structs and index ids are used?

                                                                                            Edit: I guess what I’m getting at is - is there any way to easily mix in a bit of this pattern into an existing codebase?

                                                                                            Taking the blog post as an example, I could change GameObject to storing a pointer to a FooUpdateIn, but then if those are stored in an array, how do I manage FooUpdateIn lifecycle without completely switching to an entity system (to deal with pointer invalidation on array resize) or some form of pooling/custom allocation scheme (to keep from invalidating pointers, but also re-allocating any unused indexes as deletes make the array sparse).

                                                                                            It seems like the easiest ‘drop in’ would be to have GameObject hash an id into a map of FooUpdateIns to find its own, then UpdateFoos can iterate the map’s backing array - but how do I reason about sparse entries in the backing array, or is perf still so improved that it doesn’t matter? (and annoyingly, cpp stdlib doesn’t ship with a useful map for doing that)

                                                                                            1. 2

                                                                                              You can hash from entity ID to array index, and then also pack entries tightly in the array. When you delete an entity you swap its data with the last element in the array for O(1) removal.

                                                                                              If you go full entity-component system you really want to have all entities of the same “archetype” (Unity term - entities with the same set of components) to be packed together so you can quickly iterate over all the entities with some set of components, but actually coding that is incredibly annoying in C++

                                                                                              1. 1

                                                                                                The idiom (from game programmers’ perspective) is called Data-Oriented Design. You can find online resources, including a book, about it: https://github.com/dbartolini/data-oriented-design

                                                                                                Scientific computing, coming from a very different direction, has ended up in a very similar place (originally leading back to APL I think), but the domain is different enough that it probably isn’t relevant to your interests.

                                                                                              1. 19

                                                                                                I think the principle of least power might apply here. Do you really want a full language here. Seems ripe for abuse.

                                                                                                1. 11

                                                                                                  I think it depends on the domain. For ops work, it’s quite difficult to come up with a language that is simple enough while still being powerful enough.

                                                                                                  I’ve chosen Python as config file for a Docker template product I sell. Here’s my reasoning:

                                                                                                  You can say, ok, we’ll just have them use JSON. But now they’re writing code to generate the JSON, so we’re no better off.

                                                                                                  I could do something more powerful than JSON, but less powerful than Python. But—users might want to set Docker labels based on Git. So, great, I can hard code that as flag, but—what if they use Hg? What if they use Perforce? And that’s just one detail out of a thousand.

                                                                                                  So it’s much easier to say “Here’s a language you can mostly just use to construct dictionaries, just like JSON, but if you need it, you can do whatever”. And in my case it’s a template for Python applications, so I can assume Python is there.

                                                                                                  A longer write up on the pain of writing software for ops, and how I arrived at the decision to do what this article suggests: https://pythonspeed.com/articles/developing-tools-for-ops/

                                                                                                  1. 5

                                                                                                    You should check out Starlark. It looks like Python but it doesn’t let you do a lot of the crazy dynamic stuff Python can do and it doesn’t let you do I/O or have unbounded loops. It was built for exactly this purpose: https://go.starlark.net

                                                                                                    1. 2

                                                                                                      I can see places where that would be useful, but it doesn’t help with things like “call out to a git subprocess to get the current branch”. Sometimes you just need a full programming language.

                                                                                                      1. 1

                                                                                                        I think we’re talking about different use cases then. I have a particularly hard time imagining when calling out to git is appropriate for this sort of configuration. It seems like you would want to do that ahead of time and pass the result into your configuration script as a parameter. That said, this is a pretty broad space, so maybe there are valid use cases and I’m just not imagining them. Mostly I deal with CI/CD pipelines and infra-as-code.

                                                                                                        EDIT: By the way, I don’t think it’s inherently bad to use Python, but I do think you need to be disciplined and avoid doing I/O when you don’t need to.

                                                                                                        1. 1

                                                                                                          In that scenario the code that calls out to Git “ahead of time” is configuration from my perspective, because each organization writes its own variant.

                                                                                                  2. 6

                                                                                                    In my experience it’s nearly always a mistake, and only ever viable long-term if the configuration “program” is completely dead before the configured program starts to run. Once you can have a thread moving in and out of “configuration” code at run-time, you’re lost.

                                                                                                    1. 4

                                                                                                      Agreed. Turing complete configs and config generators have often been my personal hell. If you need that much power in your configs, something has probably gone very wrong in the design phase.

                                                                                                      1. 6

                                                                                                        I think Turing completeness is a red herring, since even with a language like Dhall you can write something that will finish evaluation after you’re dead.

                                                                                                        1. 3

                                                                                                          I think the real factor is I/O. People will try to call out to the network or do other state full things that break in weird ways.

                                                                                                        2. 2

                                                                                                          The problem isn’t Turing complete configs - they’re inevitable as you try to capture all the levels of exceptions your environments have and how those interact.

                                                                                                          The problem is accidentally Turing complete configs, as a language that organically grows the power needed to do non-trivial configuration tends not to grow it in ways that are best designed in a formal sense.

                                                                                                          1. 5

                                                                                                            The problem isn’t Turing complete configs - they’re inevitable as you try to capture all the levels of exceptions your environments have and how those interact.

                                                                                                            Yes, and the goal should be to start normalizing environments, rather than shoving more and more into an increasingly complex configuration.This is precisely what I meant by something going wrong in the design phase.

                                                                                                            1. 2

                                                                                                              It’s very rare that that’s possible. There’s production vs development vs test vs developer’s machine, each cluster has a different capacity in accordance with regional needs and redundancy requirements, some clusters are on the older hardware/OS/BIOS firmware, you’re canarying some features only in some clusters, you’re in the middle of rolling out a new software version which has different resource requirements, and some clusters don’t have some subcomponents as legal doesn’t want you running that subcomponent in that jurisdiction.

                                                                                                              Real life is messy, and configuration has to be able to capture it. No matter how much you work on making things consistent, there’s always going to be new variances coming up.

                                                                                                              1. 3

                                                                                                                Yes, and given the time spent on this, I regularly find the investment in simplifying the environment is not that different from going all the way on configuration, and pays dividends in system maintainability down the line.

                                                                                                                1. 2

                                                                                                                  The environment is already as simple as it reasonably can be, this is irreducible complexity. You can’t exactly tell people to stop buying new hardware, stop traffic growth, stop developing features, or stop doing releases.

                                                                                                                  The question then is how do you deal with managing that environment. That is the true problem of configuration/change management.

                                                                                                                  1. 1

                                                                                                                    I’m not sure how you can say that with a straight face.

                                                                                                                    However, if you really want to go that way, you can do even better than turing-complete configuration languages. Just hard code things. That means that you get full visibility into your setup with your monitoring tools, can step into it with your debugger, get well integrated logging, get full checking with static analyzers (which will even allow you to catch configuration bugs at compile time), and so on.

                                                                                                                    Because you’ve exposed your current configuration to your linter or static type checker, you’ve actually the confidence you can have in your changes.

                                                                                                                    And then, rolling out a new configuration is as simple as rolling out a new binary.

                                                                                                                    1. 2

                                                                                                                      I think we’re talking about vastly different systems here in terms of scale and complexity.

                                                                                                                      1. 1

                                                                                                                        I’ve worked on Google’s display ads infrastructure.

                                                                                                                        So, why is “write your config in a real programming language” reasonable, but “and do it in the same language, statically analyzed, tested with the code it’s changing the behavior of, deployed using the same continuous rollout pipeline, and integrated with the rest of your service’s infrastructure” a bridge too far?

                                                                                                                        If your objection is incremental rollouts – keep in mind that you can already have different behavior per node by filtering by cluster, hashing the node ID and checking a threshold - which you’d probably be doing in a turing complete config anyways. (Just, if it’s like most places I worked, with fewer unit tests).

                                                                                                                        If your objection is observability, you’re already exporting all service state via /flagz or similar, right?

                                                                                                                        1. 1

                                                                                                                          I’ve worked on Google’s display ads infrastructure.

                                                                                                                          I was an SRE for those systems.

                                                                                                                          why is “write your config in a real programming language” reasonable

                                                                                                                          I never said it was reasonable, and my experience is actually that imperative languages don’t work too well once you get into the really complex use cases. You want declarative with inheritance, to capture all the nested levels of exceptions within a given configuration. Even then you need to factor it carefully, and do most of the other code health stuff you would with “real” programming.

                                                                                                                          Because you’ve exposed your current configuration to your linter or static type checker, you’ve actually the confidence you can have in your changes.

                                                                                                                          I’m not seeing how any of this has anything to do with static analysis, nor how it makes a difference either way in terms of monitoring, debugging, or logging.

                                                                                                                          Confidence comes from seeing that the settings I want to change ended up changed in the dry run of the config change. Whether it actually has the desired ultimate effect is a different question entirely.

                                                                                                                          And then, rolling out a new configuration is as simple as rolling out a new binary.

                                                                                                                          That’s assuming that the cost of rolling out a new binary is effectively free. Building and running the automated tests for the binary could take rather a long time. That’s even worse if you want to quickly iterate on a setting on a single process before doing a broader rollout. Baking settings into binaries also doesn’t cover settings that can’t be baked into binaries, such as resource and scheduling requests.

                                                                                                                          1. 1

                                                                                                                            That’s assuming that the cost of rolling out a new binary is effectively free. Building and running the automated tests for the binary could take rather a long time.

                                                                                                                            Sure, but config should also be running integration tests. Especially if it’s in a complex, turing complete language. I’d be rather surprised if config changes weren’t the source of comparable numbers of outages as code changes. Either via tickling a bug in existing code that shows up when config options are changed a certain way, or simply via being wrong.

                                                                                                                            And the more complex your configuration, and the more power you put into the config language, the closer it gets to being “just more code”, but with worse quality control.

                                                                                                                            So, if you go 90% of the way and give up on fighting config complexity, you may as well go the last few percent, and give it proper code treatment.

                                                                                                                            1. 1

                                                                                                                              You’re conflating two things there.

                                                                                                                              Firstly how should you safely deploy a config change, presuming that it’s changing exactly the config it’s meant to change. Strategies there vary, and a code-like change (e.g. enable user-visible feature X) versus an operational change (e.g. increase the size of a threadpool) are best served by different types of testing. Having to wait for multi-day integration tests that involve multiple other humans for a threadpool tweak isn’t reasonable, on the other hand looking only at CPU usage when you’ve enabled a new feature isn’t right either.

                                                                                                                              Secondly how do you ensure that you’re only changing the config that you expect to change. You can get pretty far by diffing the raw output configs, as unlike standard programming we have a complete set of possible inputs (i.e. all the config files). So it is practical to do a mass diff and have a human glance over it on every change. Unittests would likely hinder more than help here, as they’d either be overly specific and just create busy work on every change - or not specific enough and never catch anything.

                                                                                                                              1. 1

                                                                                                                                Secondly how do you ensure that you’re only changing the config that you expect to change. You can get pretty far by diffing the raw output configs, as unlike standard programming we have a complete set of possible inputs (i.e. all the config files).

                                                                                                                                The proposal in the article was ‘import config’ as pure python code, and just execute it. There’s no raw output to diff in the author’s suggestion. And it can behave differently at runtime, inspecting the node that it runs on or the time of day, and doing different things based off that. If you’re doing that, I strongly think you’re going the wrong way. But if you’re going down that wrong path, take it all the way and integrate your “config” with the application fully.

                                                                                                        3. 4

                                                                                                          I don’t know how much you know Lua but it is a prime language for using like this. There are built-in features to create sandboxes which basically allow you to run untrusted code in a very restrictive environment. You can specify exactly which functions the untrusted code has access to and using the Lua C API, you can tie that environment into useful parts of your larger app.

                                                                                                          1. 2

                                                                                                            I think it REALLY depends upon your use case. Is this config for an internal tool that’s low impact? Maybe. Do we have to worry about hostile or under-informed users? That changes things.

                                                                                                            1. 1

                                                                                                              The problem is for many domains, more dynamism really is necessary. However, you can restrict unbounded looping and various I/O by using an embedded scripting language that supports those features, such as Starlark. https://go.starlark.net

                                                                                                            1. 4

                                                                                                              In the toy example, why is exec used? Why not just from config import PEOPLE as people?

                                                                                                              1. 7

                                                                                                                Good question! Mostly the same, but I discovered few differences:

                                                                                                                • your config file might have a different name (or a name that can’t be a Python module name). That can be worked around with importlib though.

                                                                                                                • your config might not be in your PYTHONPATH, then you’d have to mess with sys.path to be able to import it

                                                                                                                • the biggest difference is that you can pass a context to exec.

                                                                                                                  In my toy example, imagine that the config was supposed to modify PEOPLE (so it could remove people from the list as well). You can achieve that with:

                                                                                                                  all_people: List = ... # some code that loads people from the database
                                                                                                                  
                                                                                                                  globs = {'PEOPLE': all_people} # execution context for the config
                                                                                                                  exec(Path('config.py').read_text(), globs)
                                                                                                                  
                                                                                                                  # all_people is now modified by the config
                                                                                                                  

                                                                                                                  If you were to use import, you’d have to define callbacks to modify PEOPLE, which may be less aethetically pleasing (more nesting).

                                                                                                                  As a real example: Jupiter config lets you modify the configuration as you want, e.g. append preprocessors. In config.py file, you call get_config and get the configuration object. In order to bring get_config into the scope, I’d imagine Jupiter does something similar and passes globs = {'get_config': get_config} to exec.

                                                                                                                So differences are fairly minor, but exec is a bit more flexible, so I prefer it now. However, for lazy and quick configuration files import config will do 95% of the job.

                                                                                                                1. 2

                                                                                                                  I used import to load a config file for an old Python project years ago, and the biggest pain points were the sys.path frobbing as you mentioned, reloading after changes (there are ways to do it, reload() in Python 2, importlib in Python 3, but there were some caveats with those, although I’ve since forgotten the details), and that it cluttered the FS with .pyc and __pycache__ stuff. Maybe there were other issues as well; this was years ago. I eventually switched it to exec().

                                                                                                                  Here’s the config for that by the way: https://github.com/arp242/battray/blob/master/data/battrayrc.py; I think it worked out rather well. It’s not hard to see how you could do with with a purely declarative config file, but basically that’s just wrapping a bunch of ifs.

                                                                                                                  1. 1

                                                                                                                    Passing a context to exec would be an anti-pattern to me. Explicit is better than implicit. The configuration should explicitly import whatever it needs.

                                                                                                                    Likewise, the PEOPLE variable is magical. If you make a typo, have fun debugging that. Instead you can import a register_people function and use it. Following that pattern, there is no need to create the namedtuple Person yourself. Import the data structure from the application and you can also use the methods from it.

                                                                                                                    I spent a while maintaining a buildbot configuration. We even had a few “unit” tests for the configuration.

                                                                                                                    1. 2

                                                                                                                      I don’t disagree – one is less nested, but implicit, another is more verbose, but explicit. Both can be type checked and linted (e.g. with Protocol), so I would say the differences are pretty minor.

                                                                                                                      Yep, you’re completely right about importing People, that’s what I would do in my tools! I just wanted to keep the example simple.

                                                                                                                      Something like jupiter config I’ve liked feels like a nice compromise? The only ‘implicit’ thing is import_config, which is hopefully not that magical (and yeah, also can be imported from the package as you suggested)

                                                                                                                  2. 1

                                                                                                                    Having made the same engineering choice (Python script for config, using exec), exec means you can just figure out the path to the code and be done with it. Imports require manipulating sys.path, which has the potential for breaking things if e.g. there’s another Python file in there with the wrong name.

                                                                                                                  1. 5

                                                                                                                    The scientific Python community is amazing. Astronomy, image processing (usually biology), earth sciences, just learned about magnetoencephalography and electroencephalography, bioinformatics, chemistry… list goes on and on.

                                                                                                                    1. 2

                                                                                                                      Doesn’t multi-stage build solve the same problem? You pass the secrets to the first stage then copy the results to the second. I’m no Docker expert so I might be missing something here.

                                                                                                                      1. 4

                                                                                                                        It does solve the problem… but introduces a new one. The new problem is that once you do that, you can’t push the first stage to the registry, because then you’re leaking secrets. Which means every time you build your image, you have to rebuild it from scratch, you can’t rely on build caching.

                                                                                                                        For Go applications that’s often fine, but for some languages that means 20 minute builds.

                                                                                                                      1. 2

                                                                                                                        CUDA is used extensively (eg. research papers about computer graphics), what makes you doubt it so much?

                                                                                                                        My general opinion on this is that a GPU has perfomace cliffs which are easier to control if you have a lower level language (if we can call CUDA low level). For example when having bank conflicts due to the size of a wave or the access pattern.

                                                                                                                        As always, if you want performance, you have to put in the work to understand the internals and doing so with a language not so high level is easier.

                                                                                                                        1. 3

                                                                                                                          Everything you say about GPUs is also true of CPUs, and compilers have gotten quite good at making code work fast on CPUs without exposing every single tiny implementation detail. And yes, you need to understand the underlying hardware, but the compiler seems a lot more helpful for CPUs than for GPUs.

                                                                                                                          And I think the issue is that C/C++ isn’t really designed for GPUs.