1. 22
  1.  

  2. 5

    As an organiser of the Berlin Meetup and the European Rust Conference (http://rustfest.eu), this sentence makes me very happy: Europe loves the language :).

    We also asked where people who responded lived, and over 1000 of the 3000 survey responses mentioned living in Europe (with USA following it up at 835).

    1. 1

      And here is a great example of how it can be really useful to subdivide your datasets a bit better. A simple sum across the bars fails to give helpful demographic information (except for the ‘no’ column, which is a nice way of saying straight white young male, one supposes).

      1. 2

        Since you could choose multiple choices, you can’t compare them against each other, only against the total number of responses. The goal of the question wasn’t to get exact demographics, but to figure out relatively where we can improve with outreach.

        1. 1

          Sure, but even that chart would’ve benefited by being broken into:

          • race
          • age
          • orientation

          For example, it’s entirely possible (not having seen your dataset) that the “woman” and “trans” results were both picked in by the same people–which would mean that you have not great representation of trans but absolutely terrible representation of women. That could lead you to incorrect conclusions.

          I don’t mind diversity work, but we should make sure that–if we do decide to care about identity stuff–we measure it properly.

          For the age one, you could’ve gotten the same information (or if you did, presented it as such) by directly polling ages or asking about what range people fall into.

          EDIT: Also, suggested the culture tag, because that’s what the article is about–the culture of the Rust community! :D

          1. 5

            For example, it’s entirely possible (not having seen your dataset) that the “woman” and “trans” results were both picked in by the same people

            “Trans women” are women (and often struggling with a combination of multiple issues), that’s the entire point of having the question asked like that.

            Age: this number globally diverges a lot. It’s better to directly ask whether people feel like they are outliers to get actionable results. The age of the person does not give me the same information.

            1. 1

              For age, asking both actually give you much better information–both the actual age distribution and also whether or not they feel underrepresented would help figure out if people are just perceiving things incorrectly. Over half of people, after all, view themselves as above average!

              As for the “trans women” thing, I would (at some risk of getting yelled at, probably) suggest that (intersectionality aside) you get worse data by effectively masking off the difference between cis and trans. Again, that’d be a good place to have different questions (like on a dating site…what is your gender and did it match your sex at birth).

              In theory, yes, transwomen are women and so it shouldn’t matter. That said, in practice, the issues the two sets of people (ciswomen and transwomen) are somewhat different. If the goal is to increase diversity, failing to recognize the unique challenges faced by one group because it sees itself as another (different) group seems to be counterproductive–especially when the challenges really are different between the groups.

              1. 4

                For age, asking both actually give you much better information–both the actual age distribution and also whether or not they feel underrepresented would help figure out if people are just perceiving things incorrectly. Over half of people, after all, view themselves as above average!

                The question is framed deliberately whether people feel as part of an underrepresented group, not to find out whether we should explain them they are wrong. There’s interesting research about mapping between perceived and actual discrimination and that’s a topic you can write PhDs about. It’s not our field of research.

                Also, the practical impact of perceived discrimination is the same as for actual discrimination from our point of view: people don’t show up if they don’t feel like the community is taking active measures or directly speaks towards them.

                As a community worker interested in growth, it is of no interest where that comes from and not my job to explain people their reasons.

                As for the “trans women” thing, I would (at some risk of getting yelled at, probably) suggest that (intersectionality aside) you get worse data by effectively masking off the difference between cis and trans. Again, that’d be a good place to have different questions (like on a dating site…what is your gender and did it match your sex at birth).

                We are not masking that. We have both stuff on record and can find out how many people are women + trans. This whole text is a brief summary and overview and both facts make sense in separation.

                In theory, yes, transwomen are women and so it shouldn’t matter. That said, in practice, the issues the two sets of people (ciswomen and transwomen) are somewhat different. If the goal is to increase diversity,

                The goal of the question is finding out the current diversity number, not to increase diversity. For that, both numbers make sense. We will pull up the numbers again when we want to document intersections between the sets.

                On all questions involving trans people, we usually ask trans people. The feedback to our method of collecting that data was very positive. On the other hand, asking for “transwomen”/“transmen” is currently a recipe for getting people annoyed.

                The same goes for the other fields. We had PoC involved in the phrasing of the PoC question, but it turns out the question ended up to be too much from a European/US standpoint. Which hurts me a little, as I’m usually the annoying voice in the Rust community that calls for thinking globally beyond the western part of the northern hemisphere.

            2. 4

              Diversity demographics are quite difficult to collect for a lot of reasons; having been tangentially involved in several efforts like this, it really requires close collaboration between data scientists and members of every group involved.

              This survey’s methodology is one of the more credible efforts I’ve seen. Among the key things it does well is that the questions being asked very likely mean the same thing to almost all respondents who pick them, and also to the experimenters. The more detail one tries to gather, the easier it is to lose that characteristic, and it takes a great deal of effort to get it back.

              This article doesn’t describe the financial resources available for the survey, but I have seen rather expensive data-gathering efforts with less meaningful results than these. I am always supportive of increasing the budget for this sort of study, but I see no reason to let the perfect be the enemy of the good.

              1. 5

                This article doesn’t describe the financial resources available for the survey, but I have seen rather expensive data-gathering efforts with less meaningful results than these.

                I can happily answer that one. Zero.

                Especially, the diversity question took a lot of consulting time.

                Also, because of all those fields intersecting rather randomly, I don’t see the proposed clustering here very useful.

                1. 1

                  Impressive, then! :)

                2. 1

                  Methodology or not, it’s still just not a very good graph–it conflates too many variables.

                  Also, as an example of the “means to the same thing to respondents” thing you mention, there’s part of the retrospective thread that touches on that issue.

                  1. 2

                    Yeah, it is very difficult to do properly. You honestly need a pre-study, or experience from previous rounds, to think of everything like that. That’s what I mean about a budget tradeoff; researchers are either donating their time or being paid, and…

                    It’s not immediately clear whether Koreans living in Korea are, by virtue of that status, members of a marginalized group within the Rust community. It would still have been nice if the question was clearer to them.

                    A hypothetical dream survey would also ask respondents about concrete effects of marginalization. It looks to me like it was a deliberate choice to frame this question in terms of “are you in this underrepresented demographic X?”, rather than “are you in demographic X?”. Whether it was thought about or not, those questions will get very different responses, and are useful for different purposes. I feel like the way it was asked is the more appropriate one for a community trying to learn to be “warm, welcoming, and inclusive”.

                    1. 2

                      Good points!

                3. 2

                  Yeah, we spent a bunch of time trying to figure out how to properly do it, and consulted with a lot of people. As an example, “What is your race?” was originally the phrase, but https://github.com/rust-community/team/issues/15#issuecomment-216649101

                  We have a thread over here: https://github.com/rust-community/team/issues/28 tracking how we want to improve for next year, I will make a comment linking to this thread.