1. 21

  2. 12

    This is interesting but the metrics seem arbitrary.

    Bugs is actually the issue count. If a team is using the issue tracker to coordinate future work, it isn’t a true measure of bugs. Teams that don’t submit issues or don’t test will have lower bug counts.

    Shouldn’t you take the size of the commit into consideration? Lots of small commits can make the metric drop, and commit frequency has more to do with the group and the developer than the language.

    It might be that certain groups or problem domains have a higher or lower tolerance for issues or value testing differently.

    I welcome empirical research in programming languages and development and understand it is really hard. Keep it up!

    1. 8

      I counted only bugs, not all issues, and filtered out any repo that had no bugs (i.e. are using another tracker).

      You’re right about commit size, I worked under the assumption that while different teams might have different acceptable commit sizes, that would average out across many teams. I don’t really think any given language would have a tendency to different commit sizes, but that would be a great thing to check for the next one!

      The code is here if you are curious: https://github.com/steveshogren/github-data/blob/master/src/github_data/core.clj

    2. 2

      I’d love to see TypeScript and Flow (but mostly TypeScript) included in future analyses, since a great deal of the pitch for those dialects of JS is the error checking provided by the compiler.