1. 29
  1.  

  2. 29

    “AI” proctoring software has had this as a failure mode for a long time, but it’s become especially pressing over the past year or so as so many things went remote.

    I recall a story about some Black students taking their bar exams who kept getting flagged for leaving during the exam because of failure of facial recognition, and some were literally told to go out and buy the brightest high-beam lights they could get and point them directly into their own faces – all through the exam – in hopes of producing a light enough tone that the facial recognition would work.

    And this is just the latest in a long long line of problems, including infamous examples like automated restroom soap dispensers that don’t register dark skin, cameras that go into “did someone blink?” mode when taking photographs of Asian people, etc. The fact that it’s been developed and tested by monoculture or nearly-monoculture teams is often painfully obvious.

    1. 21

      It’s worth noting that the “did someone blink” case happened on Nikon cameras, which is a Japanese company. The fact that their own cameras failed against Asian faces points to a biased dataset, not necessarily to a homogeneous team.

      1. 10

        Or that Nikon is really bad at developing software, which is born out by other examples.

        1. 8

          I’ve had a very dark cat for the last half year, not quite black but close enough, and it’s surprisingly hard to take a decent picture of her with my phone camera, even in good light conditions. And with less-than-good light conditions it’s just impossible. This isn’t the best camera but those facial recognition systems probably aren’t either.

          It seems to me that these kind of things are just really hard to develop in the first place; I don’t think it’s necessarily indicative of any sort of bias (although it could be). The real problem is that the current zeitgeist is to push through with automation no matter what the costs or trade-offs might be, which is a problem that extends far beyond this sort of stuff. Th classic “waving your loved one goodbye from the train platform”-scene is now impossible in many countries as you need a ticket to enter the train platform, and those machines don’t have an “I just want to wave goodbye” option.

          As a society we seem to be like a kid with a new toy that wants to do nothing other than play with this new toy and we view this sort of (alleged) “progress” as a force of nature we have no control over.

    2. 9

      The whole blog is dedicated to bashing Proctorio, how weird

      1. 17

        Eh. It claims to be about “exam spyware analysis” and bashes the similarly named ProctorTrack too. I don’t find it too weird; if I were forced to use such software, I would probably inspect it and might go so far as to write a blog complaining about it, if reporting the problems I saw in other ways brought no joy.

        1. 9

          I am forced to use Moodle when I teach and I have definitely considered launching a novelty Twitter account just to bash the software. Not so much because I think it would matter, but because doing so might be cathartic… :-)

          1. 4

            And Moodle while clumsy to death is one of the less bad of the bunch by my small experience.

          2. 2

            How’s that weird?

            1. 4

              He (or she) created a blog just for bashing a single company. Even the domain is “proctor.ninja”. Maybe they are an employee, maybe they thinks Proctorio is the worst evil, and maybe it is, but they definetly has a grudge.

              1. 10

                Proctor io has a history of suing experts who critique their shady practices. I would probably attempt to remain incognito if I was making these claims as well

            2. 1

              There was an entire blog dedicated to how xkcd sucks. It seems like there are … several now. The original (with the hyphen) was hilarious.

              There’s also an entire mastodon instance dedicated to SalesForce fandom .. or at least it seems at first. It’s difficult to tell if they’re really fans, making fun of it ironically or a little of both.

              1. 1

                Wait until you find out about https://twitter.com/memenetes. They even sell merch about bashing Kubernetes! XD

              2. 7

                I had to do some digging through the linked tweets to figure out what Proctorio was. If the author is reading, it might be worth giving a brief summary of the technology you’re criticizing for those unfamiliar with it.

                1. 1

                  Yup, it does seem to be (un)popular in a specific country or a region.

                2. 3

                  If you want to independently verify the claims made by the researcher, I made a PHP script for that: https://gist.github.com/soatok/5aca9a99d800916f2dda549f4d319a31 – Extension is available here.

                  I used the headline of the article in my submission here, but the more interesting thing to me is that they’re getting away with flagrantly violating the Google Chrome Store’s policies about code obfuscation.