Even easier. You follow these steps:
Make one that learns to adapt language to that of high scoring players that speak the most (more training data).
Put it on Xbox Live playing Modern Warfare.
Wait for a few months.
After a few months, it will be both racist and dissing everyone’s mom. Just had to play the imitation game on the noisiest players.
Whereas, countering it would be a bit more difficult. You’d have to pre-train it with an adaption ability. You’d want a delay before the new inputs really count or a roll-back to previous state that’s good. Then, a detection mechanism for trolling or hate speech to tell you when to mute players, Then, maybe one to look back at when they first started talking. Then, delete or roll back to before that point. Might need this for in-game play, too, in case a group tried to do collectively stupid things to a bot to make it learn stupidity for future games.
This is basically what happened with Microsoft’s Tay.AI, except with Twitter instead of XBL.
Very nice writeup.
Other than the little toy experiment, I find extremely valuable the “call to action” about knowing what your data is and what you should expect.
It is kind of weird that people just assume ML to be magic and that it cannot be understood.
In another star system, some programmer created a calculator. However, upon inputting 1+3 the program spits out 4. This was considered wrong because it is oddist and the program was altered to output 5 instead.
Alas that was already too late and the programmer was burned at the stake.
I understand the point you are making but there’s a difference between an algorithm picking up on correlations we’d rather not think about and an algorithm simply being fed bad input data.
The problem with Tay by Microsoft for example is that its input data was an organised attempt by trolls to feed it as much nonsense as possible. If it just turned out to be rude, that might just be it picking up on what Twitter is like. But it wasn’t.