I was really waiting for somebody to say “Holy shit, it’s a Thompson hack in the wild.”
The real problem with the story is that this sort of thing could maybe evolve, but it couldn’t really hide itself, because that requires a relatively smooth probabilistic guidance vector. Evolution works in the wild because you have a hundred thousand trials across a hundred thousand instances of interaction. But in our society, one person can find a fix and spread it to millions of people. Human cultural evolution is genuinely better in kind than biological evolution; an evolved digital virus would have to win every trial every time on the first try. And you can say “but it didn’t work” - and I’ll say “it should never even have come this far”. Forget one person noticing; given how heterogenous our systems are, the virus would have to be an AGI on its own.
Of course, given how magical this thing is, the next time it crops up I wouldn’t expect my eyes to work on it.
This is a work of fiction but I think it’s still very on topic for lobsters. Enjoy!
I was really waiting for somebody to say “Holy shit, it’s a Thompson hack in the wild.”
The real problem with the story is that this sort of thing could maybe evolve, but it couldn’t really hide itself, because that requires a relatively smooth probabilistic guidance vector. Evolution works in the wild because you have a hundred thousand trials across a hundred thousand instances of interaction. But in our society, one person can find a fix and spread it to millions of people. Human cultural evolution is genuinely better in kind than biological evolution; an evolved digital virus would have to win every trial every time on the first try. And you can say “but it didn’t work” - and I’ll say “it should never even have come this far”. Forget one person noticing; given how heterogenous our systems are, the virus would have to be an AGI on its own.
Of course, given how magical this thing is, the next time it crops up I wouldn’t expect my eyes to work on it.