TL;DR: AI generated bug reports are wasting this development team’s time.
This is the new spam. There was a SF magazine recently that gave live updates on how they had to deal with AI generated story spam. Reads who buy e-books off amazon note a rise in AI generated books that take a little bit (often post purchase) to discover.
We are going to settle in a world of trust based on reputation, but that too is flimsy, because writers have found AI generated texts trained on their works and published under their name (sourced from countries where IP infringement is not eagerly policed).
I don’t know that I like the world we are entering. It’s too easy to do evil and I have no idea of how to live in it, which makes me anxious.
Yes - this is the same feeling that email had before effective spam filters.
The cost to the sender (of sending 10k spam emails OR automating bug report spam) is small once they are set up.
The cost is spread over multiple recipients, they don’t particularly care if 99+% ignore/block them, the payoff for a small number of hits is more than the cost of sending. “I only have to have the 0.1% most gullible people respond”.
Maybe the same tools will work (I used the AI to destroy the AI), but maybe some of the rejected ideas from the antispam days are worthwhile? (micropayments - which these days might be represented by proof of work or similar). Given the failure rate, you don’t need to raise the cost of sending much for it to become marginal or uneconomical?
Your words reassure me. That’s true: I’ve blocked the pEnIs EnLaRgEmEnT email era from my memory but we did live through it. A part of me however wonders if email spam just dropped off because only a few dinosaurs like me still use email and the spam just shifted to text and twitter and instagram.
ChartGPT writes convincing nonsense (There is a mailing list post that I can’t find anymore by a senior member of the computing community that was written well, about this topic with startling examples). Soon video and audio fakes will be much cheaper to make. Then the stuff from China and Russia will really break us.
I can just see myself, in my declining years, giving away our house to a slick looking A.I. that comes visiting. The future is not getting brighter John. It wasn’t supposed to end like this.
A lot of modern spam is a work of art. I haven’t quite figured out what the one accusing me of war crimes (and cc’ing the secretary general of the UN) is trying to gain, but the one inviting me to join the Illuminati is perfect. Anyone whose estimation of their own intelligence leads them to believe that they deserve unsolicited invitations to join a shadowy cabal that runs the world, but whose actual intelligence isn’t sufficient to realise that such an invitation would probably not come from a random gmail account is exactly in the target market for scams.
I think the best I’ve gotten is an offer to purchase all of my cows. Apparently they’re considered top quality in Hong Kong. Complete with an offer to meet in person because “there are many queer character people in the cattle industry”.
A part of me however wonders if email spam just dropped off because only a few dinosaurs like me still use email and the spam just shifted to text and twitter and instagram.
It didn’t drop off. If your email address is publicly available, you’ll get that very same kind of spam even today. I already got several hundred spam emails of that very kind today, and it’s only 10am right now.
I got an AI generated paperback book as a Christmas present. The formerly reputable publisher had recently been bought out by a shadowy “management group”. A staff writer lists ChatGPT skills on his linkedin resume. It might have come from a brick&mortar bookstore for all I know. At some point, I might have to use 2022 as the cutoff publication date for deciding if I am going to read a book.
We’ve been getting spam phone calls from a bank who is using an eerily human-sounding robot to sell junk banking services. Sample conversation fragment: “Are you a robot?” “No of course I am not a robot!”. With Moore’s law, the long pauses will go away and it won’t be using the same stock phrases each time, so in the future I might be fooled. So our policy is we don’t accept unexpected calls from people we don’t have a personal relationship with. Assume other people are robots trying to scam you unless proven otherwise.
Once Elon Musk’s “neural lace” technology is deployed, it will inevitably be used to turn flesh and blood humans into robots. So I’ll need to use provenance and a web of trust to distinguish real humans from spambots.
Once Elon Musk’s “neural lace” technology is deployed,
This was an awful day to learn Musk is a fan of Iain M Banks’ novels. And yet, The Culture, which is completely post-scarcity, would have no place for a billionaire such as he.
I’ve been a big fan of 20th century science fiction, and its optimistic vision of advanced future technology. But I’ve been reflecting on the ways in which the future is not what these authors were hoping for.
Machines that pass the Turing Test were always supposed to be benevolent superintelligent AGIs that help humankind attain its potential, like the Culture Minds of Iain M Banks, or Asimov’s robots. The reality is disappointing, and makes the world feel like a more dangerous place to live.
The technological utopias predicted by 20th C scifi are largely made possible by an all-powerful top-down authoritarian power structure that benevolently keeps everything working smoothly and prevents bad outcomes. Iain Banks’ The Culture looks on the surface like an anarchist utopia, but in fact everyone’s enviable freedom and lifestyle is only made possible by the absolute benevolent dictatorship of the Culture Minds. The reality we live in is a lot messier than this. It’s self-interested humans all the way up the power structure.
We are going to settle in a world of trust based on reputation
This was always the world we lived in, just with more trust in some institutions than now. It always comes back to a network of trust – this is the ground truth for humans and always will be, I think.
but that too is flimsy, because writers have found AI generated texts trained on their works and published under their name (sourced from countries where IP infringement is not eagerly policed).
Indeed disturbing. When this (and deep fakes, etc) become prevalent enough I think some version (with a good UI) of private key signing will become standard. Currently a lot of people use twitter as a substitute for this. You just need some place you can go that a person owns to authenticate statements and works by that person. It’s not an impossible problem to solve. It’s just that most lay people aren’t familiar yet with the technology that solves it. I think the issue will be forced eventually and some solution will take over. I’d wish for it to be an open one but if history is any guide it won’t be.
It really feels like HackerOne should have a policy here and basically just ban idiots from their platform if they do this low-effort noise. That buffer overflow one is so bad that I would definitely suggest banning them entirely. Like, yeesh, it was so obviously AI gibberish but they didn’t disclose it so it’s hard to be sure.
The negative impact on someone for making this kind of shitty report needs to be serious - like, you are cut off from your source of revenue for a significant period of time. Not just a ‘reputation’ hit on H1.
This should not be “ban them reporting to me”, it should be “you have done something so fucking dumb that we are effectively firing you as a bounty hunter”.
It’s a big big bounty coordination service, but people’s view will depend very much on the project. They do open submissions for public projects, but they also can organise penetration testing by invitation only in a closed environment, with initial filtering/assessment done by their workers. So the experience with the platform is not homogeneous.
What’s the way out of drowning experts and general population alike in convincing crap? Doesn’t seem like traditional anti-spam would work, and I don’t know of a good anti-fake-news system in the large.
Will we switch to very tight allow lists based on a strict web of trust, and simply treat every post, piece of news and message coming from outside of it the same way we treat email spam (maybe checking the folder every once in a while)?
Not subsidising the cost of use of these tools. If the cost is high, as it is right now, and repercuted on users, as it is not rn, then spammers will use it less.
I’m afraid that’s not realistic, decent LLMs and image generators can already run on a laptop, and it’s only going to get better (or worse depending on how you view things) with the relentless pace of technology advancement.
Training is the costly part, but it only needs to happen once.
Not necessarily the tools, but the communication network. How about allowing recipients to charge for sending them emails, texts and phone calls. Naturally, everyone on my contact list is exempt, and if I really want to interact with you I’ll revert the fees.
TL;DR: AI generated bug reports are wasting this development team’s time.
This is the new spam. There was a SF magazine recently that gave live updates on how they had to deal with AI generated story spam. Reads who buy e-books off amazon note a rise in AI generated books that take a little bit (often post purchase) to discover.
We are going to settle in a world of trust based on reputation, but that too is flimsy, because writers have found AI generated texts trained on their works and published under their name (sourced from countries where IP infringement is not eagerly policed).
I don’t know that I like the world we are entering. It’s too easy to do evil and I have no idea of how to live in it, which makes me anxious.
Yes - this is the same feeling that email had before effective spam filters.
The cost to the sender (of sending 10k spam emails OR automating bug report spam) is small once they are set up.
The cost is spread over multiple recipients, they don’t particularly care if 99+% ignore/block them, the payoff for a small number of hits is more than the cost of sending. “I only have to have the 0.1% most gullible people respond”.
Maybe the same tools will work (I used the AI to destroy the AI), but maybe some of the rejected ideas from the antispam days are worthwhile? (micropayments - which these days might be represented by proof of work or similar). Given the failure rate, you don’t need to raise the cost of sending much for it to become marginal or uneconomical?
Or maybe I should just fill out the form
Your words reassure me. That’s true: I’ve blocked the
pEnIs EnLaRgEmEnTemail era from my memory but we did live through it. A part of me however wonders if email spam just dropped off because only a few dinosaurs like me still use email and the spam just shifted to text and twitter and instagram.ChartGPT writes convincing nonsense (There is a mailing list post that I can’t find anymore by a senior member of the computing community that was written well, about this topic with startling examples). Soon video and audio fakes will be much cheaper to make. Then the stuff from China and Russia will really break us.
I can just see myself, in my declining years, giving away our house to a slick looking A.I. that comes visiting. The future is not getting brighter John. It wasn’t supposed to end like this.
Update: Found it: https://cs.stanford.edu/~knuth/chatGPT20.txt It was extremely difficult for me to find it via a web search, which is annoying.
A lot of modern spam is a work of art. I haven’t quite figured out what the one accusing me of war crimes (and cc’ing the secretary general of the UN) is trying to gain, but the one inviting me to join the Illuminati is perfect. Anyone whose estimation of their own intelligence leads them to believe that they deserve unsolicited invitations to join a shadowy cabal that runs the world, but whose actual intelligence isn’t sufficient to realise that such an invitation would probably not come from a random gmail account is exactly in the target market for scams.
I think the best I’ve gotten is an offer to purchase all of my cows. Apparently they’re considered top quality in Hong Kong. Complete with an offer to meet in person because “there are many queer character people in the cattle industry”.
It was good enough that I kept it in my inbox.
It didn’t drop off. If your email address is publicly available, you’ll get that very same kind of spam even today. I already got several hundred spam emails of that very kind today, and it’s only 10am right now.
I got an AI generated paperback book as a Christmas present. The formerly reputable publisher had recently been bought out by a shadowy “management group”. A staff writer lists ChatGPT skills on his linkedin resume. It might have come from a brick&mortar bookstore for all I know. At some point, I might have to use 2022 as the cutoff publication date for deciding if I am going to read a book.
We’ve been getting spam phone calls from a bank who is using an eerily human-sounding robot to sell junk banking services. Sample conversation fragment: “Are you a robot?” “No of course I am not a robot!”. With Moore’s law, the long pauses will go away and it won’t be using the same stock phrases each time, so in the future I might be fooled. So our policy is we don’t accept unexpected calls from people we don’t have a personal relationship with. Assume other people are robots trying to scam you unless proven otherwise.
Once Elon Musk’s “neural lace” technology is deployed, it will inevitably be used to turn flesh and blood humans into robots. So I’ll need to use provenance and a web of trust to distinguish real humans from spambots.
It’s the new spam.
This was an awful day to learn Musk is a fan of Iain M Banks’ novels. And yet, The Culture, which is completely post-scarcity, would have no place for a billionaire such as he.
I’ve been a big fan of 20th century science fiction, and its optimistic vision of advanced future technology. But I’ve been reflecting on the ways in which the future is not what these authors were hoping for.
Machines that pass the Turing Test were always supposed to be benevolent superintelligent AGIs that help humankind attain its potential, like the Culture Minds of Iain M Banks, or Asimov’s robots. The reality is disappointing, and makes the world feel like a more dangerous place to live.
The technological utopias predicted by 20th C scifi are largely made possible by an all-powerful top-down authoritarian power structure that benevolently keeps everything working smoothly and prevents bad outcomes. Iain Banks’ The Culture looks on the surface like an anarchist utopia, but in fact everyone’s enviable freedom and lifestyle is only made possible by the absolute benevolent dictatorship of the Culture Minds. The reality we live in is a lot messier than this. It’s self-interested humans all the way up the power structure.
Actually. Passing the Turing Test is definitely not what define these in this literature. Hell it is not even that in Turing’s own writings.
We as readers chose to interpret them as such.
This was always the world we lived in, just with more trust in some institutions than now. It always comes back to a network of trust – this is the ground truth for humans and always will be, I think.
Indeed disturbing. When this (and deep fakes, etc) become prevalent enough I think some version (with a good UI) of private key signing will become standard. Currently a lot of people use twitter as a substitute for this. You just need some place you can go that a person owns to authenticate statements and works by that person. It’s not an impossible problem to solve. It’s just that most lay people aren’t familiar yet with the technology that solves it. I think the issue will be forced eventually and some solution will take over. I’d wish for it to be an open one but if history is any guide it won’t be.
It really feels like HackerOne should have a policy here and basically just ban idiots from their platform if they do this low-effort noise. That buffer overflow one is so bad that I would definitely suggest banning them entirely. Like, yeesh, it was so obviously AI gibberish but they didn’t disclose it so it’s hard to be sure.
The negative impact on someone for making this kind of shitty report needs to be serious - like, you are cut off from your source of revenue for a significant period of time. Not just a ‘reputation’ hit on H1.
This should not be “ban them reporting to me”, it should be “you have done something so fucking dumb that we are effectively firing you as a bounty hunter”.
What kind of setup is HackerOne, anyway? I’ve never heard of it before reading this blog post.
HackerOne is “bug bounty as a service”. It’s been around for a long time and has a pretty wide base of projects using it.
It’s a big big bounty coordination service, but people’s view will depend very much on the project. They do open submissions for public projects, but they also can organise penetration testing by invitation only in a closed environment, with initial filtering/assessment done by their workers. So the experience with the platform is not homogeneous.
What’s the way out of drowning experts and general population alike in convincing crap? Doesn’t seem like traditional anti-spam would work, and I don’t know of a good anti-fake-news system in the large.
Will we switch to very tight allow lists based on a strict web of trust, and simply treat every post, piece of news and message coming from outside of it the same way we treat email spam (maybe checking the folder every once in a while)?
Not subsidising the cost of use of these tools. If the cost is high, as it is right now, and repercuted on users, as it is not rn, then spammers will use it less.
I’m afraid that’s not realistic, decent LLMs and image generators can already run on a laptop, and it’s only going to get better (or worse depending on how you view things) with the relentless pace of technology advancement.
Training is the costly part, but it only needs to happen once.
Not necessarily the tools, but the communication network. How about allowing recipients to charge for sending them emails, texts and phone calls. Naturally, everyone on my contact list is exempt, and if I really want to interact with you I’ll revert the fees.
[Comment removed by moderator pushcx: Snarky asides about economics and politics rarely prompt good threads.]
Either a duplicate of, or should be merged into: https://lobste.rs/s/3f69tm/i_llm_stands_for_intelligence