Seems that you could add a minimum padding + random padding and defeat this, at the cost of larger requests. But the bottleneck for users is the LLM, not the transmission, so that seems like not a huge deal.
edit: Shoulda kept reading.
Of all the chatbots that were vulnerable to the attack, those from OpenAI and Cloudflare have implemented padding mitigations in the past 48 hours.
This + potentially buffering output slightly, especially as LLM performance improves, seems like it will make the issue impractical. Reminds me of CRIME though.
The “interactive” chat bots return tokens immediately, so you can measure the amount and length of token responses.
That’s all information that you can use to build a statistical model of the conversation, and apparently LLM output is so presictable that that is what they were able to do.
From a cryptographic standpoint this is not a new attack, this is again a failure of non-cryptographers trying to use cryptography but not understanding that using cryptography correctly means that you can’t have any bias in how and what you transmit. This time at least it’s more subtle than usual, but if they’d asked anyone who works on other canonical encrypted “conversation” applications (eg messaging apps) they would have been told they need to watch the message lengths.
But the point is that these days people are on shared networks sufficiently frequently that that’s relatively achievable even for non-government actors (wifi, etc). For governments you can just get a warrant on the target’s network traffic, and you achieve the same thing.
In general relying on “it’s hard to sniff the target’s packets” isn’t a reasonable position to take, obviously it fails the basic cryptographic threat model (e.g. the attacker has full access to all the encrypted data) but in practical sense it’s also just not a high bar anymore.
It might be harder with actual private VPNs, but I wouldn’t bet on it (i.e something like iCloud private relay, neither individual party can connect everything together, or someone who compromises one of them, but you could imagine a government agency coercing every involved party to provide all information, which would allow them to link everything together - again the issue here is “what is your threat model?”).
Seems that you could add a minimum padding + random padding and defeat this, at the cost of larger requests. But the bottleneck for users is the LLM, not the transmission, so that seems like not a huge deal.
edit: Shoulda kept reading.
This + potentially buffering output slightly, especially as LLM performance improves, seems like it will make the issue impractical. Reminds me of CRIME though.
Oh interesting but also kind of not?
The “interactive” chat bots return tokens immediately, so you can measure the amount and length of token responses.
That’s all information that you can use to build a statistical model of the conversation, and apparently LLM output is so presictable that that is what they were able to do.
From a cryptographic standpoint this is not a new attack, this is again a failure of non-cryptographers trying to use cryptography but not understanding that using cryptography correctly means that you can’t have any bias in how and what you transmit. This time at least it’s more subtle than usual, but if they’d asked anyone who works on other canonical encrypted “conversation” applications (eg messaging apps) they would have been told they need to watch the message lengths.
it also relies on being able to sniff the target’s packets.
But the point is that these days people are on shared networks sufficiently frequently that that’s relatively achievable even for non-government actors (wifi, etc). For governments you can just get a warrant on the target’s network traffic, and you achieve the same thing.
In general relying on “it’s hard to sniff the target’s packets” isn’t a reasonable position to take, obviously it fails the basic cryptographic threat model (e.g. the attacker has full access to all the encrypted data) but in practical sense it’s also just not a high bar anymore.
It might be harder with actual private VPNs, but I wouldn’t bet on it (i.e something like iCloud private relay, neither individual party can connect everything together, or someone who compromises one of them, but you could imagine a government agency coercing every involved party to provide all information, which would allow them to link everything together - again the issue here is “what is your threat model?”).