To me it feels unnecessary to contort the concept of privacy to say this feature is undesirable.
Even if you define privacy to include ownership of data, I don’t see how that is being violated. Because the data being transferred isn’t your data or metadata, it’s presumably some kind of hash. And you aren’t losing or giving away “ownership” of anything resembling your data. Hash implementations can be broken sometimes, but it sounds like you are explicitly not arguing that since “the question of technical perfection is mostly a red herring”.
Instead, we can argue the feature is undesirable from many other perspectives, like
“all new features have risk and should require consent”
or “this feature just isn’t very useful”,
or if the rumors are true, “Apple should not subject users to new features that are simply a way for some executive to save face from the failed CSAM scanning effort”,
or “Apple should not release features that further strengthens their walled garden, and iPhone-exclusive features for photos is a major part of their strategy”
Because the data being transferred isn’t your data or metadata, it’s presumably some kind of hash.
It is the data, not a hash. iOS is encrypting the image in a way Apple cannot decrypt, sending it to them, and they do image recognition using only the encrypted data (it’s homomorphic encryption aka ✨ magic maths ✨).
Hash implementations can be broken sometimes
Breaking a hash function means finding a collision, which doesn’t reveal any information about the input, thus isn’t relevant for privacy. It’s not inverting the function to find the input which is impossible because there’s information loss involved (otherwise it wouldn’t be a hash and there would be no possibility of collisions).
Ah yeah looks like it’s not a hash, but it’s not an encrypted version of the image either. It’s an encrypted embedding vector (aka a few thousand floating point numbers).
With PNNS, the client encrypts a vector embedding and sends the resulting ciphertext as a query to the server. The server performs HE computation to conduct a nearest neighbor search and sends the resulting encrypted values back to the requesting device, which decrypts to learn the nearest neighbor to its query embedding. Source
So it’s definitely not your data being sent or even an encrypted copy of your data. The only privacy argument IMO is that the implementation could be flawed, but OP seems to have backed away from that.
I didn’t have that much details, thanks for the source.
The embedding vector contains data derived from your picture which is at least enough to identify a landmark/location. So while it’s not strictly your whole data, it’s at least sensitive metadata, maybe data if the embedding is more like lossy compression than hashing.
I think it’s interesting worth pursing homomorphic encryption, differential privacy, and other schemes, but IMO those are still too new to be relied on in critical situations like one of the most used photos services ever.
Real consent will forever be key, so I would’ve made a prompt when you do the first landmark search in Photos. Granted most users wouldn’t understand the technical details fully, and I’m no exception, but something like this would be enough IMO: “Allow Photos to use a remote landmark database to fulfill this search? The contents of your photos will not be visible to Apple.” with a “learn more” option that leads to docs in laymans terms with links to dig even further.
It’s an encrypted embedding vector (aka a few thousand floating point numbers).
[…]
So it’s definitely not your data being sent or even an encrypted copy of your data.
It is definitely significant data derived from your data. How else the search could even take place? Without the homomorphic encryption this would be a blatant breach of secrecy, similar to a Google Image search. There’s also the question of timing: when searches are performed might itself contain cogent information (especially when paired with geolocation), so the user better understand when those happen.
The way in which Apple knows literally nothing about any of your information or queries? Seriously I fail to see a privacy problem here - even after lap cats original complete failure to understand what was happening - I think the reality is if you involve anything off device, even if it is entirely private, you should require consent, but I fail to see a privacy problem here.
One can consider it a slippery slope, if not an ouright privacy violation. This assumes Apple’s protections indeed work as they are supposed to, and iPhone users sign-up to the “trust us bro” mentality. Additionally, this functionality provides Apple a technical ability to determine the user’s location through Photos, even if the location services are turned off.
On the other hand, if Cupertino does not know the locations I have been to in the first place, the question of that data being abused in the future does not arise. I prefer everything on my phone to remain on my phone, unless an explicit consent is provided. Apple was sneaky about it, and this feature has flown under the radar, all red flags about the intent.
Additionally, this functionality provides Apple a technical ability to determine the user’s location through Photos, even if the location services are turned off.
“Your device privately matches places in your photos to a global index Apple maintains on our servers.” This most likely means the “global index” is downloaded to my iPhone to determine a match, and possibly some metadata will be sent back to Apple to improve the index matching accuracy. I don’t think this requires location services to be enabled.
No, they describe how it works, that’s the homomorphic encryption part, neither the location information you query nor the response information is visible to Apple.
The entire point of the design is that Apple cannot know either the query or the response. So let’s say there’s a hypothetical “the user said this response was X% accurate” they don’t have a mechanism to know what the query was or what the response was.
The only way to do that would be to independently (and I hope you can understand why this would be absurd) send the query, result, and accuracy back to Apple as unencrypted data. Thus completely defeating the entirety of the rest of the system.
I wonder if Enhanced Visual Search is related to the “private” cloud based CSAM scanning tech they had backed out of deploying.
It could be them repurposing the tech, or preparing for secret government required scanning.
To me it feels unnecessary to contort the concept of privacy to say this feature is undesirable.
Even if you define privacy to include ownership of data, I don’t see how that is being violated. Because the data being transferred isn’t your data or metadata, it’s presumably some kind of hash. And you aren’t losing or giving away “ownership” of anything resembling your data. Hash implementations can be broken sometimes, but it sounds like you are explicitly not arguing that since “the question of technical perfection is mostly a red herring”.
Instead, we can argue the feature is undesirable from many other perspectives, like
I agree with your larger point but:
It is the data, not a hash. iOS is encrypting the image in a way Apple cannot decrypt, sending it to them, and they do image recognition using only the encrypted data (it’s homomorphic encryption aka ✨ magic maths ✨).
Breaking a hash function means finding a collision, which doesn’t reveal any information about the input, thus isn’t relevant for privacy. It’s not inverting the function to find the input which is impossible because there’s information loss involved (otherwise it wouldn’t be a hash and there would be no possibility of collisions).
Ah yeah looks like it’s not a hash, but it’s not an encrypted version of the image either. It’s an encrypted embedding vector (aka a few thousand floating point numbers).
So it’s definitely not your data being sent or even an encrypted copy of your data. The only privacy argument IMO is that the implementation could be flawed, but OP seems to have backed away from that.
I didn’t have that much details, thanks for the source.
The embedding vector contains data derived from your picture which is at least enough to identify a landmark/location. So while it’s not strictly your whole data, it’s at least sensitive metadata, maybe data if the embedding is more like lossy compression than hashing.
I think it’s interesting worth pursing homomorphic encryption, differential privacy, and other schemes, but IMO those are still too new to be relied on in critical situations like one of the most used photos services ever.
Real consent will forever be key, so I would’ve made a prompt when you do the first landmark search in Photos. Granted most users wouldn’t understand the technical details fully, and I’m no exception, but something like this would be enough IMO: “Allow Photos to use a remote landmark database to fulfill this search? The contents of your photos will not be visible to Apple.” with a “learn more” option that leads to docs in laymans terms with links to dig even further.
It is definitely significant data derived from your data. How else the search could even take place? Without the homomorphic encryption this would be a blatant breach of secrecy, similar to a Google Image search. There’s also the question of timing: when searches are performed might itself contain cogent information (especially when paired with geolocation), so the user better understand when those happen.
I susbscribe to the author’s definition of privacy too, and thanks for informing about the (sneaky) way Apple is going about it.
The way in which Apple knows literally nothing about any of your information or queries? Seriously I fail to see a privacy problem here - even after lap cats original complete failure to understand what was happening - I think the reality is if you involve anything off device, even if it is entirely private, you should require consent, but I fail to see a privacy problem here.
One can consider it a slippery slope, if not an ouright privacy violation. This assumes Apple’s protections indeed work as they are supposed to, and iPhone users sign-up to the “trust us bro” mentality. Additionally, this functionality provides Apple a technical ability to determine the user’s location through Photos, even if the location services are turned off.
On the other hand, if Cupertino does not know the locations I have been to in the first place, the question of that data being abused in the future does not arise. I prefer everything on my phone to remain on my phone, unless an explicit consent is provided. Apple was sneaky about it, and this feature has flown under the radar, all red flags about the intent.
How?
“Your device privately matches places in your photos to a global index Apple maintains on our servers.” This most likely means the “global index” is downloaded to my iPhone to determine a match, and possibly some metadata will be sent back to Apple to improve the index matching accuracy. I don’t think this requires location services to be enabled.
No, they describe how it works, that’s the homomorphic encryption part, neither the location information you query nor the response information is visible to Apple.
The entire point of the design is that Apple cannot know either the query or the response. So let’s say there’s a hypothetical “the user said this response was X% accurate” they don’t have a mechanism to know what the query was or what the response was.
The only way to do that would be to independently (and I hope you can understand why this would be absurd) send the query, result, and accuracy back to Apple as unencrypted data. Thus completely defeating the entirety of the rest of the system.
I wonder if Enhanced Visual Search is related to the “private” cloud based CSAM scanning tech they had backed out of deploying.
It could be them repurposing the tech, or preparing for secret government required scanning.