Ugh… I hoped Apple would be more clever than this. I hoped they would go with a bunch of smaller specialised models for specific tasks. This is going to be a disaster. As much as people bash ChatGPT, Claude and other big LLMs this is going to be much worse.
Apple was generally OK with their models before. They have pretty good model in Photos to identify objects, people, and animals. They were finetuning very well on-device. Like, the model could identify cats of the same breed and even tell apart identical tweens. Their OCR model is great, too. Transformer-based autocomplete model is generally good, I had no noticeable issues with it and suggestions were often spot on.
I understand why they need an LLM for Siri but I don’t know why they decided to use the same general LLM for specific tasks. Everyone knew LLMs were very wonky since, I dunno… GPT-2. They knew that even the latest biggest models were hallucinating all the time. There’s just no excuse.
We already have screenshots of miscategorisation of phishig email. And the beta was out for like a week. We’re going to see much much more.
Siri wasn’t clever and its utility was limited but now it’s going to be actively misleading. Seemingly simple functions under Apple Intelligence umbrella (like summarisation) are going to be a source of so much misunderstanding.
While I’m happy for Apple with trying this out - I sincerely doubt that I’ll actually use it on my Mac. Your comments here totally summarize my feelings on the circumstance.
Based on beta, you should be able to disable it by not agreeing to “sharing audio recordings with Apple” which is a requirement. Only the compute part works on-device while Apple sucks all relevant data from the device anyway.
An open question raised by this is whether or not they are using LoRA / “adapter” models. Their paper here claims that they are: https://arxiv.org/pdf/2407.21075
While AFM-on-device is good at general summarization, we find it difficult to elicit summaries that strictly conform to the specification. Therefore, we fine tune a LoRA adapter on top of the quantized AFM-on-device for
summarization. The adapter is initialized from the accuracy-recovery adapter as described in Section 5.2. We use a data mixture consisting of input payloads covering Emails, Messages, and Notifications.
But these prompts make it look like they’re using prompt engineering techniques for things like summarization instead.
One possibility here is that they are doing both. I haven’t worked with LoRAs myself, but apparently you sometimes pair them up with a prompt - so it’s possible their LoRA adapters have been trained to provide particularly good results with the prompts we’re seeing here.
I was going to ask this and I’m not sure whether you just answered it… If I’m Apple and have money to train anything I want, it seems odd to train a general LLM, then further train it on general instruction following, then give it specific instructions that are baked into the mail app. Wouldn’t it be better to just directly train the base model on examples of mail summaries and skip the instruction following? Seems like that would avoid jailbreaks, as well.
I’m pretty certain that this is the instruction set for generating a “Memories” video with Apple Photos … the feature rejected my prompt asking for “images of sadness”
It looks like AI integrated in this fashion is going to be yet another way in which “your” devices aren’t really yours 🙄
FWIW I don’t think it works at all, not just in this instance. It won’t generate an album for “Christmas in Disneyland” but I have hundreds of photos of that.
Can and should are different words. Disabling SIP generally disables your access to support and in general you should not disable SIP unless you absolutely hard-line cannot do anything else but disable SIP.
The Apple documentation does state that for development purposes you can turn it of briefly, I presume, without consequences.
From a brief search I could not find that SIP checks whether the contents of files have changed, so maybe you could change the prompts and turn SIP back on.
Ugh… I hoped Apple would be more clever than this. I hoped they would go with a bunch of smaller specialised models for specific tasks. This is going to be a disaster. As much as people bash ChatGPT, Claude and other big LLMs this is going to be much worse.
Apple was generally OK with their models before. They have pretty good model in Photos to identify objects, people, and animals. They were finetuning very well on-device. Like, the model could identify cats of the same breed and even tell apart identical tweens. Their OCR model is great, too. Transformer-based autocomplete model is generally good, I had no noticeable issues with it and suggestions were often spot on.
I understand why they need an LLM for Siri but I don’t know why they decided to use the same general LLM for specific tasks. Everyone knew LLMs were very wonky since, I dunno… GPT-2. They knew that even the latest biggest models were hallucinating all the time. There’s just no excuse.
We already have screenshots of miscategorisation of phishig email. And the beta was out for like a week. We’re going to see much much more.
Siri wasn’t clever and its utility was limited but now it’s going to be actively misleading. Seemingly simple functions under Apple Intelligence umbrella (like summarisation) are going to be a source of so much misunderstanding.
While I’m happy for Apple with trying this out - I sincerely doubt that I’ll actually use it on my Mac. Your comments here totally summarize my feelings on the circumstance.
I hope the interface won’t be shoving It down your throat and maybe even allow you to disable it
Based on beta, you should be able to disable it by not agreeing to “sharing audio recordings with Apple” which is a requirement. Only the compute part works on-device while Apple sucks all relevant data from the device anyway.
“Do not hallucinate.”
Oh my sweet summer child.
An open question raised by this is whether or not they are using LoRA / “adapter” models. Their paper here claims that they are: https://arxiv.org/pdf/2407.21075
But these prompts make it look like they’re using prompt engineering techniques for things like summarization instead.
One possibility here is that they are doing both. I haven’t worked with LoRAs myself, but apparently you sometimes pair them up with a prompt - so it’s possible their LoRA adapters have been trained to provide particularly good results with the prompts we’re seeing here.
I was going to ask this and I’m not sure whether you just answered it… If I’m Apple and have money to train anything I want, it seems odd to train a general LLM, then further train it on general instruction following, then give it specific instructions that are baked into the mail app. Wouldn’t it be better to just directly train the base model on examples of mail summaries and skip the instruction following? Seems like that would avoid jailbreaks, as well.
I don’t have enough depth of knowledge to answer that - I’ve been asking the same question myself.
It looks like AI integrated in this fashion is going to be yet another way in which “your” devices aren’t really yours 🙄
FWIW I don’t think it works at all, not just in this instance. It won’t generate an album for “Christmas in Disneyland” but I have hundreds of photos of that.
I guess having the prompts stored as files does give the user control over them. You can change the prompts on your (non iOs) system.
You generally can’t. They’re locked by SIP.
Ah, I was a bit Linux naive. Of course you can’t edit all the files on your computer. Edit: it seems that you can turn SIP off
Can and should are different words. Disabling SIP generally disables your access to support and in general you should not disable SIP unless you absolutely hard-line cannot do anything else but disable SIP.
The Apple documentation does state that for development purposes you can turn it of briefly, I presume, without consequences.
From a brief search I could not find that SIP checks whether the contents of files have changed, so maybe you could change the prompts and turn SIP back on.