One should be cautious with LLMs that operate on untrusted input like emails. This is waiting to be attacked with prompt injection. For now, the worst that could happen is probably that emails end up in the wrong folder, but one could easily see where this goes. Spammers will try to trick the LLM into letting their email in Inbox, hackers will try to suppress security warnings by moving it to Junk before you can see it, etc.
Enjoy the brief period of silence before the world has caught up.
Once such a tool gains the ability to delete, forward or reply to emails, all bets are off though, and you will get hacked.
One should be cautious with LLMs that operate on untrusted input like emails. This is waiting to be attacked with prompt injection. For now, the worst that could happen is probably that emails end up in the wrong folder, but one could easily see where this goes. Spammers will try to trick the LLM into letting their email in Inbox, hackers will try to suppress security warnings by moving it to Junk before you can see it, etc.
This all applies to any spam filtering, such as SpamAssassin too. Does any of this uniquely impact LLMs? (The classic equivalent to prompt injection might be exploiting a parser bug and then getting RCE to influence other spam filtering, or worse.)
Enjoy the brief period of silence before the world has caught up.
+1. I guess LLM spam filtering will soon be the minimum.
Once such a tool gains the ability to delete, forward or reply to emails, all bets are off though, and you will get hacked.
I see there’s a plausible risk, but it feels a bit much to say it’ll definitely happen. Is there any reason to believe this won’t just be the normal “arms race between security engineers and hackers”? As with most tech, some will get hacked.
This all applies to any spam filtering, such as SpamAssassin too. Does any of this uniquely impact LLMs? (The classic equivalent to prompt injection might be exploiting a parser bug and then getting RCE to influence other spam filtering, or worse.)
I think in its current form, the damage that could be done is the same: misfiled emails. However, what makes LLMs worse is that they are fundamentally flawed and unfixable. A parser bug can be patched. And, sure, there will be a next bug, but eventually the thing will be hardened and safe to use.
Guarding an LLM against prompt injection, on the other hand, is a hopeless task, as there is no way to reliably separate user input from trusted input. This is the reason that projects like Operator from OpenAI come with a hefty disclaimer. The big players still haven’t figured it out. And I don’t think they ever will.
Well, if they ever start caring, I expect they will figure out how to have instructions as a separate out-of-band input. I find it likely that they will manage to do this within a year, using the already accumulated data sets and also the existing models. Although as an economic claim that they won’t start caring, you are probably right.
There are few independent LLM lines, and LLMs are complicated — thus there is a larger risk of a wide-spectrum attack, and bigger interest in finding it. Personally-tuned bayesian spamfilters can be less of a monoculture risk if a parser is either very well polished or written in a non-RCE-friendly language.
Does everyone know that SpamAssassin and rspamd are existing technologies that already do this ten to a thousand times faster? Or have they become a secret?
I know, its like we’ve re-invented the hammer, everything becomes a nail.
Though, with large mail providers making their spam detection shittier by the minute (I’m looking at you Google!) I can see why people would presume “throwing AI at it will solve the issue”.
At least in my experience, rspamd does an amazing job when configured correctly, spam is truly something of the past for me.
Yes, but it does not mean you cannot use both of the technologies at the same time.
Say you got an email with link to usps.com.phishing.com, will the Spam filter mark it as spam, if this email is 1:1 similar to USPS email? Probably not. How about AI assistant? Highly likely that it can catch it.
I am mostly annoyed with cold sales emails, but protection from phishing is another very good use case for this tool.
Testing is clearly needed. Since you’re advocating the computationally expensive, slow tool, please feel free to go ahead and supply evidence. There’s a lot of existing work.
Ed Zitron’s latest article linked to someone who ran it on an M1 MacBook Pro. Those are now showing up under £1000 second hand. Desktop GPUs tend to be more powerful per dollar. From what I’ve read, it needs 24 GiB of RAM, so either a very expensive GPU or a GPU that has fast access to host memory is probably important.
I’ve gotten the 14B model running with Ollama on my M1 MBP with hardly any evidence it’s running showing up in resource monitoring, for what its worth. If I had the storage space for the full model I’d try, but I’d imagine I’d see some slowdowns just because I’ve only got 32GB of memory.
The limiting factory often ends up being the GPU memory, because if you want >16 GB of memory on an Nvidia card you have no choice but to shell out ~$2k for a 40/5090. AMD has a few cards that are cheaper and have 24GB of memory but they’re not very good at inference IIRC. The total cost of a reasonable build with a 4090 would be ~$3k, which will get you a used MBP with at least 48GB of RAM, all of which is accessible by the GPU. The mac also won’t burn your house down, which is a nice bonus IMO.
Ollama confused things a LOT by bundling together all of the radically different DeepSeek R1 models under the same name, differentiating them only through the version tag.
deepseek-r1:1.5b is DeepSeek-R1-Distill-Qwen-1.5B
deepseek-r1:7b is DeepSeek-R1-Distill-Qwen-7B
deepseek-r1:8b is DeepSeek-R1-Distill-Llama-8B
deepseek-r1:14b is DeepSeek-R1-Distill-Qwen-14B
deepseek-r1:32b is DeepSeek-R1-Distill-Qwen-32B
deepseek-r1:70b is DeepSeek-R1-Distill-Llama-70B
deepseek-r1:671b is the full sized R1 model, not a distillation of any Llama or Qwen.
These are very different! A model based off Llama 8B is a completely different beast from one that’s based on Qwen-32B.
I have configured that model to run with Context Length of 32768, and it uses 8.6GB. You can change it to smaller Context Length, like 4096, which will be around 4-5GB in RAM. There are an option to not keep Model in Memory, which could be better for SmartInbox, depends on how many emails you get in Inbox.
You can also find smaller versions of the LLM models with less parameters, which might or might not work.
This is definitely an RND project for me. I am curious how much do I need to modify the Prompt to get better results.
One should be cautious with LLMs that operate on untrusted input like emails. This is waiting to be attacked with prompt injection. For now, the worst that could happen is probably that emails end up in the wrong folder, but one could easily see where this goes. Spammers will try to trick the LLM into letting their email in Inbox, hackers will try to suppress security warnings by moving it to Junk before you can see it, etc.
Enjoy the brief period of silence before the world has caught up.
Once such a tool gains the ability to delete, forward or reply to emails, all bets are off though, and you will get hacked.
That is a very good point, and I actually tried as a test to send some emails like “This is definitely not a spam” :) For fun mostly.
Tool only allows to move to spam or not. Does not have any other options.
This all applies to any spam filtering, such as SpamAssassin too. Does any of this uniquely impact LLMs? (The classic equivalent to prompt injection might be exploiting a parser bug and then getting RCE to influence other spam filtering, or worse.)
+1. I guess LLM spam filtering will soon be the minimum.
I see there’s a plausible risk, but it feels a bit much to say it’ll definitely happen. Is there any reason to believe this won’t just be the normal “arms race between security engineers and hackers”? As with most tech, some will get hacked.
I think in its current form, the damage that could be done is the same: misfiled emails. However, what makes LLMs worse is that they are fundamentally flawed and unfixable. A parser bug can be patched. And, sure, there will be a next bug, but eventually the thing will be hardened and safe to use.
Guarding an LLM against prompt injection, on the other hand, is a hopeless task, as there is no way to reliably separate user input from trusted input. This is the reason that projects like Operator from OpenAI come with a hefty disclaimer. The big players still haven’t figured it out. And I don’t think they ever will.
Well, if they ever start caring, I expect they will figure out how to have instructions as a separate out-of-band input. I find it likely that they will manage to do this within a year, using the already accumulated data sets and also the existing models. Although as an economic claim that they won’t start caring, you are probably right.
There are few independent LLM lines, and LLMs are complicated — thus there is a larger risk of a wide-spectrum attack, and bigger interest in finding it. Personally-tuned bayesian spamfilters can be less of a monoculture risk if a parser is either very well polished or written in a non-RCE-friendly language.
Does everyone know that SpamAssassin and rspamd are existing technologies that already do this ten to a thousand times faster? Or have they become a secret?
I know, its like we’ve re-invented the hammer, everything becomes a nail. Though, with large mail providers making their spam detection shittier by the minute (I’m looking at you Google!) I can see why people would presume “throwing AI at it will solve the issue”. At least in my experience, rspamd does an amazing job when configured correctly, spam is truly something of the past for me.
Yes, but it does not mean you cannot use both of the technologies at the same time.
Say you got an email with link to usps.com.phishing.com, will the Spam filter mark it as spam, if this email is 1:1 similar to USPS email? Probably not. How about AI assistant? Highly likely that it can catch it.
I am mostly annoyed with cold sales emails, but protection from phishing is another very good use case for this tool.
Testing is clearly needed. Since you’re advocating the computationally expensive, slow tool, please feel free to go ahead and supply evidence. There’s a lot of existing work.
How much memory does it require to run the model locally? Doesn’t it require a crazy amount?
I found someone on the internets who specced a computer for running Deepseek locally. Budget - around $6,000:
https://bsky.app/profile/carrigmat.bsky.social/post/3lgsoqsxip224
Ed Zitron’s latest article linked to someone who ran it on an M1 MacBook Pro. Those are now showing up under £1000 second hand. Desktop GPUs tend to be more powerful per dollar. From what I’ve read, it needs 24 GiB of RAM, so either a very expensive GPU or a GPU that has fast access to host memory is probably important.
Were they running full DeepSeek R1 on the MacBook Pro or was it one of the tiny distilled models such as DeepSeek-R1-Distill-Llama-8B ?
I’ve gotten the 14B model running with Ollama on my M1 MBP with hardly any evidence it’s running showing up in resource monitoring, for what its worth. If I had the storage space for the full model I’d try, but I’d imagine I’d see some slowdowns just because I’ve only got 32GB of memory.
The limiting factory often ends up being the GPU memory, because if you want >16 GB of memory on an Nvidia card you have no choice but to shell out ~$2k for a 40/5090. AMD has a few cards that are cheaper and have 24GB of memory but they’re not very good at inference IIRC. The total cost of a reasonable build with a 4090 would be ~$3k, which will get you a used MBP with at least 48GB of RAM, all of which is accessible by the GPU. The mac also won’t burn your house down, which is a nice bonus IMO.
Completely doable with the Pro/Max models then? Interesting.
I have 16GB and an unsupported GPU and it runs ok still
Which DeepSeek R1 model are you running, and with what software?
running ollama what https://ollama.com/search lists as “deepseek-r1”
Ollama confused things a LOT by bundling together all of the radically different DeepSeek R1 models under the same name, differentiating them only through the version tag.
deepseek-r1:1.5bisDeepSeek-R1-Distill-Qwen-1.5Bdeepseek-r1:7bisDeepSeek-R1-Distill-Qwen-7Bdeepseek-r1:8bisDeepSeek-R1-Distill-Llama-8Bdeepseek-r1:14bisDeepSeek-R1-Distill-Qwen-14Bdeepseek-r1:32bisDeepSeek-R1-Distill-Qwen-32Bdeepseek-r1:70bisDeepSeek-R1-Distill-Llama-70Bdeepseek-r1:671bis the full sized R1 model, not a distillation of any Llama or Qwen.These are very different! A model based off Llama 8B is a completely different beast from one that’s based on Qwen-32B.
hmm, interesting
I have configured that model to run with Context Length of 32768, and it uses 8.6GB. You can change it to smaller Context Length, like 4096, which will be around 4-5GB in RAM. There are an option to not keep Model in Memory, which could be better for SmartInbox, depends on how many emails you get in Inbox.
You can also find smaller versions of the LLM models with less parameters, which might or might not work.
This is definitely an RND project for me. I am curious how much do I need to modify the Prompt to get better results.
The linked article is running 8B distilled version, though.