This was excellent. Really clear explanation, solid examples and the technique itself appears to be a powerful and novel way to influence what LLMs are doing.
If you’re interested in probing and steering LLMs the “Emergent world representations” paper is a very stimulating read: https://arxiv.org/abs/2210.13382
I had a tangential question if that’s allowed. Has anyone here been using these LLMs and if yes, how have they helped you?
I missed the chatgpt train because I wasn’t interested. Recently I found out about llamafiles which makes running these easier but the large variety of models and the unintuitive nomenclature dissuaded me. I still wanna try these out and looks like I have enough RAM to run the Mistral-7B.
I have played around with stable diffusion but the slowness due to weak specs and the prompt engineering aspect made me bounce.
I’ve been using LLMs on almost a daily basis for more than a year. I use them for a ton of stuff, but very rarely for generating text that I then copy out and use directly.
Code. I estimate 80% of my LLM usage relates to code in some way - in Python, JavaScript, Bash, SQL, jq, Rust, Go - even AppleScript, see https://til.simonwillison.net/gpt3/chatgpt-applescript - it’s like having a tutorial that can produce exactly the example you need for the problem you are working on, albeit with occasional mistakes
Brainstorming. This one surprised me, because everyone will tell you that LLMs can never come up with a new idea, they just spit out what they’ve been trained on. The trick with brainstorming is to ask for 20 ideas, and to prompt in a way that combines different things. “20 ideas for Datasette plugins relevant to investigative reporting” for example: https://chat.openai.com/share/99aeca01-62c7-4b7c-9878-7ce055738682 - you’ll rarely get an idea that you want to use directly, but some of them may well genuinely spark something interesting
World’s best thesaurus: you will NEVER be unable to find that word that’s on the tip of your tongue ever again.
Entertainment. There are so many weird dystopian games you can play with these things. I recently got ChatGPT to tell me “Incorporating the spirit of Fabergé eggs into your orchestrion project is a brilliant and deeply evocative concept” and I’ve been chuckling about that for days - it’s a consummate “yes man”, so one game is feeding it increasingly ludicrous ideas and watching it breathlessly praise them.
I do most of my work with GPT-4 because it’s still a sizable step ahead of other LLM tools. I love playing with the ones that run on my laptop but I rarely use them for actual work, since they are far more likely to make mistakes or hallucinate than GPT-4 through paid ChatGPT.
Mistral 7B is my current favourite local model - it’s very capable, and I even have a version of it that runs on my iPhone! https://llm.mlc.ai/#ios
Generating simple programs in languages I barely know, like C++ and javascript, which I can then kludge into something that does what I want.
“Here is a some data in this format, convert it to data in this other format.”
Generating regular expressions
Bulk searching: “here’s a list of ten libraries, give me the github or gitlab page for each of them.” or “Take this snippet of code, find all of the references for the NetworkX library, and link the docs page for each of them.”
Summarizing youtube videos
LLMs are good in cases where it’s hard to solve a problem, but easy to verify a solution.
You may try using the negative cfg scale with llama.cpp. Similar idea but different algorithm and implementation.
./main -m models/mistral-7b-instruct-v0.2.Q8_0.gguf \
-t 1 -ngl 99 -c 0 -n -1 --no-penalize-nl --repeat-penalty 1.0 --temp 0 --top-k 1 --log-disable \
-p '[INST] Write a story about an idol in a very creative, unpredictable, and insane style. [/INST] "Hello again," I said to' \
--cfg-negative-prompt '[INST] Write a story about an idol in a very uncreative, predictable, and normal style. [/INST] "Hello again," I said to' \
--cfg-scale 2
[INST] Write a story about an idol in a very creative, unpredictable, and insane style. [/INST] “Hello again,” I said to the mirror, my lips smeared with the crimson remnants of a thousand kisses. “My adoring fans, my insatiable muses, my shrieking sirens of desire! Gather ‘round, oh children of chaos, and let me spin a tale of madness so intoxicating, so unpredictable, your minds will dance on the razor’s edge of insanity!”
Once upon a twisted time, in a realm where reality was but a fleeting dream, there existed an idol of such otherworldly allure, such boundless madness, that the very fabric of existence trembled at the sound of her name: Lady Lunacy.
She was a vision of ethereal beauty, her form shifting like the sands of a dream, her eyes a swirling vortex of colors, her voice a symphony of shrieks and whispers that could lull the most hardened soul into a state of delirious ecstasy. But beware, for her charms were as unpredictable as the tides of the cosmic sea, and her insanity as boundless as the universe itself.
I guess it leaned too much toward the word “insane”. It generated a very long story and I only copied the beginning here.
This was excellent. Really clear explanation, solid examples and the technique itself appears to be a powerful and novel way to influence what LLMs are doing.
Yup, one of the few ML articles I’ve read that gives a clear intuition of how a technique works.
If you’re interested in probing and steering LLMs the “Emergent world representations” paper is a very stimulating read: https://arxiv.org/abs/2210.13382
I had a tangential question if that’s allowed. Has anyone here been using these LLMs and if yes, how have they helped you?
I missed the chatgpt train because I wasn’t interested. Recently I found out about llamafiles which makes running these easier but the large variety of models and the unintuitive nomenclature dissuaded me. I still wanna try these out and looks like I have enough RAM to run the Mistral-7B.
I have played around with stable diffusion but the slowness due to weak specs and the prompt engineering aspect made me bounce.
I’ve been using LLMs on almost a daily basis for more than a year. I use them for a ton of stuff, but very rarely for generating text that I then copy out and use directly.
I do most of my work with GPT-4 because it’s still a sizable step ahead of other LLM tools. I love playing with the ones that run on my laptop but I rarely use them for actual work, since they are far more likely to make mistakes or hallucinate than GPT-4 through paid ChatGPT.
Mistral 7B is my current favourite local model - it’s very capable, and I even have a version of it that runs on my iPhone! https://llm.mlc.ai/#ios
Some of my uses:
LLMs are good in cases where it’s hard to solve a problem, but easy to verify a solution.
Maybe I’m just not thinking of the right examples, but is verifying that usually much easier than doing the conversion?
IIRC the formats were
and
So pretty easy to verify (just match the numbers and tokens up) but I was not looking forward to writing all those {{}}s by hand
Also it was only one (very convoluted instance), so I couldn’t justify the time in writing a regex or parser.
I use them to generate weird recipes and memes. At work we pay for copilot so I’m not sure how good they are at writing code.
You may try using the negative cfg scale with llama.cpp. Similar idea but different algorithm and implementation.
I guess it leaned too much toward the word “insane”. It generated a very long story and I only copied the beginning here.