Why llamafile not llama.cpp?
Easier for people (including me) to install because there’s only one file to download - it’s basically a convenient wrapper around llama.cpp.
Also I only wrote about things I’ve got working myself, and I haven’t run Llama 3 with llama.cpp personally yet.
Useful comment on Mastodon from someone who got llama.cpp to work: https://social.brunk.io/@soeren/112315889818024916
git clone https://github.com/ggerganov/llama.cpp.git make huggingface-cli download MaziyarPanahi/Meta-Llama-3-70B-Instruct-GGUF --include '*Q5_K_M.gguf' --local-dir . ./server -m Meta-Llama-3-70B-Instruct.Q5_K_M.gguf -c 8192 -n 2048 llm -m llamafile "3 neat characteristics of a pelican"
Why llamafile not llama.cpp?
Easier for people (including me) to install because there’s only one file to download - it’s basically a convenient wrapper around llama.cpp.
Also I only wrote about things I’ve got working myself, and I haven’t run Llama 3 with llama.cpp personally yet.
Useful comment on Mastodon from someone who got llama.cpp to work: https://social.brunk.io/@soeren/112315889818024916