You are a maintainer of a software that depends on ollama, so you should know th... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

jxy on July 23, 2024 | parent | context | favorite | on: Llama 3.1

You are a maintainer of a software that depends on ollama, so you should know that ollama depends on llama.cpp. And as of now, llama.cpp doesn't support the new ROPE: https://github.com/ggerganov/llama.cpp/issues/8650, and all ollama can do is wait for llama.cpp: https://github.com/ollama/ollama/issues/5881

ajhai on July 23, 2024 [–]

I've tested Q4 on M1 and it works though the quality may not likely be the same as you'd expect as others have pointed out on the issue.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact