They *also* need to tell people what quantisation they're using. Because some 4 ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

girvo 15 days ago | parent | context | favorite | on: GLM-5.3-Flash

They also need to tell people what quantisation they're using. Because some 4 bit version is not the same as BF16, no matter what KLD suggests.

skohan 15 days ago [–]

I'm using the unsloth dynamic Q4 and getting good results. I was running Q5, but Q4 gives more context headroom so I can run two agents in parallel with ~100k context each with 32GB vram.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact