Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
trucks-refinish
13 days ago
|
parent
|
context
|
favorite
| on:
Small Models Have Arrived
I'm able to run qwen 3.6 35b a3b with my rtx 2060 super which also has 8gb of vram. It's a very capable model, not as smart as a frontier model but it is still very useful.
I use an 6bit quant and get around 20 tokens per second.
help
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
I use an 6bit quant and get around 20 tokens per second.