Try 3.8 27B in MTPLX; I get about 30 tok/s with the same hardware as you. (Although it does use around 90-95W of power, compared to the ~60W that 3.6 35B-A3B uses to generate 55 tok/s. That’s about 3 J/tok instead of 1.)
NGL: I don’t really have a good way to find out right now. It also doesn’t matter that much because the way the models use the tokes varies a lot. Qwen 3.8 is known for overthinking while Muse Glimmer may be a little slower per token, but it uses them very efficiently, caveman style.
Generation speed isn’t the bottleneck anyway, at least on pre M4/M5 devices (the newer chips got significant processing acceleration). It’s prompt processing time. OpenCode’s system prompt can take up to 3 minutes to process, which is why good prompt caching is essential.
For that I use omlx, which can persist the KV cache to disk, chunked so you can reuse parts. This helps with the usability a lot, when an agentic session is warm it runs pretty smoothly. New requests can take a couple seconds (sometimes many, which must be fixable somehow).
So: It’s not fast, but I also don’t find it awfully slow. My use is typically semi-interactive, for fully interactive use you have to wait a bit, but it’s possible. I personally am still regularly amazed that something even close to this is possible on completely local hardware.
>Private citizen fund scientific research [under threat of prison or deadly force.]
I mean I'm not inherently opposed to laws or government, but I think a lot of people need to be more measured and considerate of what they are using tax money for when it is being taken from their fellow citizens at gunpoint.
ignoring a gratuitous reference to use of force, absolutely. having a discussion as society about what we are funding is in fact democracy. sadly there is an issue in the sciences in that lay people may have difficulty seeing the benefit of connecting the dots. but we should try. and as flawed as it is, the adversarial system we have in the US is at least a forum for those discussion.
using grep to defund grants that contain words we don't like is the exact opposite of measured and considerate. so is punishing scientists for the sin of working for a 'woke' institution. in fact all this seems extremely punitive, and not in the spirit of optimizing outcomes for costs at all.
note that this policy explicitly removes the requirement to provide any kind of rationale. that sort of directly contradicts the notion that this is a measured discussion about priorities.
I agree. This administration is ground zero for mismanagement of funds and outright corruption. Just look at the director of the FBI and former secretary of DHS. Both have used and continue to use tax payer money for personal use. It should make every tax payer livid.
It does make me livid, just as much as the waste of taxpayer money on pointless (and sometimes outright racist) research here makes me livid: https://news.ycombinator.com/item?id=48335722
Believe it or not, it's possible to hate both Kash Patel/Kristi Noem and the unelected bureaucrats burning tax money on awful research.
Sure that’s fine. They funded research that you think is wasteful and you would like to have tools to provide that feedback. The issue with this OMB change is that it is not that. The OMB change is a change that allows the administration to cancel any grant for any reason without answering to citizens about it.
BTW your “believe it or not” is quite condescending. Do you talk to people in real life like that?
So you want every decision in government to be made by a politician instead of an expert? We should start with the military right? Get those bureaucrats generals out of the battlefield and put some elected officials in charge of our battle plans so our plans aren’t wasting tax dollars!
Again, the status quo is that funding recommendations are made by expert peer review by PhD scientists. Political appointees literally do not have the knowledge to make the calls this policy directs them to.
That’s even worse, the incentives are completely misaligned for a political appointee vs unelected civil service workers who are just carrying out their jobs.
The paper linked in that post proposes a bottleneck at 900Kya in the ancestors of all modern humans. There is a bottleneck associated with the migration out of Africa and the peopling of the world that many populations have, but not all. Based on genetic data the timing is between 100-50Kya, with a lot of the uncertainty coming from converting generation times to years (i.e. how many years on average between parents and offspring). This is a nice reference: https://sci-hub.se/https://www.nature.com/articles/nature213...
Hello, I submitted this. Thanks for your concern. I've been lurking on Hacker News since ~2014 when I was in college. I made an account and started posting because people on the board have something of an interest in biology and genetics (for example this story on cancer detection from yesterday: https://news.ycombinator.com/item?id=43035147 ) and I wanted to contribute. I posted this because the NIH funds a lot of research related to that and this story reports on a massive change to the NIH, which seemed worthy of discussion.
"Having 50% of their grant leeched off" is not how indirects work.
You know what's a much bigger problem for people doing research? That the budget for a non-modular R01 from the NIH hasn't changed since the Clinton administration.
When the government hires a consultant, the rate charged is usually 2-3x the salary of that consultant; 50% is well below the 100-200+% rates you see all the time in business.
This is a huge cut. Back of the envelope calculations for UCLA (a big research university with public finance information): $200M cut in operating revenue.
In 2023, UCLA had $270M in indirect costs [1] and they negotiated a rate of 57% with the NIH [2]. So, they had about $473M in direct costs. The new rate would be 15%, which is ~$71M. $270M-$71M = $200M.
This is a huge cut. Back of the envelope calculations for UCLA (a big research university with public finance information): $200M cut in operating revenue.
In 2023, UCLA had $270M in indirect costs [1] and they negotiated a rate of 57% with the NIH [2]. So, they had about $473M in direct costs. The new rate would be 15%, which is ~$71M. $270M-$71M = $200M.
