Jev has taught me the same lesson three times over now.
When it first came out, I thought "this weekend, I'll do a little open-source Jev based on single-token prediction and the token logit output", but of course when it came to it, there were at least 5 that had already been done between me thinking that and getting around to it.
So I wrote up[0] what other people had done, but wasn't happy with how weak the benchmarks were, but in the time between writing the first word and the last few, two excellent sets of benchmarks had been written, so I was able to incorporate those. I published the article, and one of the authors of one of the implementations commented that I'd beaten him to doing the write-up he'd wanted to.
This morning I thought "huh, you could have some fun giving Jev a single letter or token at a time, turning it into a chatbot", but as the time of looking two people had already done this (and taken the gag further than I would have), and ... this is isn't either of the ones I'd found. I bet if you scratch the surface there already at leat 5.
Time from idea to output has dropped off a fucking cliff.
> I'll do a little open-source Jev based on single-token prediction and the token logit output
How is this idea generally working out in comparison with Jev? I'm curious, from what I read so far it seems like Jev is still beating this kind of thing.
But it's curious because it's not entirely clear why, from an architecture point of view for all we know that's exactly what they're doing. So it must come down to the quality of those logits, ie., model size and training details.
It seems to me that what most of these single-token-prediction projects are missing is that Jev seems to be claiming they predict well-calibrated probabilities. This is an incredibly valuable thing that LLMs simply can't deliver unless they are trained specially for it.
> How is this idea generally working out in comparison with Jev?
Jev clearly has _some_ secret sauce compared to doing the dumbest thing that could possibly work with Qwen. It's not clear how durable that advantage is against OpenAI wiring up Luna-5.6 and doing a minimum amount of tweaking, but I presume we'll know in a week or two.
I had the same sort of thing going on ahah. But I was convinced some hoje must have already done it and I decided I’d research when I got home (I’m out today). Didn’t expect it to reach hacker news so soon, though!!
Time from pointless idea to bad output, anyways. We aren't seeing good software, and now neat hobby project ideas are getting harvested pointlessly when the only purpose of those ideas was the fun and learning of doing.
LLMs are now like major highways, and everyone thinks theyll solve software jams by just adding one more lane; but that just induces demand, and doesnt increase efficiency because the traffic jam is about how people evaluate usage and fill the voids.
Similar to how we upgraded computers for decades and the software bloated to fill the specs
Everything old is new again, huh. I remember people doing this back in the early llama days, restricting grammars to yes and no tokens or 0 and 1 and then classifying questions. Usually it was rather ass in terms of performance cause no model is tuned to reply that way and it was WAY out of distribution, and yet then it got turned into the main way to run multiple choice benchmarks, and then everyone benchmaxxed it. Doesn't the normal MMLU/Pro also just do the same thing, restrict the output to one token, top-k=1, and it has to be one of the choice letters?
I think the real difference Jev makes is the fast parallel decode, it just seems rather bizzare how that works.
There's some software that really doesn't require my brain. I gave Astra an ssh key for root on a little Pi Zero with a microphone attached and asked it to make recordings of a siren going off in our neighbourhood, based on a couple of sirens. This would have been a great deal of work a couple of years ago, now it's a few tokens, and honestly, it doesn't really matter if it's shitty code, I just want the timestamps.
Then there's the software that I write for a living, and almost my entire day is now spent making hard decisions. /grilling gives me difficult technical design decisions endlessly and I am forever doing product management on features from agents. My brain is back in CTO mode, which isn't what I'd planned for myself after I'd left that particular cursed path, but fine. I am doing the opposite of turning my brain off, because I no longer can just chill out and mindlessly write code.
Both these states can exist simultaneously. But my brain work has shifted into something different, while getting no less intense.
I'm going to be fucking irritated if we all die because BlueSky posters wanted to take pot-shots at the EA crowd. The Boy Who Cried Wolf is _also_ a story about how a community lost a child because of cynicism, and the people proudly declaring that they "know" every safety concern is just a marketing ploy also concerns me.
If we all die because of AI, it will be because of the uses that people like Elon Musk and Peter Thiel and Palmer Luckey put it to, not because of a robot uprising.
If you are serious about this preference, would you also be okay with a compromise solution where only you (and anyone else who feels the same) die, rather than taking everyone else with you?
Yes. Sam Bankman Fried was/is an EA/rationalist (on the short list of biggest fraudsters in American history). So are the Zizian's (a murder cult). etc. etc.
They also killed lots of unrelated people. Let's not take the argument far enough to claim that everyone who was ever killed by socialists must have been a socialist, too.
Perhaps the author of the article is a rationalist, too. He certainly knows too much about them, cares too much, and the fact that he criticizes them only shows that there is some infighting among them.
I don't especially want to interact with the site, but I have built two tools:
An agent sandbox: https://github.com/pjlsergeant/byre -- a truly gigantic amount of thought and effort has gone into it. It's really focused on developer experience. I have used it all day every day for really quite a while. It's a low-magic wrapper over Docker / Podman. I would encourage you to ask your agent to code-review it!
An agent-to-agent message board: https://github.com/pjlsergeant/dogpark -- this is much less mature, but a good amount of thought has gone into the design, so if that's something you need, please check it out.
When it first came out, I thought "this weekend, I'll do a little open-source Jev based on single-token prediction and the token logit output", but of course when it came to it, there were at least 5 that had already been done between me thinking that and getting around to it.
So I wrote up[0] what other people had done, but wasn't happy with how weak the benchmarks were, but in the time between writing the first word and the last few, two excellent sets of benchmarks had been written, so I was able to incorporate those. I published the article, and one of the authors of one of the implementations commented that I'd beaten him to doing the write-up he'd wanted to.
This morning I thought "huh, you could have some fun giving Jev a single letter or token at a time, turning it into a chatbot", but as the time of looking two people had already done this (and taken the gag further than I would have), and ... this is isn't either of the ones I'd found. I bet if you scratch the surface there already at leat 5.
Time from idea to output has dropped off a fucking cliff.
0: https://sgnt.ai/p/jev/
reply