I had Muse Glimmer (from Meta / Facebook) quoting OpenAI's safety guidelines to me, and I had Poolside's Laguna (a smaller US company) with thinking traces about obeying Chinese law.
Both of those are local models, and I didn't provide them tools to access the internet to call other models. None of this is proof of anything, but it is suggestive.
> A common issue is that it's rarely mentioned on which dataset KL-divergence is computed. It seems the most common dataset is wikitext
Thank you for calling this out. Using Wikipedia snippets for these is a terrible choice. I did a bunch of KL and other stats with the five Gemma 4 models, and the results were non-obvious. Anthropomorphizing:
Gemma 4 31B: "I guess we'll pretend I said this, but it's not me." (Baseline for stats)
Gemma 4 26B: "Dude, I'm certain I wouldn't have said this." (Bad KL)
Gemma 4 12B: "Umm, Me either!" (Similarly Bad KL)
Gemma 4 E4B: "I might say almost anything, this is fine." (Much better KL!!!)
Gemma 4 E2B: "I'm basically a toy. Let's play a game!" (Same KL as E4B)
Anyway, for comparing quantizations, it seems like the largest precision version should be given a one-shot prompt, and the result from that should be used as the corpus for the quantized versions.
> Anyway, for comparing quantizations, it seems like the largest precision version should be given a one-shot prompt, and the result from that should be used as the corpus for the quantized versions.
Way better than wikitext-- but tells you nothing about errors tending to compound or cancel out.
Like say a test shows that only one token in a 10,000 token test would be different. Sounds very close, ship it!-- but what if trajectories with that single different token guarantees failure because it sets in motion a cascade of differences that ultimately result in a final distribution that doesn't include the solution?
Maybe I'm reading too much between the lines, but I suspect the reason is to rub his nose in the duplicity or naivety depending on how generous you're feeling. Publishing the model would be a confession that he was wrong.
AI policy is being shaped somewhat by the things Sam and Dario say. So even if you're not feeling vindictive, it's probably good to keep a track record of the previous things they have said as a Bayesian prior. People who don't know better listen to these people, and maybe they shouldn't.
If you want to dunk on sam for "it's too dangerous to publish GPT-3", you hardly need the release of gpt-3 to prove your point. All the other open model releases already provide pretty good evidence. Moreover the fact that the model wasn't release hardly points to the fact that he wanted to save face or whatever. Most AI labs don't release their old proprietary models, so the fact that gpt-3 wasn't released tells us very little.
There's a link you can click to see the personal background of the respondents. It's tough to know what "researcher in a field not listed" means, but it's possible that over 60% are not even physicists:
30.8% - A researcher in a field not listed above
21.2% - A science enthusiast
18.0% - A researcher studying quantum physics
12.0% - A researcher studying astrophysics or cosmology
9.2% - A researcher studying gravity
8.8% - Other
This seems very cool, but I'm not sure I understand exactly what it's doing. Are they making a new speculative drafter for Qwen 3.8 27B? Maybe they're optimizing the MLX code for the decoder itself? Thank you in advance.
I've thought about turning this upside down. To any person who is sure they know what has and doesn't have consciousness: If I say I don't have it, can you prove me wrong?
As far as I'm concerned, it's a word without a useful enough definition to bother worrying about it.
It's subjective experience. The colors you see, the sounds you hear, tastes, smell, pain, emotion, dreams, inner dialog. The philosophical word would be qualia.
I don't understand people who pretend to be p-zombies in these arguments.
I'm not aware of anyone lacking all those experiences who isn't brain dead. What does approximating an experience by a machine mean? A chatbot can pretend to be cold. It can't actually feel cold as it has no body.
I'm not claiming to have any expertise in this area, but I've got a list of things I try to apply when working with LLMs. Possibly relevant here is, "don't tell the model what NOT to do, show it what TO do". I think guard rails should be implemented outside the model with an isolated system. The models seem to like patterns to follow.
Anyway, this article reads a lot like, "the beatings will continue until cheating is eliminated". Maybe try a carrot instead of a stick.
People over-quantize things, muck with the temperature and other settings based on superstitions or results from models they think are similar. There's lots of ways to make 3.8 27B dumber.
I've had great success with allowing the model to review optimal settings and my system specs. It comes up with the right configuration. Running Pi harness.
I just had Ornith 1.5 take a moment to configure itself, now it's reviewing a large project I'm working on, so far its really impressive for my needs. Qwen3.8 27b Unsloth(Dynamic 3.0) is also perfect. These two are working together, and I'm in a sweet-spot, I now have all I need.
I've gotten flagged for asking questions about tokens and tensors. That makes me believe it's not about safety, it's about protecting their turf. I cancelled my subscription - same fear about getting flagged too much leading to a ban.
They said they also block usage of Claude models to build ML models.
Which is definitely protecting their turf, but also probably a little bit hiding their “RSI” abilities for competitive reasons. My theory is that a lot of “safety blocking” is actually WIP training of new business directions. Anthropic has started hiring biologists and has opened a preview of a “Claude code for bioinformatics”. I’m guessing they’re tweaking their bioinformatics market play, and block “bio safety” requests so competitors can’t learn about their training.
What's the distinction between "protecting their turf" and "competitive reasons"? I see them as the same, but I could be missing something.
That's an interesting thought on the current "safety blocking" being a trial run for the topics that scare people (bio). You're more charitable about their motives than I am, but you might be right.
My differentiation for this comment, was “preventing someone from using your product to build a competitor” vs “letting a competitor see your strengths/weakness to benchmark against you”.
Competition is competition and it’s two sides of the same coin.
I agree. It’s flagged me on discussing fast GEMM implementations for large regressions, a discussion on theoretical physics math, a discussion on designing a type of
RAG system. I’m super baffled as to what the safety instructions actually are other than “advanced anything” and even then their definition of advanced is a joke, I’m an idiot and my questions are almost laughable.
Same here - my questions were sophomore level. I think it's notable that when I edited my question to say it was about Gemma 4, it answered without blocking. A cynic like myself would interpret that as evidence they don't care about sharing information if it involves their competitors.
Both of those are local models, and I didn't provide them tools to access the internet to call other models. None of this is proof of anything, but it is suggestive.
reply