Or better context curation - less lossy compression saving back to context. Maybe even jettisoning part context into an external semantic store instead of conpression. Or placing less data into context to start with.

Or a combination of all those things.

Great story!

Verifying sources is a recursive problem - where do you stop? Humans have intuitive feel for it, but agents don’t or at least not yet (I wonder if intuition is just a secondary neural net which is currently being added to the agents as we speak).

Also as a human you are able to examine agents erroneous trajectory, real or imaginary, without contaminating your own. Agent have a problem with that - as soon as someone else’s thought is in the context it can lose track of provenance and veracity. Sometimes I think we need a bloom filter to retroactively assign “dirty” flag to invalidated or questionable token spans already in the context.

Even before AI I strongly believed the internet was eventually going to have to move to a web of trust model. I think World ID (formerly Worldcoin) can be a really cool part of the solution to this, though people widely criticize it without fully understanding it and make assumptions that are wrong. But I agree that trust and verification is increasingly a large problem and one of the best ways to combat that is to actually choose unique identities to trust.

> I wonder if intuition is just a secondary neural net which is currently being added to the agents as we speak

Arguably, intuition is primary neural net, the only thing an LLM has without CoT, it just is spiky so humans only notice where it's below-average and just dismiss the rest as normal. Of course it's going to lag behind in some areas compared to others.

I observed the same, and generalized it as inability to recognize salience and more broadly apply discretion.

In turn it makes me wonder how do humans do those things? Perhaps it is our human job to apply discretion going forward.

Probably a quote from 3-body problem.

The Trump and former president terms were likely firmly stuck together in the embedding space. The model doesn’t validate every single token it produces because validation itself requires tokens. A bloom filter of outdated embeddings will help, when the labs get around to adding it.

How so?

It’s probably brittle though? Replication implementation has to change in some ways from one version to another.

Right, it takes some effort, but it’s worth it for the benefits of sub-second latency and reduced operational overhead. We expect the WAL format for existing commands to remain relatively stable across Postgres versions, while newer commands may need additional handling.

I like to think the real world lessons in failures are valuable.

If I have this idea one day I will search for it and then think “how am I different fro that which already failed?”.

You may be interested in TITANS:

Test-Time Learning: The model updates its own memory weights while running an inference task.

Perhaps our own statefullness is a hack of nature. We have electrical signals in our brains, neurotransmitters, neuron growth. By any reasonable measure it’s a hack on top of a hack. But it works well enough for us to get buy. So it does for the agents.

That is true that intelligence is kind of a "freak of nature" in a way, but it's also true that even single neurons are extremely efficient, and the brain has a lot of recurrence that isn't represented at all really in today's artificial neural networks.