Programming has a long standing culture of accepting the code to be somewhat wrong, so we have various tests, linters, reviews and error handling. Also in programming there are many ways to do something and it's the end result that matters most.

Not so in other knowledge work. There's no test harness for a contract and error is non-recoverable. Likewise in finance. There are specific ways of doing things and these ways are many times set in regulations. LLMs can assist all day, sure. But replacing the human, in highly regulated, zero tolerance for error environment?

The highly regulated, zero tolerance for error environment is a huge problem because anyone doing these jobs is more like a small sample size LORA than a general model.

It doesn't matter how smart someone is, they need specialized training to be good at these jobs. Specialized training in the area the company specializes in.

There is a category error in all this that is hard to think about because of the normal discourse and ordinary language. We say people work in "finance" but no one works in just "finance". They work at a company that has a specialization within "finance", inside a hierarchy that has specialization on top of specialization.

What we really need is exactly what we don't have and aren't going to get. A type of LORA that generalizes the task specific intelligence needed from a very small sample size and that in practice makes so many less mistakes in a highly regulated, zero tolerance for error environment that it is irresponsible to not use the model.

I have worked in this type of environment for 3 years and I have made zero mistakes in 3 years. The people that make even a small number of mistakes get fired.

Any real automation in this area is going to be incredibly slow and piecemeal over a long period of time because even an amazing model would need a long time to prove itself against what the human standards for error rates are.

Even the ensemble average error rate on a large number of tasks in space would not be good enough. It needs to be an average error rate over time.

You could say the Salvatorian Clause in contracts is like exception handling: a "catch (all)": even if some clauses in this contract are illegal, the remaining contract stays in place.

Logically, this actually doesn't make sense strictly speaking because the sentence creates a paradox: doesn't it make clear whether it includes itself or not, and each reading ends up in trouble. There is a "tradition" in law around the world to accept the only benign reading of such clauses, which I always found funny given that in all other ways lawyers adopt the most adversarial mindset imaginable.

I don't understand. I just provided advice about getting better output. Are you trying to reply to someone else?

You made some points worthy of expansion:

>> You should know - for coding they make terrible mistakes as well.

>> But programmers have this concept of a "code review" where another person looks at the code to look for problems.

You’re right; given that most of the money in the AI market is injected through OpenAI and Anthropic (which collect it through both selling equity and through customer revenue), the 7-8T is just a derivative of that.

It can't hallucinate, but it doesn't mean it can't make wrong decisions. Just because it adheres to a specific output format at all time, while LLMs have the output format at their mercy, then the claim of not hallucinating is made technically true.

I think that this specific part is not super interesting if your harness just recovers from invalid LLM outputs.

The latency and cost - yes, those are super interesting.

You can get rigid output format from "classic" LLMs https://docs.vllm.ai/en/latest/features/structured_outputs/ though model support is limited.

Would like to have something like in the original post but open weights.

We have an agentic system that produces insights for end users, and runs most of its work on DeepSeek v4.1 Flash but as an output stage transforms the resulting text through Gemini 3.8 Flash for readability, and it works.

On my TODO is try and run all of the analysis pipeline in dense "machine speak" to save on tokens and just let Gemini sort it out at the end.

In the savanna it’s not about who outruns the lion but who outruns their peers escaping the lion.

Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.

So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.

We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.

Code cost is almost down to zero. If you move the point of “just leave it to the machine” from the compiler (where humans used to do the coding) to the high level logic (now with LLMs) then in most cases more code does not really matter. Like, why build and maintain an abstraction where the LLM could implement this many times over each time with different subtleties? Why use a library with its own constraints when you could have exactly what you want? Why use cross platform frameworks when you can just one shot the thing to N different platforms? It’s not even slower. You can have code that’s larger yet more performant (stripping away abstractions can do that).

From time to time I try to do a pass of coalescing flows and cases and removing dead code to reduce the context and prevent the LLM from tripping over itself. But if it’s exclusively LLM maintained code I don’t care too much if there’s more of it.

The "exactly" part is the problem.

Just last two weeks I had to slap Fable, three times, to stop writing 1000-2000 lines of defensive code... because of DB columns I just forgot should be NOT NULL. That was it. Nothing else. I told it that, boom, -4800 coding lines: gone.

LLMs defend the status quo and they regularly lose sight of everything bigger than the current PR they are working on.

I too am gradually making peace with the fact that LLM-maintained code does not have to be 100% readable for humans.

But this is not about readability. It's about the data model. So one concession I am willing to make is: don't care too much about the code _BUT_ manually curate the data model. So far: small wins on iteration turns and code volume producing. Too early to tell but for now I am happy with the results.

Longer stem means bigger case. The case on the Pro is nice, the regular one not so much.

As a pretty avid Star Trek watcher from childhood, I found most of the things there technologically plausible other than the conversational nature of the ship's computer. Well, LLMs now are far more impressive conversation counterparts than those ships were ever depicted.

> As a pretty avid Star Trek watcher from childhood, I found most of the things there technologically plausible other than the conversational nature of the ship's computer.

so the faster-than-light travel seemed plausible?

Warp drive is among the more plausible ways to get FTL sure - it is compatible with relativity at least...

you're being extremely generous.

I share the sentiment but the question here is how long can we maintain the balance point where the human in the agentic loop is required. It might be a window lasting only a few years, or for the foreseeable future; I think the answer lies the opaque compute economics of the frontier lab: how well models keep scaling and how economically sustainable is serving those models under the current market conditions.

If my job gets automated, I'll find something else to do. I wouldn't have wanted lamplighters to succeed in preventing electrification, so it would be unfair for me to prevent the automation of my job if it can be done.

What if that something else pays less? Or doesn't pay enough to live at all?

That's life. I'm lucky to be interested in and also good at a profession that pays well. If I lose that and in exchange everyone gets to be good at making software, then that's a sacrifice I feel obligated to make for the benefit of humanity, and I'll still have more savings than someone not in this profession.

I use Astra to drive Fable; I drivel into my phone while walking in the forest and it builds. I don’t need to check; I do as clients need to pay, but it always is great. And it surprises me with things I did not know were possible even (never encountered them before so why would I know). We are at the point where our clients send voice messages and they get what they want without humans basically. This costs 10+10 max2 subs but that’s nothing compared to hiring people. We didn’t fire anyone; we just have 100+ more clients and make almost 50x more money. It’s boring but great as long as it lasts, we are already where you say for what enterprises generally need for the boring parts. That’s 99%.

I find I still add value, but I don't know if my value is real. Am I just biased and expect things to be done a certain way and penalize the model for doing something different? and am I providing the model with enough high quality context to align with my expectations in one-shot?