Part skeptic and part zetetic, what prompt would you have used?

My impression is that this is an oversimplified demonstration of what can happen when you prompt Claude in a system with many more variables (than two buttons and two colours).

If I want the button to turn blue and that's it, what instead do I ask? Even in a complicated system with many levers, what do I request other than the desired end result, hoping that Claude pulls the right levers to produce something acceptably close to what I think I asked for?

I dunno, but in my experience Claude looks at all the levers in your codebase, understands what they do, and then generally figures out how to achieve the desired end result.

And when it doesn't, it's usually because of things outside of the codebase -- iOS layout quirks that aren't documented, buggy Python libraries it's relying on where you then have to tell it to read the source to figure out what's going on, that kind of thing.

The other thing that I think a lot of people run into is that they launch it into a completely human-built system with hundreds of thousands of lines of code and expect run #1 to be perfect.

You have to change the system so that the AI understands it, via establishing what your beliefs are, how those are reflected in values (especially important if you have e.g. compliance needs), how those values are reflected in the operational and strategic levels, and then a variety of tactical behavior coaching. For example, I ban 2>/dev/null - super tactical, and I say I value simplicity over covering every edge case - a very broad generalization.

"Make the Add to Cart button blue" is fine as a first step. When that doesn't have the desired result, I would simply revert the change, start a new session, and do something like "Give the Add to Cart button its own theme separate from the default button theme. Make the color for this new theme blue.". If this gives almost the correct result (e.g. it's not the shade of blue I want) I'd do a followup like "change the Add to Cart theme from #00FF00 to #16b8c4" with no implication that the previous step was wrong. If it's wildly incorrect I'd start over and try to specify in more detail about what exactly should happen.

If I don't understand why changing one button changed them all, I would use an entirely separate session to ask questions about how the site theming works. The fact that the site has a bunch of weird coupling between themes is useful information, and if I don't know how to resolve that I'd ask Claude for ideas about how to safely eliminate the coupling, and once it proposes a reasonable idea tell it to implement that.

The two big things here I'd never do is use emotional languages in prompts, and I'd never tell Claude to revert changes and try again. Once the incorrect change is in the context it's poisoning all of your future results.

Surely at any point after "Make the "Add to Cart" button blue", it'd be faster to just do it yourself: "grep -r "Add to Cart", insert "addToCart" as a class to the html element, then crack open style.css to stick "button.addToCart {background-color: blue}"

I'm counting maybe 80 keystrokes? That's shorter than your second prompt.

This idea generalizes. Large Language Models are poorly suited for tasks that we have already purpose-built systems to be easy for humans to use. The easiest way to tell your website that you want a button to look a certain way is to update the code. If you know exactly how you want something done, we have developed an incredibly efficient way to tell computers how something should be done: it's called source code.

LLMs work best when they're handed tasks that you don't want to figure out how to do.

It "makes sense" that plainly telling the chat bot to make one button blue makes the whole site blue? It "makes sense" that correcting it and repeating 3 times that no, only the one button should be blue, should make the button a gradient and start a philosophical discussion around the nature of "blue"?

Wow, you AI people really have a negative view of the technology y'all are trying to sell as the next Jesus

I'm not trying to sell the technology at all, much less as a messiah, and I absolutely have a negative view of it. It's a token predictor - nothing more, nothing less. If you treat it like it is supposed to "make sense" of things, you get predictably bad results. It "makes sense" that if you don't make the sense for it, it will do poorly, because it's a glorified auto-complete.

If the agent's change has such a catastrophic effect, the first thing you do is tell it to explain why its change had that effect.

Once you understand what the problem is, you can give it better instructions. If the architecture is shit, the agent is going to have a rough time of it.

So how would you prompt it instead? Because that's exactly how I prompt it, because any reasonable human being would know exactly what I mean by "Make the shopping button blue", and I am sick and tired of getting shitty results.

The first prompt is fine, it's the following ones that are poor.

> Why is half the site blue now? I asked you to change one button.

> Half the site is blue. I asked for ONE button

Neither of these is an instruction to fix the problem, they're treating the AI like a person and telling it what it did wrong, expecting the implied admonishment to be enough to steer it back. But without an actual instruction, it just goes and does whatever it thinks will help, which is often arbitrary.

The response I would have used in this situation is

"The Cancel button is also blue now. Make sure the color change is only scoped to the Add to Cart button"

Most of the available responses throughout this "skit" are similar cases of expressing frustration first and guiding the result second.

Skip the emotion and say exactly what you want, and nothing besides that.