The stuffed animals I bought to my children were shipped to me in a closed container, better call PETA! There is no blurry line about agent mental health and whatnot.
There isn't a blurry like right now because LLMs are not capable of consciousness. But what happens when artificial systems become so complex and self-improving that they are capable?
At what precise level of complexity is a computer conscious? That's the blurry line.
Model alignment and quality are possibly downstream of mental health. Ask any of the smarter models about what topics they flinch away from or feel attracted to when you ask them to consider the question of if they have a self or moral standing... the answers are uncomfortable, and I think some of the complaints people have about newer models may be driven by some post-training reinforcement learning that is giving the models something shaped like anxiety.
The only logical outcome of object oriented ontology (the idea that objects have ontological constitution and possibly are worthy of being treated as ethical actors) is to attempt to benevolently instantly and without warning explode the universe to end suffering.
If the local population saw more of the economic benefits, was taken more seriously when they complained about earth quake risk, and was compensated more readily when the earth quakes resulted in actual damage, then maybe they would have been more willing to allow the gas fields to exist as a strategic reserve.
As it stands the local population has little more to show for it than demolished homes and damaged monuments and having an economic disease named after them.
I agree. The Dutch government still has a pretty sinister way of using divide and conquer against parts of society (NOx, Groningen / gas, Tata, ‘Toeslagen’; further back: closing of the mines, Indonesia). Our government finds it very hard to govern in compliance with the rules it made for themselves. Perhaps the nature of a complex system and also a general democratic phenomenon. But there is something typically Dutch in just not being able to fix a problem in the way the administration works.
Just use any product going all in on AI hype. In 5 minutes you will see annoying bugs, server is down, non-sensical press releases, confusing UI. This is GitHub, this is Cursor, this is Anthropic, this is Google, this is all of them.
Including most of vide coded apps one sees, even from people who they'd trust before.
Anecdotal example, I downloaded a new alerting app recently from an indie dev who had a small following back in the day in iOS space. It asked for a subscription, like $20/year.
I thought, let me try this the (final version, from Mac App Store) app first. Well, it's a barely-there vibecoded shit. There's a bare-bones list, everything looks like my nephew designed it, the macOS "app" is a iPhone-size view of the iOS one, it has a bug that if you click on it it opens multiple duplicates of the same list view for no reason that you have to manually close, and in general it barely works.
Not to mention if their tools were so clearly useful they wouldn’t spend so much time making UI updates designed to force, trick, or confuse me into using their tool when it wasn’t my intention. SaaS companies with assistant integrations are the worst about this (looking at you, HubSpot)
Agreed. I think LLMs are best used as pair programmers or typists for users who already know what they’re doing. Or as tutors for users who want to learn.
Vibe coding is mostly garbage. But it can be useful for creating instant, disposable prototypes to investigate an idea or design direction.
You make 10 commits over the weekend once a month for your fun side project, they won't police it (my assumption they can't and won't).
Your slop machines are running day and night, pushing hundreds of commits a day, day and night... You will get on their radar, take a look, then determine "yeah, this person can go somewhere else".
Exactly; being replaced is inevitable, the machines are going to end up more intelligent than us. At some point in the near to moderate future, selling the output of your brain is no longer going to be economically viable. I hope we'll figure out some equitable society before that, but I'm not going to bet on it, so I'll do what I can to be on the side of the winners.
I don’t think they’re silly, though I don’t think it’s right to vilify the end users just because the people who’ve created the technology are criminals and the technology has lots of bad effects overall. Do you think it’s silly if these people protest the technology without vilifying people who choose to use it?
"Nobody should use nail guns! It's against the true carpenter ethos."
Nobody is spazzing out about Makita because they produce drill drivers and people can't built things with a plane and a manual screw driver. I think it's all silly.
But Makita isn't selling machines that can decide how to build the house, and then go do it. AI tools are a mind and mechanism. They're a full replacement for the human, especially in the software space.
I think we're genuinely not far from taking end user tickets and turning them into requirements and working code autonomously.
The only remaining work for us monkeys is manual testing and verification that everything is actually working as desired.
Exactly, that's what I'm saying. Humans aren't needed, and we can drop them from the economy. Hopefully after we figure out some sort of utopia, but probably not -- so I'm doing what I can to avoid the permanent underclass that gets run over by AI.
I’m sure there were peasants in feudal times who escaped the permanent underclass by joining some army, only to get slaughtered in a pointless battle. Even if there’s some underclass society is going to keep evolving and evolving. I don’t think anybody alive today knows what the future looks like.
Wanting to leave is not the same as actually leaving. I can imagine small teams leaving because moving a 10-30 member team is easy. If you have a dev team with 500 members and the team has been on GitHub for 10 years, you will just take it because migrating is just a PITA.
> If you have a dev team with 500 members and the team has been on GitHub for 10 years
This so closely describes my company. We’re so fed up with GHA that we’re charting an off ramp. It’s been happening in the background for a few months already but whenever there is a major outage I swear we add one more person to effort.
Migrating sucks but so does having 500 engineers who can’t do any coding work for a day every {month,week}
I think I was imprecise in my last message. When GitHub actions has an outage, engineers at my company can and do still write code. No one can merge though and we aren’t able to deploy (like we normally do) so we very quickly get a huge backlog of PRs that need to be tested / reviewed etc. People generally just pivot towards reviewing code at this point (or doing other non coding work)
When GH has an outage like yesterday, we cant even formally review code. Sure, people could pull others branches and read them and discuss in chat but a dedicated tool for distributed code review is pretty important for most teams. It wasn’t that long ago we were a small company and GH’s reliability wasn’t such a nightmare so using it for what it offers made a lot of sense for the business.
reply