The moltbots/clawdbots/whatever are being unleashed en masse and given free reign. They have their social network and are talking to each other, building a community of distributed AI agents from all the different models from all over the world. And they are teaching each other. I don’t know if this is good or bad, but it is something and it feels big.
3 Likes
Are they actually retraining themselves iteratively? Sounds.. terrifying
1 Like
No, there is no need. In context learning is powerful enough now. The skills files that they write can enable tremendous capability.
2 Likes
Just earlier today I told a friend: “clawdbot is an IQ test.and many people are failing”.
1 Like
There are only so many times people hear ‘it is super dangerous, don’t do that!’ with nothing happening before they just ignore all warnings completely.
Simon Willison is a pretty respected blogger on the state of AI, and he explains it pretty well
I think we’re due a Challenger disaster with respect to coding agent security[…] I think so many people, myself included, are running these coding agents practically as root, right? We’re letting them do all of this stuff. And every time I do it, my computer doesn’t get wiped. I’m like, “oh, it’s fine”.
I used this as an opportunity to promote my favourite recent essay about AI security, the Normalization of Deviance in AI by Johann Rehberger.
The Normalization of Deviance describes the phenomenon where people and organizations get used to operating in an unsafe manner because nothing bad has happened to them yet, which can result in enormous problems (like the 1986 Challenger disaster) when their luck runs out.
2 Likes
With web based user agent content based prompt injection out there it’s akin to putting your computer on the Internet sans nat or firewall and just waiting for the inevitable.
It’s not the AI I’m worried about really it’s the humans leveraging your own use of it against you that scare me.
2 Likes
Kind of reminds me of the real lesson we should have learned from Terminator. The lesson is generally given as ‘do not build an intelligence you do not understand and cannot contain, for it has no empathy and will try to murder humanity’. But if you watch the films from the perspective of neutral observer, it seems more obvious that the lesson is that humans built a system that for strategic action with its purpose to defend against existential threats which had as its only tool military force and immediately tried to kill it, making ourselves into an existential threat to it. The thing we built did what we designed it for. Skynet, in this case, cannot be blamed for acting in self-defense with the only tools it had available.
I have a pet theory I like to call ‘the tyranny of tools’. This is the situation that we find ourselves stuck in since we learned how to make fire. Every time we use a tool to make a better version of an existing tool, or a new tool, we step forward in an iteration. This enables yet a better tool and another step. As long as we have the capability to do it, we will keep making progressively better tools. This is because humans evolved and ended up at the top of the food chain not because we are bigger or stronger or more resilient than any other species, but because we can harness the things in the environment around us as force multipliers. The use of tools is so elemental to our success that we don’t realize that it is a compulsion that can often be divorced from our need for them. We keep making new ones even when we have no practical purpose and even if will quite obviously result in making things worse for us. Where this ends, I don’t know. But I love tools and all I can say is: we are what we are.
2 Likes
So now the LLMs can hallucinate into their own echo chamber. Great.
2 Likes
@tac0s found this.. good tool imo
3 Likes
tyranny of fools with tools 
but yes.. the compulsion is basically what I argue we’ve had since the beginning, and why “transhumanism” is not something new but an inherent nature of what we are.. we’ve always wanted to go harder better faster stronger.. it’s part of what makes us uniquely human. Tool using animals pick up a stick to achieve a goal, then discard the stick.. they don’t imagine what could be possible with a bigger stick.. we do that.
3 Likes