@movq@www.uninformativ.de I’m very curious…
What I like about this whole computer stuff is that you can explore how
things work. You can dig through problems and solve them. Nothing is
more satisfying than finally understanding something after you scratched
your head for some hours.
Surely you could do the same with AI? Tinker with how it works, study it, understand it, build your own and realize what it really is (without all the big tech hype)?
@prologic@twtxt.net Yeah, it’s hard to get my point across here. I tried to address that a few paragraphs down.
Yes, I can tinker with AI techniques on a general level. That’s cool but not really my area of interest.
What I certainly can’t do is learn how specific AI products work. I can’t possibly find out why Claude Code produced that particular line of code. Claude is just a magic box that does something and I have to trust it.
@prologic@twtxt.net Ahh, I see. Okay, I’m with you there. On this high level, I can understand how the thing works.
Maybe my wording isn’t good. 🤔 Let’s take a real life example from what we do at work.
There’s this AI chatbot. It gets support requests from users, so the user says something like “I need access to a particular system”. This triggers the bot to “run” the instructions stored in a large Markdown file, like “check if the user is authorized to do this, then issue the following API requests”, and so on. This is essentially like running a little script, except it’s written in natural language (German) and there’s no “script interpreter” but just the AI.
Now, suppose that the AI doesn’t quite do what was intended. There’s some subtle bug. How do you debug this? How do you find out how the AI came to the “conclusion” to run step A instead of step B? And how do you find out how exactly you have to change your prompt so this doesn’t happen again next time?
If this was an actual script/program instead of AI, you could repeat the request and attach a debugger or throw in some printf() or whatever. How do you do that kind of thing with AI? How do you pinpoint exactly what the problem was?
(Or is this just a stupid idea? Do we have to give up that way of thinking when using AI? Is the era of debuggability over?)
It’s one of the reasons in fact I’ve been working on bob so I have a very concrete and strong foundation for how these things work, how they behave and how bad or good they can be. I am on-purpose building bob to be not only a decent coding tool and general task completion tool, but with serious security boundaries, sanitation, auditing and compliance. If I’m going to succeed at building autoonmous agents that can cope with a wider array of varying inputs (mostly natural language, some structural language) then it needs to be both a) Safe and b) Robust
it’s “probabilistic” not “deterministic”
Yep, I know. And when I tell that to people and tell them “if we use AI here, we lose the ability to debug this stuff”, then all I get is: “But it’s good enough. We don’t need to debug this. Non-deterministic computing has its use cases.”
But that is just not how I’d like to model/implement our business processes. 🤔 I want something reliable, not “it mostly works”.
@prologic@twtxt.net (I hope I’m not too incoherent. I didn’t sleep very well recently and have a lot of unrelated stuff on my mind. 🤣)
@prologic@twtxt.net Oh yeah, same here. 😞 Let’s all just win the lottery and stop with this damn work thing. 🤣
@prologic@twtxt.net You actually did? 😅 Good luck. 😅 I never dared to, I’d probably get addicted. 🤣
@prologic@twtxt.net lol, well, better than nothing, eh? What did the tickets cost? 😅
@prologic@twtxt.net Jesus, that’s expensive. 🥴