Ain't Interested
My take on AI is, essentially, everybody who’s against it is too against it and everybody who’s for it is too for it.
I have written probably three pieces on this in the past year that I have chosen to not publish.
My main problem with transformer-based, LLM-based AI, aside from its supporters, and very occasionally some of its detractors, is this: it doesn't know what the fuck it's doing. I don't mean that the output is always gibberish, I mean that in a literal sense. It can't know what it's doing. It's the output of a statistical, mathematical process.
We see this plainly in the early iterations of various AIs. In images, we see detached limbs appear in thin air, McDonald's signs in impressionistic Linear B; in neural networks trained to recognize images, we see railway platforms not being recognized as such because the training set all contained a certain type of watch; in text we see it when the syntax is word salad. But when they improve, we compare it to human learning and human evolution (until now a fairly rational conclusion). We think: oh great, it learned.
Some models are better than others, some techniques of training the models or architectural decisions pay better dividens. But at no point does reasoning come into existence. You get out what you put in, both for training and for prompting. If we had a magical technology that would be able to differentiate good answers from bad answers, that would be able to tell facts from the wrong thing, we would use that technology. (The closest things we have are training and safeguards, which should neither be under- nor overestimated.)
What characterizes the output of an LLM is not its prescience or accuracy, but its ability to hew to patterns in a primordial corpus. Or to be more pedestrian: what you get out isn't wisdom, it's bullshit. It's not that what you get out isn't sometimes the truth. It's that there is no mechanism to which it matters whether or not it is. "Reasoning", "effort" and "thinking" are epistemologically equivalent to uttering spells while shaking the dice.
What scares the crap out of me from a professional and personal viewpoint are these things:
- It's not deterministic.
- It does not know the right thing, it's only reasonably capable of approximating a relevant answer from its training set.
- People will behave as if whatever answer given was produced by a competent person, who has experience and empathy, to whom there would be consequences if they were misleading or incorrect, and thus trust it to be correct, on their side, consistent.
It is a miracle that the things that work well work as well as they do; a miracle built on an upside down pyramid, with a whole industry trying to make it balance on its tip. That something of the kind can be useful in some applications, maybe even more so than it "should" be - well, I should hope so.
Certain players in the world economy are deciding to bet everything on this horse, as are some individuals in some industries. I am electing not to. We are nowhere near ready to deal with the consequences of any of this, but we're going to have to, in a world already marred by distrust and inequality.
See also: