Take

A calculator does arithmetic for you — thus automating the tedious, repetitive part — but you still have to know which buttons to press to get the answer you want. You can’t just type the entire problem in and get Whatever — something that sounds plausible, with a microscopic disclaimer that checking it for accuracy is your problem.

Calculators do have limitations at their extremes, and if you’re working with extremes, you have to be aware of those. Table saws will (or, used to) cut through fingers just as happily as wood. Tools have edge cases — at their edges. LLMs have edge cases everywhere, and they are constantly changing, even minute to minute, even for exactly the same input fed to exactly the same model.

This entire piece is more or less personal catnip, including being upset both at the immovable payment landscape, the universal adoption of the word "content" by the same people who are otherwise overly eager to refer to the movie Idiocracy as prescient, and the sudden redefinition of autocompleting quotes and brackets everywhere.

I have a lot of respect for Simon Willison, whose quote apparently sparked the post. I am closer to Eevee's conclusion than to Simon's, but I don't doubt that when Simon says it is making him more productive, he's telling the truth. (Maybe the even darker conclusion is that the twin clusterfucks of a) the 57 varieties of packages and virtual environments in Python ("there should be one-- and preferably only one --obvious way to do it" indeed) and b) "modern" web development are somehow still amenable to big pushes of potentially broken code being a net positive compared to the alternative.)

I don't write about "AI" or LLMs here. It is everywhere, and like the post (read it), I don't like it. It has the mouth-feel of something that bubbles up, is annoying for a time and then eventually goes away, and I prefer to not spend time writing about it. (I think "NFT" and "Web 3.0" will be the first time they're mentioned here.) But aside from billion dollar companies acting like if they get this right by pushing this into every nook and cranny of everywhere, aside from the ethics and the legal implications and people being stark raving idiots with this tool too, there is a way that it affects me.

The way it affects me is that yes, it is cute to get it to write things for you. I enjoy writing. You may have noticed.

It is also maybe interesting to get it to help you write code, assuming that it is reliable, or at least consistently, discernibly wrong in the places where it falters. But it isn't. I am not bothered by statement completion, which is just a way to narrow down/home in on alternatives with the added benefit of showing you what the tooling thinks is available at that point. But I am made anxious by swaths of code popping up that I constantly have to evaluate to see if it's right or not. It robs me of my momentum and makes me lose my train of thought.

Okay, but what about just it coding for you and you don't have to watch things flickering? For larger bodies of code, I think it essentially boils down to what Mark Russinovich lays out (20:03 and going for a few minutes). It does an okay job, until it doesn't. It understands you, until it doesn't. And, as Eevee's piece brilliantly captures, it produces Whatever. It doesn't recognize when it falls off a conceptual cliff. It picks out the next Scrabble tile out of the bag on prehistoric Earth, and whatever it spells is what you get. Sometimes it's wild; sometimes it's a death threat.

It may, somehow, make economic sense to replace people with a bunch of those LLMs just going hog wild, if you ignore the bit where there are consequences and someone will have to clean it all up. (If we're all very lucky, maybe they'll have time to spoil the human reservoir of knowledge first! It was all worth it, if it meant we could outperform the bearish analysts in Q3.)

Anyway. I am not currently afraid for my job. But I am concerned about the rest of existence.