In a recent(ish) issue of TD, I came across an article titled "Prompting is the problem," which piqued my interest (archive link in case you don't have a subscription).

I think there are pros and cons here, so I don't want to dwell on just the negative. For example, the author writes that...

"Here's an inconvenient truth: Because AI systems are probabilistic and in motion, the same request won't always give the same answer—and the same model won't behave the same way month to month. Research demonstrates measurable behavioral drift across major large-language-model providers."

"That scenario illustrates why syntax-first prompt training backfires. When we teach people to chase consistency from a tool designed for variation, the result is hesitation, not value. Rather than treating AI prompting as the skill, start teaching people how to think with AI—iterate, question assumptions, and use conversation to move the work forward."

My gripe here is that for the last year (or two...or three) the blame about LLM outputs has been resting solely on the shoulders of end-users.  "Oh, you're doing it wrong," or "maybe if your prompts were better, you'd get better outputs." This drove folks to books and workshops for better prompting, but it's all pretty pointless, IMO, because results are still probabilistic slop. Sure, you may be able to get something that approximates what you had in mind through "better" prompting, but in a teaching and learning scenario, you'd want consistency (I'd argue) over probabilistic outputs.

The author claims that...

"AI systems are valuable because they show users their own thinking back. The real skill is metacognitive. Instead of users stopping at what the tool gives them, they should push it to help them see what they missed. That is the antidote to cognitive offloading. Users should actively engage with the AI model's outputs to strengthen their own thinking. Every output is a chance to challenge a user's defaults, surface blind spots, and strengthen the individual's reasoning."

There are two threads here.  In one thread, I feel like the author is trying to get a bit Vygotskian here and treating the LLM like a more knowledgeable other, which they are not. An MKO consciously pushes you just beyond the reach of your own understanding and helps you grow.  An MKO, by definition, is more knowledgeable, and you (in theory) don't need to second-guess what they know. You can trust that they are presenting you with current information and know-how, and you can grow your practice. With an LLM, you'd need to know what's already been given to you as an output in order to better assess it, and at that point, what's the point of using this as a learning tool?  The other thread is that the LLM becomes a mirror for your thoughts.  Cool, I guess, but do we really need a modern-day ELIZA?  The evidence, so far, is that LLMs basically become our little yes-men.  Is this helpful in learning?  If what is being reflected to us is not something that is useful and continues to steer us down the wrong path, is this helpful or detrimental to the learning process.

Finally, the author writes...

"Technique-first training creates exactly the pattern that Your Brain on ChatGPT warns about: People learn to focus on getting the prompt right instead of determining whether the output is valid, complete, or useful."

I think this is the biggest failure in my view.  Let's say that I want an LLM to give me a literature review on a corpus of 100 articles.  I haven't read any of them, so I don't know if they are useful or quality research, and I wouldn't be able to tell you what the main points are with regard to how I want to use them in my research. To do that, I'd need to do the actual work of reading, assessing, thinking, and cognitively processing those 100 articles.  To ask an end-user to determine whether the output is valid, complete, and useful negates the point of having the LLM do the work in the first place.

In the end, it feels like a lot of these articles are just making excuses as to why the technology doesn't work, and how it can work better, rather than writing this tech off completely, or going back to the lab to experiment a bit with it, rather than trying to shoehorn it into everything.