If you expect to be able to ask it an underspecified question without context and without telling it what role it should take and how it should act, sure, that often fails entirely. It's not a productive use of ChatGPT at all.
If, on the other hand you actually put together a prompt which tells it what you expect, the results are very different.
E.g. I've experimented with "co-writing" specs for small projects with it, and I'll start with a prompt of the type "As a software architect you will read the following spec. If anything is unclear you will ask for clarification. You will also offer suggestions for how to improve. If I've left "TODO" notes in the text you will suggest what to put there." and a lot more steps, but the key element is to 1) tell it what role it should assume - you wouldn't hire someone without telling them what their job is, 2) tell it what you expect in return, and what format you want it in if applicable, 3) if you want it to ask for clarifications, either ask for it and/or tell it to follow a back and forth conversational model instead of dumping a large / full answer on you.
The precise type of prompt you should use will depend greatly on the type of conversation you want to be able to have.
I'm starting to wonder if it's just easier to actually program?
Like I know ChatGPT-4 can generate a bunch of code really quickly, but is coding in python using pretty well known libraries so hard that it wouldn't just be easy to write some code? It's super neat that it can do what it does, but on the other hand, modern editors with language servers are super efficient too.
If I want it to just write code where I know exactly what I want, I will have it write code. ChatGPT can write code and fill in things when you give it something well specified very quickly.
My point was that if you just ask it an ambiguous question it will return something that is a best guess. It's what it does. To get it to act the way the person above want it to, you need to feed it a suitable prompt first.
You don't need to write a new set of instructions every time. When I "co-write" specs with it, I cut and paste a prompt that I'm gradually refining as I see what works, and I get answers that fit the context I want. When I want it to spit out Systemd unit files, I cut and paste a prompt that works for that.
The stuff I'm using it for is stuff where I couldn't possibly produce what it spits out as productively not because it's hard but because even typing well above average typing speed I couldn't possibly type that fast.
It's perfectly fine if it's approaching a Clever Hans type situation as long as it's producing sufficient quality output fast enough that it's producing it faster than I can do manually.
There are many categories of usage for them, and relatively "dumb" completion and boilerplate is still hugely helpful. In fact, probably 3/4 of my use of ChatGPT are uses where I have a pretty good idea what it'll output for a given input and that is why I'm using it, because it saves me writing and adjusting boilerplate that it can produce faster. Most of the time I don't want it to be smart, I want it to reliably do almost the same as it it's done for me before, but adjusted to context in a predictable way (the reason I'll reach for it over e.g. copying something and adapting it manually).
We use far dumber agents all the time and still derive benefits from it. Sure it'd be nice if it gets smarter, but it's already saving me a tremendous amount of time.
Yes, I think using it for code that you could write yourself fairly easily is a sweet spot since you can quickly check it over and are unlikely to be fooled by hallucinations. It can save significant time on typing out boilerplate, refreshing on api calls and type signatures, error handling, and so on.
It’s a save 15 minutes here, 20 minutes there kind of thing that can add up to hours saved over the course of a day.
One of my go-to examples is that I asked ChatGPT to write a DNS server. It didn't get it perfect and needed followup questions, but it 1) got it better than I could (having written a couple in the past) without spending time reading the RFCs, 2) because it's something I have done before, even though I'd have to look up docs to do it again, I could instantly tell which areas it doing the right thing, and where I needed to check specs and adjust details.
But if I didn't know how a DNS server works, on the other hand, it'd have been little help to do it that way because I'd have no idea what.it got right or not, and it's have been far more productive in that case to ask it for a step by step guide of what to do and which portions of the specs to look up.
You have to treat it as a junior:
Either give it tasks you can trivially validate and improve because you know the subject, or ask it to help condense down and explore the search space.
Don't try to get it to complete a task you don't understand how to quickly validate, because you'll waste tremendous amounts of time trying to figure out whether the answer is right.
Most of the people complaining seems to expect it to work well for the latter even when told it's not (yet, anyway) a good use.
If, on the other hand you actually put together a prompt which tells it what you expect, the results are very different.
E.g. I've experimented with "co-writing" specs for small projects with it, and I'll start with a prompt of the type "As a software architect you will read the following spec. If anything is unclear you will ask for clarification. You will also offer suggestions for how to improve. If I've left "TODO" notes in the text you will suggest what to put there." and a lot more steps, but the key element is to 1) tell it what role it should assume - you wouldn't hire someone without telling them what their job is, 2) tell it what you expect in return, and what format you want it in if applicable, 3) if you want it to ask for clarifications, either ask for it and/or tell it to follow a back and forth conversational model instead of dumping a large / full answer on you.
The precise type of prompt you should use will depend greatly on the type of conversation you want to be able to have.