I recently found a fun CLI application and was playing with it when I found out it didn't have proper handling for when you passed it invalid files, and spat out a cryptic error from an internal library which isn't a great UX.
I decided to pull the source code and fix this myself. It's written in Swift which I've used very little before, but this wasn't gonna be too complex of a change. So I got some LLMs to walk me through the process of building CLI apps in Xcode, code changes that need to be made, and where the build artifact is put in my filesystem so I could try it out.
I was able to get it to compile, navigate to my compiled binary, and run it, only to find my changes didn't seem to work. I tried everything, asking different LLMs to see if they can fix the code, spit out the binary's metadata to confirm the creation date is being updated when I compile, etc. Generally when I'd paste the code to an LLM and ask why it doesn't work it would assert the old code was indeed flawed, and my change needed to be done in X manner instead. Even just putting a print statement, I couldn't get those to run and the LLM would explain that it's because of some complex multithreading runtime gotcha that it isn't getting to the print statements.
After way too much time trouble-shooting, skipping dinner and staying up 90 minutes past when I'm usually in bed, I finally solved it - when I was trying to run my build from the build output directory, I forgot to put the ./ before the binary name, so I was running my global install from the developer and not the binary in the directory I was in.
Sure, rookie mistake, but the thing that drives me crazy with an LLM is if you give it some code and ask why it doesn't work, they seem to NEVER suggest it should actually be working, and instead will always say the old code is bad and here's the perfect fixed version of the code. And it'll even make up stuff about why the old code should indeed not work when it should, like when I was putting the print statements.
I decided to pull the source code and fix this myself. It's written in Swift which I've used very little before, but this wasn't gonna be too complex of a change. So I got some LLMs to walk me through the process of building CLI apps in Xcode, code changes that need to be made, and where the build artifact is put in my filesystem so I could try it out.
I was able to get it to compile, navigate to my compiled binary, and run it, only to find my changes didn't seem to work. I tried everything, asking different LLMs to see if they can fix the code, spit out the binary's metadata to confirm the creation date is being updated when I compile, etc. Generally when I'd paste the code to an LLM and ask why it doesn't work it would assert the old code was indeed flawed, and my change needed to be done in X manner instead. Even just putting a print statement, I couldn't get those to run and the LLM would explain that it's because of some complex multithreading runtime gotcha that it isn't getting to the print statements.
After way too much time trouble-shooting, skipping dinner and staying up 90 minutes past when I'm usually in bed, I finally solved it - when I was trying to run my build from the build output directory, I forgot to put the ./ before the binary name, so I was running my global install from the developer and not the binary in the directory I was in.
Sure, rookie mistake, but the thing that drives me crazy with an LLM is if you give it some code and ask why it doesn't work, they seem to NEVER suggest it should actually be working, and instead will always say the old code is bad and here's the perfect fixed version of the code. And it'll even make up stuff about why the old code should indeed not work when it should, like when I was putting the print statements.