In the short term, it may be unrealistic (as you illustrate in your story) to try to successfully navigate the increasingly fragmented, fragile, and overly complex technological world we have created without genAI's assistance. But in the medium to long term, I have a hard time seeing how a world that's so complex that we can't navigate it without genAI can survive. Someday our cars will once again have to be simple enough that people of average intelligence can understand and fix them. I believe that a society that relies so much on expertise (even for everyday things) that even the experts can't manage without genAI is too fragile to last long. It can't withstand shocks.
While I generally agree with this, I have mixed feelings. On the one hand, AI could be smart enough to reach the "enlightened master engineer" and can reach super-human levels of...simplification. In some ways, complexity can result from improper layering and abstraction inversion. It takes a holistic view to realize that the lines (i.e. interfaces) between layers were drawn improperly and redesign everything together, achieving an overall simplification.
A good example is the web platform. It's just enormous...to the point that no human can really understand how it all even works. And I say that as someone who worked for a long time on a narrow part of that stack (V8). It being only a little over a million lines of code, it is incredibly intricate and subtle, because it implements a pretty weird language, has lots of optimizations, advanced GC, multiple compilers, etc. And that's just the JS engine. Add in the layout engine, rendering engine, multi-process architecture...it's beyond the comprehension a single mind.
We're not yet at the level that an AI can understand code really deeply yet, but may we will reach the point where an AI understands enough of it and can code competently enough to start over from scratch and build something we can both understand and does the things we actually want it to do.
I do agree with the fragility argument. Though if/when the shock comes, I doubt we’ll be anywhere near being able to build cars. Especially taking into account that all the easily accessible ore has long been mined and oxidized away.
