Idle question: how did we end up with the term "inverted index" for this sort of thing?
The term "index" is used because of an analogy to the index of a book. But the so-called "inverted" index is the same way round as a book's index: you look up a word and it gives something analogous to page numbers.
Books (especially technical books) often have two indices: The forward index in the beginning of the book, which gives you a list of chapters/sections and where to find them; and the reverse index at the end that gives you a list of terms and gives you a list of of places to find each term.
It's the opposite of 'forward index', the wikipedia page (and its references) cover some of the background. I think you're overestimating how much this terminology is based on one particular kind of book index - indexing even before electronic computers was more sophisticated than that.
I don't think that's the story, again, I think you're overemphasizing a particular kind of book index you're familiar with. Here's usage of 'direct' and 'inverted' index from 1903 that I just googled up
I think it shows quite clearly that your theory that Information Retrieval nerds just got confused/misapplied a particular kind of book index is inaccurate and that IR nerds have been around for longer than one might think.
The term "index" is used because of an analogy to the index of a book. But the so-called "inverted" index is the same way round as a book's index: you look up a word and it gives something analogous to page numbers.