Came here to say this. Glad to see /bin getting respect.
To anyone processing huge quantities of text and text files, someone very likely had the same problem you faced back in the 1980's and there's a Unix/GNU tool for it already.
I was introduced to *nix from processing very large text files that text editors I was familiar with choked and died. Someone showed me sed/awk/grep, and it took seconds to process when other GUI editors couldn't open the file. Never looked back.
To anyone processing huge quantities of text and text files, someone very likely had the same problem you faced back in the 1980's and there's a Unix/GNU tool for it already.