I had a very similar experience, but I started with plain Emacs, and little by little adding what I needed. I was afraid of getting something really bloated, or not understanding how it all bundles together. I think it has paid off a bit, but the learning curve has been steep.