I think the intended meaning by those who say it is: I'm currently caring too much because I'm talking about it even a little bit. I could easily care even less than this.