If you are using it with something like fuse or expandrive you are not using it over SSH but on local files.
If you are using it in a remote Vim via SSH, you are not using it over SSH either.
If you did something like :e scp://foo//bar/ and see a directory listing you are not using NERDTree.