How to delete a gem path?
You can’t! I also wanted to do it, and haven’t found a way. Then I looked for in RubyGems source (at github.com) and learned that it tries to get the path from etc/gemrc and ~/.gemrc. If it fails (and fails GEM_PATH environment variable too), it assumes ~/.gem to be the default path.
Therefore, Max William is right, and you have either to export GEM_PATH or to create a ~/.gemrc. And it’s not a hack, for sure! :-)
Hope to have cleared things a bit!