Why won't bundler install JSON gem?
So after a half day on this and almost immediately after posting my question I found the answer. Bundler 1.5.0 has a bug where it doesn't recognize default gems as referenced here
The solution was to update to bundler 1.5.1 using gem install bundler -v '= 1.5.1'
Run this command then everything will be ok
sudo apt-get install libgmp-dev