VS 2010 setting non-GUI class file as Component
If you want to keep all the classes in one file, you can use the [System.ComponentModel.DesignerCategory("Code")] attribute on the GPSPort class to override the default behavior. Details here, note that you must use the fully qualified attribute even if you have a using System.ComponentModel statement or VS will ignore it.
At a guess, I'd say it's due to your GPSPort class, which extends SerialPort, which extends Component. Try removing that (or moving it into a separate file) and see if it fixes the problem.