Debug symbols will not slow down the application. You can work with split debug symbols as follows.
gcc -ggdb -o target obj1.o obj2.o ...strip target --only-keep-debug -o target.dbgstrip target
Then in gdb, use symbol-file target.dbg
symbol-file target.dbg