What to ignore in bower.json
You're right, just the minimum. I would go a step further and drop the minified file, which is the best practise in Bower, as users are most likely using a build system and having an additional minified file is just wasting space.
You can easily ignore everything but one file by using globbing in the "ignore" property in bower.json like this:
"ignore": ["./!(genie.js)"]