WPF MouseDown event not firing everywhere in a control
<StackPanel Background="Transparent" ... >or
<Border Background="Transparent" ... >should do the trick...
This makes the grid transparent, however visible for mouse-clicks.
Look here form more Information.
There are some other circumstances, where you have to use the PreviewXXX- event, this is when a child element "swallows" the event away, but in your case, the above should be what you're looking for.