How to make Apache forbid a file only if it exists?
There are two reasons why it's not matching app-968b520079.css
The first is the "app" bit, which is not contained in the
(admin|staff)
at the beginning of your regex.
You should change that to
(admin|staff|app)
if you want to match anything starting with "app-".
The second is the fact that the 968b520079 contains two zeros, and you're testing for
[a-zA-Z1-9]
which means: a to z, A to Z and 1 to 9
You should test for 0 to 9, change it to
[a-zA-Z0-9]
and your regex shoud work just fine.
Your first "what I tried" example only matches when the translated URL is simultaneously a directory and a file, which is never true. You need [OR] to join the two conditions or drop the -d condition completely (due to your regex matching things that would commonly be files not directories)
RewriteCond %{REQUEST_FILENAME} -fRewriteRule 403 (admin|staff)-?[a-zA-Z1-9]*\.(css|js|php)(.map)?$