Apache permission denied

This was the correct way to do it: (thanks to DaveRandom)

<Directory "C:/SITE/localhost/www">    Options ExecCGI    AllowOverride all    Require all granted</Directory>

Dave Random explains further:

After a little experimentation with this, I have discovered the nuance that makes this the correct answer, which is specific to Apache 2.3+. It seems that mod_authz_host directives take precedence over mod_access_compat directives, and this bubbles all the way up the directory tree. What this means is that if you are migrating from Apache 2.2 to Apache 2.4 and you use your 2.2 httpd.conf verbatim, it will work.

If, however, you perform a new install of 2.4 and base your config on the default 2.4 httpd.conf, Allow directives won't work, because the default top level section uses a Require all denied directive instead of Deny from all, and this takes precedence over any subsequent Allow directives higher up the tree. The long of the short of this is that if you are migrating your Order/Allow/Deny directives to their equivalent Requires, then you must chance all of them or you will find you get 403s you weren't expecting.