Apache still serving PHP file with incorrect permissions
Most likely what's happening is you have opcache running. PHP compiles itself at runtime. To save overhead, PHP 5.5 or later has Opcache, which saves that compile step. So Apache is probably caching the file. Apache would then store the cache file with its own permissions. Apache might also be caching it in memory as well. When you restart Apache it has to go back to the base PHP file, which it no longer has permissions to.