how to detect search engine bots with php?
I use the following code which seems to be working fine:
function _bot_detected() { return ( isset($_SERVER['HTTP_USER_AGENT']) && preg_match('/bot|crawl|slurp|spider|mediapartners/i', $_SERVER['HTTP_USER_AGENT']) );}update 16-06-2017 https://support.google.com/webmasters/answer/1061943?hl=en
added mediapartners
Here's a Search Engine Directory of Spider names
Then you use $_SERVER['HTTP_USER_AGENT']; to check if the agent is said spider.
if(strstr(strtolower($_SERVER['HTTP_USER_AGENT']), "googlebot")){ // what to do}
Check the $_SERVER['HTTP_USER_AGENT'] for some of the strings listed here:
http://www.useragentstring.com/pages/useragentstring.php
Or more specifically for crawlers:
http://www.useragentstring.com/pages/useragentstring.php?typ=Crawler
If you want to -say- log the number of visits of most common search engine crawlers, you could use
$interestingCrawlers = array( 'google', 'yahoo' );$pattern = '/(' . implode('|', $interestingCrawlers) .')/';$matches = array();$numMatches = preg_match($pattern, strtolower($_SERVER['HTTP_USER_AGENT']), $matches, 'i');if($numMatches > 0) // Found a match{ // $matches[1] contains an array of all text matches to either 'google' or 'yahoo'}