cURL - How to fetch page only if it has changed since last fetch? cURL - How to fetch page only if it has changed since last fetch? curl curl

cURL - How to fetch page only if it has changed since last fetch?


I looked for answer for more than 2 days, and nobody could give me universal answer.

So I implemented etag and if-modified-since headers (as Matt Raines and sowa posts here), also to lower traffic I used compression like gzip.

Also there is request header Range, so that i could grap only part of the page as someone told me, but i think it is used only for files not web pages.

Thank you all for your time


Update local file with remote, iff remote is newer

Cut and paste answer for those who want to
check if a remote file is more up to date than a local one, and update the local file if so:

    // $remotePath = 'http://blahblah.com/file.ext';     // $localPath = '/usr/whatever/app/file.ext';    $headers = get_headers( $remotePath , 1 );    $remote_mod_date = strtotime( $headers['Last-Modified'] );    $local_mod_date = filemtime( $localPath );    if ( $local_mod_date >= $remote_mod_date ) {        // Local version up to date     } else {        // Remote file is newer        $ch = curl_init();        curl_setopt($ch, CURLOPT_URL, $remotePath);        // other options here, eg: curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2);        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);        $result = curl_exec($ch);        if (curl_errno($ch)) {            // handle error : curl_error($ch)         }        curl_close ($ch);        if ( $result ) {            // Update local file with remote file contents            file_put_contents( $localPath, $result );        }     }

With thanks to OP question here, and also this answer.
Created to solve automatic OIDC CA cert renewal (this, and this).


Does curl_setopt($curl, CURL_HTTPHEADER, ["If-Modified-Since: 2016-04-30 21:00:00"]); work? I get a 304 Not Modified response on a resource that was last modified earlier in the month.