PHP Curl request not working but works fine in POSTMAN

The site does a redirect, so you need to add

CURLOPT_FOLLOWLOCATION => 1

to your options array. When in doubt with cURL, try

$status = curl_getinfo($curl);echo json_encode($status, JSON_PRETTY_PRINT);

giving :

{"url": "http:\/\/www.mca.gov.in\/mcafoportal\/loginValidateUser.do?userNamedenc=hGJfsdnk%601t&passwordenc=675894242fa9c66939d9fcf4d5c39d1830f4ddb9&accessCode=-825374456","content_type": "text\/plain","http_code": 302,"header_size": 1560,"request_size": 245,"filetime": -1,"ssl_verify_result": 0,"redirect_count": 0,"total_time": 1.298891,"namelookup_time": 0.526375,"connect_time": 0.999786,"pretransfer_time": 0.999844,"size_upload": 0,"size_download": 0,"speed_download": 0,"speed_upload": 0,"download_content_length": 0,"upload_content_length": -1,"starttransfer_time": 1.298875,"redirect_time": 0,"redirect_url": "http:\/\/www.mca.gov.in\/mcafoportal\/login.do","primary_ip": "115.114.108.120","certinfo": [],"primary_port": 80,"local_ip": "192.168.1.54","local_port": 62524}

As you can see, you got a 302 redirect status, but a redirect_count was 0. After adding the option, i get:

{"url": "http:\/\/www.mca.gov.in\/mcafoportal\/login.do","content_type": "text\/html;charset=ISO-8859-1","http_code": 200,"header_size": 3131,"request_size": 376,"filetime": -1,"ssl_verify_result": 0,"redirect_count": 1,"total_time": 2.383609,"namelookup_time": 1.7e-5,"connect_time": 1.7e-5,"pretransfer_time": 4.4e-5,"size_upload": 0,"size_download": 42380,"speed_download": 17779,"speed_upload": 0,"download_content_length": 42380,"upload_content_length": -1,"starttransfer_time": 0.30734,"redirect_time": 0.915858,"redirect_url": "","primary_ip": "14.140.191.120","certinfo": [],"primary_port": 80,"local_ip": "192.168.1.54","local_port": 62642}

EDIT url encode the request parameters , and follow redirects

 $str = urlencode("userNamedenc=hGJfsdnk%601t&passwordenc=675894242fa9c66939d9fcf4d5c39d1830f4ddb9&accessCode=-825374456");curl_setopt_array(    $curl , array (    CURLOPT_URL            => "http://www.mca.gov.in/mcafoportal/loginValidateUser.do" , // <- removed parameters here    CURLOPT_RETURNTRANSFER => true ,    CURLOPT_ENCODING       => "" ,    CURLOPT_FOLLOWLOCATION => 1 ,    CURLOPT_MAXREDIRS      => 10 ,    CURLOPT_TIMEOUT        => 30 ,    CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_1_1 ,    CURLOPT_CUSTOMREQUEST  => "POST" ,    CURLOPT_POSTFIELDS     => $str,       // <- added this here    CURLOPT_HTTPHEADER     => array (        "cache-control: no-cache"    ) ,));

The simplest thing you can do, since you already have it working in POSTMAN, is to render out the PHP code in POSTMAN. Here is link about to get PHP code from POSTMAN. Then you can compare the POSTMAN example to your code.

<?php$curl = curl_init();curl_setopt_array($curl, array(  CURLOPT_URL => "http://www.mca.gov.in/mcafoportal/loginValidateUser.do?userNamedenc=hGJfsdnk%601t&passwordenc=675894242fa9c66939d9fcf4d5c39d1830f4ddb9&accessCode=",  CURLOPT_RETURNTRANSFER => true,  CURLOPT_ENCODING => "",  CURLOPT_MAXREDIRS => 10,  CURLOPT_TIMEOUT => 30,  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,  CURLOPT_CUSTOMREQUEST => "POST",  CURLOPT_HTTPHEADER => array(    "cache-control: no-cache",    "postman-token: b54abdc0-17be-f38f-9aba-dbf8f007de99"  ),));$response = curl_exec($curl);$err = curl_error($curl);curl_close($curl);if ($err) {  echo "cURL Error #:" . $err;} else {  echo $response;}

What is immediately popping out to me is this 'hGJfsdnk`1t'. The backward quote can be an escape character '`'. This could very well be throwing an error where error handling redirects back to the login page.POSTMAN likely has something built in to render out the escape character to 'hGJfsdnk%601t'. Thus, this works in POSTMAN, but not in your code.

Here is the status of this request:

{"url": "http:\/\/www.mca.gov.in\/mcafoportal\/login.do","content_type": "text\/html;charset=ISO-8859-1","http_code": 200,"header_size": 3020,"request_size": 821,"filetime": -1,"ssl_verify_result": 0,"redirect_count": 1,"total_time": 2.920125,"namelookup_time": 8.2e-5,"connect_time": 8.7e-5,"pretransfer_time": 0.000181,"size_upload": 0,"size_download": 42381,"speed_download": 14513,"speed_upload": 0,"download_content_length": -1,"upload_content_length": -1,"starttransfer_time": 0.320995,"redirect_time": 2.084554,"redirect_url": "","primary_ip": "115.114.108.120","certinfo": [],"primary_port": 80,"local_ip": "192.168.1.3","local_port": 45086}

Here is shows the successful login.

This is honestly one of weird sites I have seen in a long time. First thing was to know how it works. So I decided to use chrome and see what happens when we login with wrong data

Observations:

• Blanks username and passwords fields
• Generates SHA1 hashes of username and password fields and sets then in userNamedenc and respectively
• We can override username and password directly in JavaScript and login to your account just by overriding the details from console.
• There are lot of different request which generates cookies but none of them look any useful

So the approach to solve the issue was to follow below steps

• Get the login url login.do
• Fetch the form details from the response for the access code
• Submit the form to loginValidateUser.do

The form sends below parameters

Now one interesting part of the same is below post data

displayCaptcha:trueuserEnteredCaptcha:strrty

If we override the displayCaptcha to false then captcha is no more needed. So a wonderful bypass

displayCaptcha: false

Next was to code all of the above in PHP, but the site seemed so weird that many of the attempts failed. So finally I realized that we need to take it a bit closer to the browser login and also I felt delays between calls are needed

<?php    require_once("curl.php");    $curl = new CURL();    $default_headers = Array(        "Accept" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",        "Accept-Encoding" => "deflate",        "Accept-Language" => "en-US,en;q=0.8",        "Cache-Control" => "no-cache",        "Connection" => "keep-alive",        "DNT" => "1",        "Pragma" => "no-cache",        "Referer" => "http://www.mca.gov.in/mcafoportal/login.do",        "Upgrade-Insecure-Requests" => "1"    );    // Get the login page     $curl        ->followlocation(0)        ->cookieejar("")        ->verbose(1)        ->get("http://www.mca.gov.in/mcafoportal/login.do")        ->header($default_headers)        ->useragent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36")        ->execute();    // Save the postfileds and access code as we would need them later for the POST field    $post = $curl->loadInputFieldsFromResponse()        ->updatePostParameter(array(            "displayCaptcha" => "false",            "userNamedenc" => "hGJfsdnk`1t",            "passwordenc" => "675894242fa9c66939d9fcf4d5c39d1830f4ddb9",            "userName" => "",            "Cert" => ""))        ->referrer("http://www.mca.gov.in/mcafoportal/login.do")        ->removePostParameters(            Array("dscBasedLoginFlag", "maxresults", "fe", "query", "SelectCert", "newUserRegistration")        );    $postfields = $curl->getPostFields();    var_dump($postfields);    // Access some dummy URLs to make it look like browser    $curl        ->get("http://www.mca.gov.in/mcafoportal/js/global.js")->header($default_headers)->execute()->sleep(2)        ->get("http://www.mca.gov.in/mcafoportal/js/loginValidations.js")->header($default_headers)->execute()->sleep(2)        ->get("http://www.mca.gov.in/mcafoportal/css/layout.css")->header($default_headers)->execute()->sleep(2)        ->get("http://www.mca.gov.in/mcafoportal/img/bullet.png")->header($default_headers)->execute()->sleep(2)        ->get("http://www.mca.gov.in/mcafoportal/getCapchaImage.do")->header($default_headers)->execute()->sleep(2);    // POST to the login form the postfields saved earlier    $curl        ->sleep(20)        ->header($default_headers)        ->postfield($postfields)        ->referrer("http://www.mca.gov.in/mcafoportal/login.do")        ->post("http://www.mca.gov.in/mcafoportal/loginValidateUser.do")        ->execute(false)        ->sleep(3)        ->get("http://www.mca.gov.in/mcafoportal/login.do")        ->header($default_headers)        ->execute(true);    // Get the response from last GET of login.do    $curl->getResponseText($output);    //Check if user name is present in the output or not    if (stripos($output, "Kiran") > 0) {        echo "Hurray!!!! Login succeeded";    } else {        echo "Login failed please retry after sometime";    }

After running the code it works few times and few times it doesn't. My observations

• Only one login is allowed at a time. So not sure if others were using the login when I was testing
• Without delays it would fail most of the time
• There is no obvious reason when it fails to login except the site doing something on server side to block the request

The reusable curl.php I created and used for chaining methods is below

<?phpclass CURL{    protected $ch;    protected $postfields;    public function getPostFields() {        return $this->postfields;    }    public function newpost()    {        $this->postfields = array();        return $this;    }    public function addPostFields($key, $value)    {        $this->postfields[$key]=$value;        return $this;    }    public function __construct()    {        $ch       = curl_init();        $this->ch = $ch;        $this->get()->followlocation()->retuntransfer(); //->connectiontimeout(20)->timeout(10);    }    function url($url)    {        curl_setopt($this->ch, CURLOPT_URL, $url);        return $this;    }    function verbose($value = true)    {        curl_setopt($this->ch, CURLOPT_VERBOSE, $value);        return $this;    }    function post($url='')    {        if ($url !== '')            $this->url($url);        curl_setopt($this->ch, CURLOPT_POST, count($this->postfields));        curl_setopt($this->ch, CURLOPT_POSTFIELDS, http_build_query($this->postfields));        return $this;    }    function postfield($fields)    {        if (is_array($fields)){            $this->postfields = $fields;        }        return $this;    }    function close()    {        curl_close($this->ch);        return $this;    }    function cookieejar($cjar)    {        curl_setopt($this->ch, CURLOPT_COOKIEJAR, $cjar);        return $this;    }    function cookieefile($cfile)    {        curl_setopt($this->ch, CURLOPT_COOKIEFILE, $cfile);        return $this;    }    function followlocation($follow = 1)    {        curl_setopt($this->ch, CURLOPT_FOLLOWLOCATION, $follow);        return $this;    }    function loadInputFieldsFromResponse($response ='')    {        if ($response)            $doc = $response;        else            $doc = $this->lastCurlRes;        /* @var $doc DOMDocument */        //simplexml_load_string($data)        $this->getResponseDoc($doc);        $this->postfields = array();        foreach ($doc->getElementsByTagName('input') as $elem) {            /* @var $elem DomNode */            $name = $elem->getAttribute('name');//            if (!$name)//                $name = $elem->getAttribute('id');            if ($name)                $this->postfields[$name] = $elem->getAttribute("value");        }        return $this;    }    function retuntransfer($transfer=1)    {        curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, $transfer);        return $this;    }    function connectiontimeout($connectiontimeout)    {        curl_setopt($this->ch, CURLOPT_CONNECTTIMEOUT, $connectiontimeout);        return $this;    }    function timeout($timeout)    {        curl_setopt($this->ch, CURLOPT_TIMEOUT, $timeout);        return $this;    }    function useragent($useragent)    {        curl_setopt($this->ch, CURLOPT_USERAGENT, $useragent);        return $this;    }    function referrer($referrer)    {        curl_setopt($this->ch, CURLOPT_REFERER, $referrer);        return $this;    }    function getCURL()    {        return $this->ch;    }    protected $lastCurlRes;    protected $lastCurlResInfo;    function get($url = '')    {        if ($url !== '')            $this->url($url);        curl_setopt($this->ch, CURLOPT_POST, 0);        curl_setopt($this->ch, CURLOPT_HTTPGET, true);        return $this;    }    function sleep($seconds){        sleep($seconds);        return $this;    }    function execute($output=false)    {        $this->lastCurlRes = curl_exec($this->ch);        if ($output == true)        {            echo "Response is \n " . $this->lastCurlRes;            file_put_contents("out.html", $this->lastCurlRes);        }        $this->lastCurlResInfo = curl_getinfo($this->ch);        $this->postfields = array();        return $this;    }    function header($headers)    {        //curl_setopt($this->ch, CURLOPT_HEADER, true);        curl_setopt($this->ch, CURLOPT_HTTPHEADER, $headers);        return $this;    }    function getResponseText(&$text){        $text = $this->lastCurlRes;        return $this;    }    /*     *    * @param DOMDocument $doc    *    *    */    function getResponseDoc(&$doc){        $doc = new DOMDocument();        libxml_use_internal_errors(false);        libxml_disable_entity_loader();        @$doc->loadHTML($this->lastCurlRes);        return $this;    }    function removePostParameters($keys) {        if (!is_array($keys))            $keys = Array($keys);        foreach ($keys as $key){            if (array_key_exists($key, $this->postfields))                unset($this->postfields[$key]);        }        return $this;    }    function keepPostParameters($keys) {        $delete = Array();        foreach ($this->postfields as $key=>$value){            if (!in_array($key, $keys)){                array_push($delete, $key);            }        }        foreach ($delete as $key) {            unset($this->postfields[$key]);        }        return $this;    }    function updatePostParameter($postarray, $encoded=false)    {        if (is_array($postarray))        {            foreach ($postarray as $key => $value) {                if (is_null($value))                    unset($this->postfields[$key]);                else                    $this->postfields[$key] = $value;            }}        elseif (is_string($postarray))        {            $parr = preg_split("/&/",$postarray);            foreach ($parr as $postvalue) {                if (($index = strpos($postvalue, "=")) != false)                {                    $key = substr($postvalue, 0,$index);                    $value = substr($postvalue, $index + 1);                    if ($encoded)                        $this->postfields[$key]=urldecode($value);                    else                        $this->postfields[$key]=$value;                }                else                    $this->postfields[$postvalue] = "";            }        }        return $this;    }    function getResponseXml(){        //SimpleXMLElement('<INPUT/>')->asXML();    }    function SSLVerifyPeer($verify=false)    {        curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, $verify);        return $this;    }}?>