www::curl - how to upload (post) large files

Test 16formpost.t is relevant. As you can see, it's completely disabled. This fact and my fruitless experiments with various return values for the callback function lets me believe the CURLOPT_READFUNCTION feature is known broken in the Perl binding.

I have to return the whole content through this callback. Is that right?

No, you can feed it the request body piecewise, suitable for chunked encoding. The callback will be necessarily called several times, according to the limit set in CURLOPT_INFILESIZE.

Which data does CURLOPT_READFUNCTION callback have to return?

A HTTP request body. Since you do a file upload, this means Content-Type multipart/form-data. Following is an example using HTTP::Message. CURLOPT_HTTPPOST is another way to construct this format.

use HTTP::Request::Common qw(POST);use WWW::Curl::Easy 4.14;my $curl = WWW::Curl::Easy->new or die $!;$curl->setopt(CURLOPT_POST, 1);$curl->setopt(CURLOPT_URL, 'http://localhost:5000');$curl->setopt(CURLOPT_HTTPHEADER, [    'Content-type: multipart/form-data', 'Transfer-Encoding: chunked']);$curl->setopt(CURLOPT_READFUNCTION, sub {    return POST(undef, Content_Type => 'multipart/form-data', Content => [        name    => 'upload',        action  => 'keep',        backup1 => [ '/tmp/backup1.zip' ],   # 1st file for upload    ])->content;});my $r = $curl->perform;

The CURLOPT_READFUNCTION callback is only used for chunked tranfer encoding. It may work, but I haven't been able to get it to and found that doing so wasn't necessary anyway.

My use case was for upload of data to AWS, where it's not ok to upload the data as multi-part form data. Instead, it's a straight POST of the data. It does require that you know how much data you're sending the server, though. This seems to work for me:

my $infile = 'file-to-upload.json';my $size = -s $infile;open( IN, $infile ) or die("Cannot open file - $infile. $! \n");my $curl = WWW::Curl::Easy->new;$curl->setopt(CURLOPT_HEADER,       1);$curl->setopt(CURLOPT_NOPROGRESS,   1);$curl->setopt(CURLOPT_POST,         1);$curl->setopt(CURLOPT_URL,          $myPostUrl);$curl->setopt(CURLOPT_HTTPHEADER,       ['Content-Type: application/json']); #For my use case$curl->setopt(CURLOPT_POSTFIELDSIZE_LARGE, $size);$curl->setopt(CURLOPT_READDATA, \*IN);my $retcode = $curl->perform;if ($retcode == 0) {    print("File upload success\n");} else {    print("An error happened: $retcode ".$curl->strerror($retcode)."\n");}

The key is providing an open filehandle reference to CURLOPT_READDATA. After that, the core curl library handles the reads from it without any need for callbacks.