libcurl delays for 1 second before uploading data, command-line curl does not
The reason for the delay was:
- libcurl was sending an
Expect: 100-Continueheader - The server (which is based on mongoose) is not configured to send
100 Continueresponses automatically. - libcurl waits up to 1 second for this response. If it does not receive it after this time then it proceeds to send the request body anyway.
A solution on the client side is to disable the Expect header like so:
headers = curl_slist_append(NULL, "Expect:");curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);// ...result = curl_easy_perform(curl);curl_slist_free_all(headers);
This answer is not an Answer, but shows where the problem is happening in curl. I had the same problem with libcurl versus Civet which is based on mongoose.
I had the same problem and did run strace to attempt to debug. This is the output from strace (with a timestamp in seconds on each line). The issue is that there is a sequence of poll( ...events=POLLIN...) calls that block for a total of 1000 milliseconds (1+98+1+900) after the header is sent, before the body is sent. You can see the gap in timestamps from 17.935024 to 18.931842.
17.934881 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 517.934898 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)17.934916 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 017.934934 connect(5, {sa_family=AF_INET, sin_port=htons(37034), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)17.934952 poll([{fd=5, events=POLLOUT|POLLWRNORM}], 1, 0) = 1 ([{fd=5, revents=POLLOUT|POLLWRNORM}])17.934970 getsockopt(5, SOL_SOCKET, SO_ERROR, [0], [4]) = 017.934988 getpeername(5, {sa_family=AF_INET, sin_port=htons(37034), sin_addr=inet_addr("127.0.0.1")}, [16]) = 017.935006 getsockname(5, {sa_family=AF_INET, sin_port=htons(46830), sin_addr=inet_addr("127.0.0.1")}, [16]) = 017.935024 sendto(5, "POST /foo/1/bar HTTP/"..., 168, MSG_NOSIGNAL, NULL, 0) = 16817.935042 poll([{fd=5, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)17.935060 poll([{fd=5, events=POLLIN}], 1, 1) = 0 (Timeout)17.935078 poll([{fd=5, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)18.029421 poll([{fd=5, events=POLLIN}], 1, 98) = 0 (Timeout)18.029518 poll([{fd=5, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)18.030576 poll([{fd=5, events=POLLIN}], 1, 1) = 0 (Timeout)18.030681 poll([{fd=5, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)18.931571 poll([{fd=5, events=POLLIN}], 1, 900) = 0 (Timeout)18.931642 poll([{fd=5, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)18.931698 poll([{fd=5, events=POLLIN}, {fd=5, events=POLLOUT}], 2, 1000) = 1 ([{fd=5, revents=POLLOUT}])18.931726 poll([{fd=5, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}, {fd=5, events=POLLOUT|POLLWRNORM}], 2, 0) = 1 ([{fd=5, revents=POLLOUT|POLLWRNORM}])18.931842 sendto(5, "{\"foo\":[{\"bar\":\"A\",\"e"..., 1642, MSG_NOSIGNAL, NULL, 0) = 1642I had the same problem with both the command-line curl as well as with curl_easy_perform, but I did not have the problem using a different REST client such as Postman.
Thanks to @finnw's answer and the discussion here (https://groups.google.com/g/mongoose-users/c/92fD1Elk5m4?pli=1), I was able to patch CivetWeb to support the 100-continue feature with the following (see #ifdef DEXPECT_100_CONTINUE_FIX).
voidCivetServer::getPostData(struct mg_connection *conn, std::string& dst){ struct mg_request_info *ri = mg_get_request_info(conn); assert(ri != NULL); CivetServer *me = (CivetServer*) (ri->user_data); assert(me != NULL); mg_lock_context(me->context); CivetConnection &conobj = me->connections[conn]; mg_lock_connection(conn); mg_unlock_context(me->context);#ifdef EXPECT_100_CONTINUE_FIX const char * expect_str = mg_get_header(conn, "Expect"); if (expect_str) { printf("CivetServer::getParam, Expect:%s\n", expect_str); if (strcmp(expect_str,"100-continue") == 0) { mg_printf(conn, "%s", "HTTP/1.1 100 Continue\r\n\r\n"); } } else { printf("CivetServer::getParam, Expect: blank\n"); }#endif