How to only barely use the bandwidth available to your program

As has been pointed out in the comments, you can't do this well enough to make any guarantees. But, suppose you want to do your best, anyways.

There are two parts to this problem:

1. Determine the bandwidth available.
2. Control the bandwidth being consumed.

Roughly controlling the bandwidth consumed can be accomplished in a user space program by limiting the rate at which you read from the socket. The TCP/IP stack will notify the other end of the connection that the queue is is maintaining on your application's behalf has gotten full, and nothing more will be sent. A convenient way of implementing that rate limiting is with token buckets.

Quick token bucket implementation:

 int bucket = 0; start_thread({ while(transfer_in_progress) {                    bucket += bytes_per_second_limit;                    sleep(1);                }); while(transfer_in_progress) {    bytesread = read(socket, buffer, min(bucket, buffersize), );    bucket -= bytesread; }

If bytes_per_second_limit is set to roughly the bandwidth available, expressed in bytes/second, then that should read as fast as the connection allows. If the connection is faster, you'll be limited to bytes_per_second_limit. If the connection is slower, then bucket will grow forever, at a rate proportional to the difference between the speed limit, and the available bandwidth.

Hmm!

If you run another thread, and keep an eye on bucket, you can watch for two conditions:

1. If bucket is always 0, then there is more bandwidth available, and you can increase bytes_per_second_limit, limited perhaps by your most recent best guess for available bandwidth (from #2). Or start an additional download.
2. If bucket is larger than the last time you looked, and the last few seconds of data points appear to indicate continuing growth (maybe do a linear regression; whatever you like), the rate of that growth expressed in bytes/second is how much you can reduce bytes_per_second_limit by in order to match your download rate with the available bandwidth.

The problem with all this is that there is no guarantee your bandwidth will stay constant. The thread monitoring bucket might bounce back and forth between increasing the rate, and limiting it. I'd suggest you start by averaging over at least 10 or 20 seconds before making rate limit changes.