Web Api + HttpClient: An asynchronous module or handler completed while an asynchronous operation was still pending
Your problem is a subtle one: the async lambda you're passing to PushStreamContent is being interpreted as an async void (because the PushStreamContent constructor only takes Actions as parameters). So there's a race condition between your module/handler completing and the completion of that async void lambda.
PostStreamContent detects the stream closing and treats that as the end of its Task (completing the module/handler), so you just need to be sure there's no async void methods that could still run after the stream is closed. async Task methods are OK, so this should fix it:
private static PushStreamContent CopyContentStream(HttpResponseMessage sourceContent){ Func<Stream, Task> copyStreamAsync = async stream => { using (stream) using (var sourceStream = await sourceContent.Content.ReadAsStreamAsync()) { await sourceStream.CopyToAsync(stream); } }; var content = new PushStreamContent(stream => { var _ = copyStreamAsync(stream); }); return content;}If you want your proxies to scale a bit better, I also recommend getting rid of all the Result calls:
//Controller entry point.public async Task<HttpResponseMessage> PostAsync(){ using (var client = new HttpClient()) { var request = BuildRelayHttpRequest(this.Request); //HttpCompletionOption.ResponseHeadersRead - so that I can start streaming the response as soon //As it begins to filter in. var relayResult = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead); var returnMessage = BuildResponse(relayResult); return returnMessage; }}Your former code would block one thread for each request (until the headers are received); by using async all the way up to your controller level, you won't block a thread during that time.
I would like to add some wisdom for anyone else who landed here with the same error, but all of your code seems fine. Look for any lambda expressions passed into functions across the call-tree from where this occurs.
I was getting this error on a JavaScript JSON call to an MVC 5.x controller action. Everything I was doing up and down the stack was defined async Task and called using await.
However, using Visual Studio's "Set next statement" feature I systematically skipped over lines to determine which one caused it. I kept drilling down into local methods until I got to a call into an external NuGet package. The called method took an Action as a parameter and the lambda expression passed in for this Action was preceded by the async keyword. As Stephen Cleary points out above in his answer, this is treated as an async void, which MVC does not like. Luckily said package had *Async versions of the same methods. Switching to using those, along with some downstream calls to the same package fixed the problem.
I realize this is not a novel solution to the problem, but I passed over this thread a few times in my searches trying to resolve the issue because I thought I didn't have any async void or async <Action> calls, and I wanted to help someone else avoid that.
A slightly simpler model is that you can actually just use the HttpContents directly and pass them around inside the relay. I just uploaded a sample illustrating how you can rely both requests and responses asynchronously and without buffering the content in a relatively simple manner:
It is also beneficial to reuse the same HttpClient instance as this allows you to reuse connections where appropriate.