distcp hdfs to s3 fails
I tried the suggested answer, but with no luck. I experienced the issue when copying many small files (in the order of thousands, which in total did not account for more than half a gigabyte). I couldn't make distcp command work (same error as posted by OP), so switching to hadoop fs -cp was my solution. As a side note, in the same cluster, using distcp for copying other, much larger files worked ok.