distcp failing with error "No space left on device"
So I ran into this same problem and here is the what ultimately worked for me:
hadoop distcp -D mapreduce.job.maxtaskfailures.per.tracker=1 ...
I tried a few things (with the help of a colleagues) but the main thing that worked for me was - Changed max task failures per tracker to 1. This is mostly the key. Basically individual nodes were running out of space. So by doing this I am forcing the job not to retry on a node once it has failed on it already.
Other things that I tried but didn't work1. Increase number of mappers. (-m )2. Increased the number of retries from 3 to 12. (-D yarn.app.mapreduce.client.max-retries=12)