Maven build gets connection reset when downloading artifacts
TL;DR
Try upgrading to Maven 3.6.1. It has a new version of Wagon HTTP with improved retry behavior. This fixed the problem for us.
Background
Our builds ran fine for months (in ec2). Then one day they started sporadically failing downloading Maven artifacts (from Maven central) with Connection reset. We never got to the bottom of it. Maybe something in ec2 changed. Maybe something in our build changed. Dunno.
The bottom line is that networks are unreliable, and Maven (or more specifically Wagon) was not very resilient to connection failures. Thankfully it looks like they've improved that.
Maven 3.6.1 includes a new version of Wagon that supports configureable retry behavior. By default it appears to have a retry count of 3 -- which was good enough for us. Some details:
- Wagon Configuration properties: describes how to configure retry behavior.
- WAGON-526: the issue requesting configurable retry handling
- The PR for WAGON-526: from this you can inspect the source and see the default values for the configuration properites
Few Pointers,
1) This looks to be EC2 network setting problem. It's not a maven http wagon problem.
2) Look at this similar problem. More specifically this.
3) You can temporarily try following.
sudo sysctl -w net.ipv4.tcp_keepalive_time=60sudo sysctl -w net.ipv4.tcp_keepalive_intvl=60sudo sysctl -w net.ipv4.tcp_keepalive_probes=10
Hope it helps you in solving your problem.
We have a similar issue in Azure VM. I posted this question here as well: Maven build download artefacts connection slow/reset when running inside a Docker Container on Azure
Our setup is a bit more complex - we run a docker container that includes a build step for maven.
Both when running the Container in ACI and in an Azure VM - we encounter the same issue, the maven central downloads timeout or succeeded but at very slow speeds.
14:10:48,505 [BasicRepositoryConnector-repo.maven.apache.org-27-0] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/regexp/regexp/1.3/regexp-1.3.jar14:10:48,506 [BasicRepositoryConnector-repo.maven.apache.org-27-2] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/scm/maven-scm-provider-cvs-commons/1.7/maven-scm-provider-cvs-commons-1.7.jar14:10:48,505 [BasicRepositoryConnector-repo.maven.apache.org-27-1] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/scm/maven-scm-provider-git-commons/1.7/maven-scm-provider-git-commons-1.7.jar14:10:48,521 [BasicRepositoryConnector-repo.maven.apache.org-27-3] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/tmatesoft/sqljet/sqljet/1.0.4/sqljet-1.0.4.jar14:10:48,523 [BasicRepositoryConnector-repo.maven.apache.org-27-4] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/antlr/antlr-runtime/3.1.3/antlr-runtime-3.1.3.jar14:10:48,540 [BasicRepositoryConnector-repo.maven.apache.org-27-0] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/regexp/regexp/1.3/regexp-1.3.jar (25 kB at 706 kB/s)14:10:48,540 [BasicRepositoryConnector-repo.maven.apache.org-27-0] [INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/antlr/stringtemplate/3.2/stringtemplate-3.2.jar14:10:48,564 [BasicRepositoryConnector-repo.maven.apache.org-27-0] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/antlr/stringtemplate/3.2/stringtemplate-3.2.jar (172 kB at 4.0 MB/s)14:26:32,150 [BasicRepositoryConnector-repo.maven.apache.org-27-2] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/scm/maven-scm-provider-cvs-commons/1.7/maven-scm-provider-cvs-commons-1.7.jar (80 kB at 84 B/s)14:26:32,157 [BasicRepositoryConnector-repo.maven.apache.org-27-4] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/antlr/antlr-runtime/3.1.3/antlr-runtime-3.1.3.jar (151 kB at 159 B/s)14:26:32,199 [BasicRepositoryConnector-repo.maven.apache.org-27-3] [INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/tmatesoft/sqljet/sqljet/1.0.4/sqljet-1.0.4.jar (744 kB at 788 B/s)
Notice the suspicious: (151 kB at 159 B/s), (80 kB at 84 B/s), (744 kB at 788 B/s)We have examples of executions that run just fine - and examples of executions that timeout (1 hour) - and examples of executions that take close to 1h.
We have attempted to alter the mvn command - we initially ran:
mvn clean package
but then we added:
mvn compile dependency:resolve
mvn dependency:resolve-plugins
mvn clean package
These changes only altered the place where the connection slows down - instead of downloading some jars, it slowed down at dowloading .pom files.