Random 'Connection reset' error in build pipelines using Maven on Kubernetes runners

We're seeing a random 'connection reset' error on our CI Pipelines building Java apps with Maven. The error occurs randomly, if you rerun a job that has the error it will eventually run to completion with no errors.

When the error occurs, re-running the job may see the same error but downloading a different dependency, or the job may complete with no errors.

Example of error:

Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact com.google.cloud:google-cloud-bigquerystorage-bom:pom:3.2.0 from/to central (https://repo.maven.apache.org/maven2): Connection reset

Example of recent jobs with the error:

On some pipelines the build job output appears to get truncated but fails randomly without showing the 'Connection Reset' error in the log, but we're assuming these are failing for the same reason too as they are also random, and if you re-run they will complete eventually:

https://gitlab.developers.cam.ac.uk/uis/devops/hr/tes/tes-webapp/-/jobs/2126501

From research, this question describes the same symptoms we are seeing, and the linked article describes a potential issue with invalid packets running Docker on Kubernetes causing the issue, and a suggestion for a iptables config change to drop the invalid packages that avoids the issue

Edited Feb 20, 2025 by Kevin Hooke