You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2020/03/27 16:15:00 UTC
[jira] [Resolved] (SPARK-31200) Docker image build fails with
Mirror sync in progress? errors.
[ https://issues.apache.org/jira/browse/SPARK-31200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-31200.
-----------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 27966
[https://github.com/apache/spark/pull/27966]
> Docker image build fails with Mirror sync in progress? errors.
> --------------------------------------------------------------
>
> Key: SPARK-31200
> URL: https://issues.apache.org/jira/browse/SPARK-31200
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 3.1.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Fix For: 3.0.0
>
>
> Following errors would appear, in spite of trying to switch between various mirrors, while building spark docker image.
> {code}
> bash-3.2$ bin/docker-image-tool.sh -r scrapcodes -t v3.1.0-f1cc86 build
> Sending build context to Docker daemon 203.4MB
> Step 1/18 : ARG java_image_tag=8-jre-slim
> Step 2/18 : FROM openjdk:${java_image_tag}
> ---> 381b20190cf7
> Step 3/18 : ARG spark_uid=185
> ---> Using cache
> ---> 65c06f86753c
> Step 4/18 : RUN set -ex && apt-get update && ln -s /lib /lib64 && apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && mkdir -p /opt/spark && mkdir -p /opt/spark/examples && mkdir -p /opt/spark/work-dir && touch /opt/spark/RELEASE && rm /bin/sh && ln -sv /bin/bash /bin/sh && echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && chgrp root /etc/passwd && chmod ug+rw /etc/passwd && rm -rf /var/cache/apt/*
> ---> Running in 96bcbe927d35
> + apt-get update
> Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
> Get:2 http://deb.debian.org/debian buster-updates InRelease [49.3 kB]
> Get:3 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
> Err:3 http://deb.debian.org/debian buster/main amd64 Packages
> File has unexpected size (13217 != 7906744). Mirror sync in progress? [IP: 151.101.10.133 80]
> Hashes of expected file:
> - Filesize:7906744 [weak]
> - SHA256:80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc
> - MD5Sum:80b6d9c1b6630b2234161e42f4040ab3 [weak]
> Release file created at: Sat, 08 Feb 2020 10:57:10 +0000
> Get:5 http://deb.debian.org/debian buster-updates/main amd64 Packages [7380 B]
> Err:5 http://deb.debian.org/debian buster-updates/main amd64 Packages
> File has unexpected size (13233 != 7380). Mirror sync in progress? [IP: 151.101.10.133 80]
> Hashes of expected file:
> - Filesize:7380 [weak]
> - SHA256:6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d
> Release file created at: Fri, 20 Mar 2020 02:28:11 +0000
> Get:4 http://security-cdn.debian.org/debian-security buster/updates InRelease [65.4 kB]
> Get:6 http://security-cdn.debian.org/debian-security buster/updates/main amd64 Packages [183 kB]
> Fetched 419 kB in 1s (327 kB/s)
> Reading package lists...
> E: Failed to fetch http://deb.debian.org/debian/dists/buster/main/binary-amd64/by-hash/SHA256/80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc File has unexpected size (13217 != 7906744). Mirror sync in progress? [IP: 151.101.10.133 80]
> Hashes of expected file:
> - Filesize:7906744 [weak]
> - SHA256:80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc
> - MD5Sum:80b6d9c1b6630b2234161e42f4040ab3 [weak]
> Release file created at: Sat, 08 Feb 2020 10:57:10 +0000
> E: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/main/binary-amd64/by-hash/SHA256/6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d File has unexpected size (13233 != 7380). Mirror sync in progress? [IP: 151.101.10.133 80]
> Hashes of expected file:
> - Filesize:7380 [weak]
> - SHA256:6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d
> Release file created at: Fri, 20 Mar 2020 02:28:11 +0000
> E: Some index files failed to download. They have been ignored, or old ones used instead.
> The command '/bin/sh -c set -ex && apt-get update && ln -s /lib /lib64 && apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps && mkdir -p /opt/spark && mkdir -p /opt/spark/examples && mkdir -p /opt/spark/work-dir && touch /opt/spark/RELEASE && rm /bin/sh && ln -sv /bin/bash /bin/sh && echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && chgrp root /etc/passwd && chmod ug+rw /etc/passwd && rm -rf /var/cache/apt/*' returned a non-zero code: 100
> Failed to build Spark JVM Docker image, please refer to Docker build output for details.
> {code}
> Soon, I found that, it was due to my ISP (Since we are working from home now !) trying to intercept http traffic. So it may not happen to others. But, just in case, if some is hit by such errors. Changing mirrors to https from http helped resolve it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org