You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/22 07:51:56 UTC

[GitHub] [spark] Yikun opened a new pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Yikun opened a new pull request #31923:
URL: https://github.com/apache/spark/pull/31923


   ### What changes were proposed in this pull request?
   We added the gnupg installation in https://github.com/apache/spark/pull/30130 , we should do apt update before gnupg isntallation, otherwise we will get a fetch error when package is updated.
   
   
   ### Why are the changes needed?
   add a apt-update cmd before gnupg installation to avoid invaild package cache list.
   
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   
   ### How was this patch tested?
   K8s Integration test passed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804089727


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136345/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Yikun commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804017917


   ```shell
   Step 6/12 : RUN   apt-get update &&   apt install -y gnupg &&   echo "deb http://cloud.r-project.org/bin/linux/debian buster-cran35/" >> /etc/apt/sources.list &&   (apt-key adv --keyserver keys.gnupg.net --recv-key 'E19F5F87128899B192B1A2C2AD5F960A256A04AF' || apt-key adv --keyserver keys.openpgp.org --recv-key 'E19F5F87128899B192B1A2C2AD5F960A256A04AF') &&   apt-get update &&   apt install -y -t buster-cran35 r-base r-base-dev &&   rm -rf /var/cache/apt/*
    ---> Running in 890d910fdc12
   ... ...
   
    ---> 6fc29805545e
   Step 7/12 : COPY R ${SPARK_HOME}/R
    ---> 98781dd121aa
   Step 8/12 : ENV R_HOME /usr/lib/R
   ```
   
   From [1] (Step 6/12 succeed), we can see the failed fetch problem of [SPARK-34820](https://issues.apache.org/jira/browse/SPARK-34820) had been fixed.
   
   [1] https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40920/consoleFull


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803941652


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] shaneknapp commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
shaneknapp commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804161577


   thanks for doing this!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Ngone51 commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Ngone51 commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-806304063


   Hi all, I'd like to get your continued attention on the K8s integration test issue. After this PR fixed the gnupg installation issue, another issue shows up, which is almost a constant failure:
   
   ```shell
   - Run SparkR on simple dataframe.R example *** FAILED ***
     io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://192.168.39.219:8443/api/v1/namespaces/b80cb1250aba4d92a99fc87b609b2328/pods/spark-test-app-411d038edc8b4e9b8e3761052cd44bd8/log?pretty=false. Message: container "spark-kubernetes-driver" in pod "spark-test-app-411d038edc8b4e9b8e3761052cd44bd8" is waiting to start: trying and failing to pull image. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=container "spark-kubernetes-driver" in pod "spark-test-app-411d038edc8b4e9b8e3761052cd44bd8" is waiting to start: trying and failing to pull image, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=BadRequest, status=Failure, additionalProperties={}).
     at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:570)
     at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:509)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.doGetLog(PodOperationsImpl.java:189)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.getLog(PodOperationsImpl.java:198)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.getLog(PodOperationsImpl.java:85)
     at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.$anonfun$logForFailedTest$3(KubernetesSuite.scala:89)
     at org.apache.spark.internal.Logging.logInfo(Logging.scala:57)
     at org.apache.spark.internal.Logging.logInfo$(Logging.scala:56)
     at org.apache.spark.SparkFunSuite.logInfo(SparkFunSuite.scala:61)
     at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.$anonfun$logForFailedTest$2(KubernetesSuite.scala:86)
   ```
   
   And here is [PR](https://github.com/apache/spark/pull/31102) that has tried K8s integration test multiple times but all failed. 
   Could someone please help take a look? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803918760


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40920/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Yikun commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803851097


   cc @Ngone51


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Ngone51 commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Ngone51 commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-809346172


   Thanks for the effort @attilapiros  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804066208


   **[Test build #136345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136345/testReport)** for PR 31923 at commit [`7418fdb`](https://github.com/apache/spark/commit/7418fdb5059b7dc7f94d0b5eef5635c48f6d36e7).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803864620


   **[Test build #136335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136335/testReport)** for PR 31923 at commit [`7418fdb`](https://github.com/apache/spark/commit/7418fdb5059b7dc7f94d0b5eef5635c48f6d36e7).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803854171


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803855821


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Ngone51 commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Ngone51 commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804054259


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803864898


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136335/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Ngone51 commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Ngone51 commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804054496


   Let's retry test see if it's flaky


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803941652


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804120617


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40929/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] attilapiros commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-807006483


   I hope this will help to troubleshoot this and similar errors:
   https://github.com/apache/spark/pull/31962
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804135700


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40929/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804056880


   **[Test build #136345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136345/testReport)** for PR 31923 at commit [`7418fdb`](https://github.com/apache/spark/commit/7418fdb5059b7dc7f94d0b5eef5635c48f6d36e7).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] attilapiros commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-806706204


   It would be wonderful to see  `kubectl describe pod <pod>` output. 
   Actually I think I can do this in one of my PR...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803855382


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803891779


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40920/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun closed pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #31923:
URL: https://github.com/apache/spark/pull/31923


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804135700


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40929/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804096007


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40929/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] attilapiros commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-806885852


   Of course locally all the tests passed:
   ```
   All tests passed.
   ...
   [INFO] Spark Project Kubernetes Integration Tests ......... SUCCESS [25:12 min]
   [INFO] ------------------------------------------------------------------------
   [INFO] BUILD SUCCESS
   [INFO] ------------------------------------------------------------------------
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803854171


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803864898






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Yikun commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804130717


   ![image](https://user-images.githubusercontent.com/1736354/112010075-f006d400-8b61-11eb-97c0-d48851d09ce4.png)
   
   The SparkPullRequestBuilder-K8s (Kubernetes integration test) back to green again, I think the PR is ready to merge.
   
   [1] https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] shaneknapp commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
shaneknapp commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-809801068


   fwiw the k8s and minikube installs on the workers haven't changed...  i
   went through earlier today and ensured that we didn't have any dangling
   minikube processes on the workers, and killed any leftover build procs as
   well.  i'm not sure why this is happening...
   
   also, we'll be upgrading both k8s and mk in the next week or so.
   
   On Mon, Mar 29, 2021 at 5:43 AM wuyi ***@***.***> wrote:
   
   > Thanks for the effort @attilapiros <https://github.com/attilapiros>
   >
   > —
   > You are receiving this because your review was requested.
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/spark/pull/31923#issuecomment-809346172>, or
   > unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/AAMIHLCLOTD75VQB646XATTTGBYXHANCNFSM4ZSS2MKQ>
   > .
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804056880


   **[Test build #136345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136345/testReport)** for PR 31923 at commit [`7418fdb`](https://github.com/apache/spark/commit/7418fdb5059b7dc7f94d0b5eef5635c48f6d36e7).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803855605


   **[Test build #136335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136335/testReport)** for PR 31923 at commit [`7418fdb`](https://github.com/apache/spark/commit/7418fdb5059b7dc7f94d0b5eef5635c48f6d36e7).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] attilapiros commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-806654897


   @Ngone51 I checked the PR you mentioned. 
   
   My findings based on [the last failure](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41014/consoleFull). 
   
   Here the first error is:
   
   ```
   - Launcher client dependencies
   - SPARK-33615: Launcher client archives *** FAILED ***
     io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://192.168.39.147:8443/api/v1/namespaces/09e9c94160d543c1a338f364722d49a6/pods/spark-test-app-c80c78f512574b36b2608f7d92c24503/log?pretty=false. Message: container "spark-kubernetes-driver" in pod "spark-test-app-c80c78f512574b36b2608f7d92c24503" is waiting to start: trying and failing to pull image. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=container "spark-kubernetes-driver" in pod "spark-test-app-c80c78f512574b36b2608f7d92c24503" is waiting to start: trying and failing to pull image, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=BadRequest, status=Failure, additionalProperties={}).
     at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:570)
     at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:509)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.doGetLog(PodOperationsImpl.java:189)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.getLog(PodOperationsImpl.java:198)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.getLog(PodOperationsImpl.java:85)
     at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.$anonfun$logForFailedTest$3(KubernetesSuite.scala:89)
     at org.apache.spark.internal.Logging.logInfo(Logging.scala:57)
     at org.apache.spark.internal.Logging.logInfo$(Logging.scala:56)
     at org.apache.spark.SparkFunSuite.logInfo(SparkFunSuite.scala:61)
     at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.$anonfun$logForFailedTest$2(KubernetesSuite.scala:86)
     ...
   ```
   
   Checking the last successful and first failed ones we can see just a very few differences in the code: the failed one uses a `--archives` and the successful uses `--files`.
   
   I do not think this difference could lead to an error such "is waiting to start: trying and failing to pull image."
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803855382


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Ngone51 commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Ngone51 commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803854699


   ok to test


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803855605


   **[Test build #136335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136335/testReport)** for PR 31923 at commit [`7418fdb`](https://github.com/apache/spark/commit/7418fdb5059b7dc7f94d0b5eef5635c48f6d36e7).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803937743


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40920/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804089727


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136345/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Ngone51 commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Ngone51 commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803852696


   @Yikun thanks for the quick fix. cc @dongjoon-hyun @holdenk 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-803855821


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Yikun commented on pull request #31923: [SPARK-34820][K8S][R] add apt-update before gnupg install

Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #31923:
URL: https://github.com/apache/spark/pull/31923#issuecomment-804023215


   ```
   - Run SparkR on simple dataframe.R example *** FAILED ***
     io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://192.168.39.219:8443/api/v1/namespaces/5b12a03f6e2c40308f8c12bf4d9ea3e3/pods/spark-test-app-f99d40879bc94e569c8ae32b79a88970/log?pretty=false. Message: container "spark-kubernetes-driver" in pod "spark-test-app-f99d40879bc94e569c8ae32b79a88970" is waiting to start: trying and failing to pull image. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=container "spark-kubernetes-driver" in pod "spark-test-app-f99d40879bc94e569c8ae32b79a88970" is waiting to start: trying and failing to pull image, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=BadRequest, status=Failure, additionalProperties={}).
     at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:570)
     at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:509)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.doGetLog(PodOperationsImpl.java:189)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.getLog(PodOperationsImpl.java:198)
     at io.fabric8.kubernetes.client.dsl.internal.core.v1.PodOperationsImpl.getLog(PodOperationsImpl.java:85)
     at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.$anonfun$logForFailedTest$3(KubernetesSuite.scala:89)
     at org.apache.spark.internal.Logging.logInfo(Logging.scala:57)
     at org.apache.spark.internal.Logging.logInfo$(Logging.scala:56)
     at org.apache.spark.SparkFunSuite.logInfo(SparkFunSuite.scala:61)
     at org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.$anonfun$logForFailedTest$2(KubernetesSuite.scala:86)
     ...
   ```
   
   and there are still some other errors triggerd, looks like unrelated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org