You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2020/12/05 08:01:00 UTC
[jira] [Assigned] (SPARK-33668) Fix flaky test "Verify logging
configuration is picked from the provided SPARK_CONF_DIR/log4j.properties."
[ https://issues.apache.org/jira/browse/SPARK-33668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-33668:
------------------------------------
Assignee: (was: Apache Spark)
> Fix flaky test "Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties."
> ----------------------------------------------------------------------------------------------------------
>
> Key: SPARK-33668
> URL: https://issues.apache.org/jira/browse/SPARK-33668
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes, Tests
> Affects Versions: 3.1.0
> Reporter: Prashant Sharma
> Priority: Major
>
> The test is flaking, with multiple flaked instances - the reason for the failure has been similar to:
> {code:java}
> The code passed to eventually never returned normally. Attempted 109 times over 3.0079882413999997 minutes. Last failure message: Failure executing: GET at: https://192.168.39.167:8443/api/v1/namespaces/b37fc72a991b49baa68a2eaaa1516463/pods/spark-pi-97a9bc76308e7fe3-exec-1/log?pretty=false. Message: pods "spark-pi-97a9bc76308e7fe3-exec-1" not found. Received status: Status(apiVersion=v1, code=404, details=StatusDetails(causes=[], group=null, kind=pods, name=spark-pi-97a9bc76308e7fe3-exec-1, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "spark-pi-97a9bc76308e7fe3-exec-1" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=NotFound, status=Failure, additionalProperties={}).. (KubernetesSuite.scala:402)
> {code}
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36854/console
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36852/console
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36850/console
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36848/console
> From the above failures, it seems, that executor finishes too quickly and is removed by spark before the test can complete.
> So, in order to mitigate this situation, one way is to turn on the flag
> {code}
> "spark.kubernetes.executor.deleteOnTermination"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org