[jira] [Commented] (SPARK-28673) Failed to start executors for spark-sql-perf on kubernetes

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-28673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904920#comment-16904920 ] 

Hyukjin Kwon commented on SPARK-28673:
--------------------------------------

[~k82cn], please specify a full reproducer step. Otherwise no one know what issue it is. I am leaving this resolved until the details are provided.

> Failed to start executors for spark-sql-perf on kubernetes
> ----------------------------------------------------------
>
>                 Key: SPARK-28673
>                 URL: https://issues.apache.org/jira/browse/SPARK-28673
>             Project: Spark
>          Issue Type: Improvement
>          Components: Kubernetes, SQL
>    Affects Versions: 2.4.3
>            Reporter: Klaus Ma
>            Priority: Major
>
> Start Spark SQL Perf test with following command, the `spark.executor.instances` is 10, but no executors was start :(
>  
> {code:java}
> spark-submit \
> --master k8s://https://XXX \
> --deploy-mode cluster \
> --name k82cn-spark-test-per \
> --conf "spark.local.dir=/myfs/tmp" \
> --conf "spark.executor.extraJavaOptions=-Djava.io.tmpdir=/myfs/tmp" \
> --conf "spark.driver.extraJavaOptions=-Djava.io.tmpdir=/myfs/tmp" \
> --class org.apache.spark.sql.execution.benchmark.TPCDSQueryBenchmark \
> --conf spark.kubernetes.volcano.enable=false \
> --conf spark.driver.memory=3G \
> --conf spark.executor.instances=10 \
> --conf spark.kubernetes.container.image.pullSecrets=default-secret \
> --conf spark.kubernetes.container.image=XXX/spark-test:origin-4 \
> local:///opt/spark/jars/spark-sql_2.11-2.4.4-SNAPSHOT-tests.jar \
> --data-location file://XXXXX/spark-tpcds-data --query-filter "q2,q5"
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org