[jira] [Created] (SPARK-34900) Some `spark-submit`  commands used to run benchmarks in the user's guide is wrong

["Yang Jie (Jira)" <ji...@apache.org>]

Yang Jie created SPARK-34900:
--------------------------------

             Summary: Some `spark-submit`  commands used to run benchmarks in the user's guide is wrong
                 Key: SPARK-34900
                 URL: https://issues.apache.org/jira/browse/SPARK-34900
             Project: Spark
          Issue Type: Bug
          Components: Tests
    Affects Versions: 3.2.0
            Reporter: Yang Jie


For example, the guide for running JoinBenchmark as follows:

 
{code:java}
/**
 * Benchmark to measure performance for joins.
 * To run this benchmark:
 * {{{
 *   1. without sbt:
 *      bin/spark-submit --class <this class> --jars <spark core test jar> <spark sql test jar>
 *   2. build/sbt "sql/test:runMain <this class>"
 *   3. generate result:
 *      SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain <this class>"
 *      Results will be written to "benchmarks/JoinBenchmark-results.txt".
 * }}}
 */
object JoinBenchmark extends SqlBasedBenchmark {
{code}
 

 

but if we run JoinBenchmark with commnad

 
{code:java}
bin/spark-submit --class org.apache.spark.sql.execution.benchmark.JoinBenchmark --jars spark-core_2.12-3.2.0-SNAPSHOT-tests.jar spark-sql_2.12-3.2.0-SNAPSHOT-tests.jar 
{code}
 

The following exception will be thrown：

 
{code:java}
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/sql/catalyst/plans/SQLHelper
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369){code}
 

because SqlBasedBenchmark trait extends BenchmarkBase and SQLHelper, SQLHelper def in spark-catalyst-tests.jar.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org