You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2019/10/02 04:06:00 UTC
[jira] [Assigned] (SPARK-29313) Failure on writing to `noop`
datasource in benchmarks
[ https://issues.apache.org/jira/browse/SPARK-29313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun reassigned SPARK-29313:
-------------------------------------
Assignee: Maxim Gekk
> Failure on writing to `noop` datasource in benchmarks
> -----------------------------------------------------
>
> Key: SPARK-29313
> URL: https://issues.apache.org/jira/browse/SPARK-29313
> Project: Spark
> Issue Type: Test
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Minor
>
> This failure happens on overwriting results of ExtractBenchmark via command:
> {code}
> SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.ExtractBenchmark"
> {code}
> with the exception:
> {code}
> [info] Running benchmark: Invoke extract for timestamp
> [info] Running case: cast to timestamp
> [error] Exception in thread "main" org.apache.spark.sql.AnalysisException: TableProvider implementation noop cannot be written with ErrorIfExists mode, please use Append or Overwrite modes instead.;
> [error] at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:284)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$doBenchmark$1(ExtractBenchmark.scala:46)
> [error] at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:47)
> [error] at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:31)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.withSQLConf(ExtractBenchmark.scala:37)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.doBenchmark(ExtractBenchmark.scala:47)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$run$1(ExtractBenchmark.scala:57)
> [error] at org.apache.spark.benchmark.Benchmark.$anonfun$addCase$1(Benchmark.scala:77)
> [error] at org.apache.spark.benchmark.Benchmark.$anonfun$addCase$1$adapted(Benchmark.scala:75)
> [error] at org.apache.spark.benchmark.Benchmark.measure(Benchmark.scala:139)
> [error] at org.apache.spark.benchmark.Benchmark.$anonfun$run$1(Benchmark.scala:106)
> [error] at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
> [error] at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
> [error] at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
> [error] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
> [error] at scala.collection.TraversableLike.map(TraversableLike.scala:238)
> [error] at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
> [error] at scala.collection.AbstractTraversable.map(Traversable.scala:108)
> [error] at org.apache.spark.benchmark.Benchmark.run(Benchmark.scala:104)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$2(ExtractBenchmark.scala:101)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$2$adapted(ExtractBenchmark.scala:95)
> [error] at scala.collection.immutable.List.foreach(List.scala:392)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$1(ExtractBenchmark.scala:95)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$1$adapted(ExtractBenchmark.scala:94)
> [error] at scala.collection.immutable.List.foreach(List.scala:392)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.runBenchmarkSuite(ExtractBenchmark.scala:94)
> [error] at org.apache.spark.benchmark.BenchmarkBase.main(BenchmarkBase.scala:57)
> [error] at org.apache.spark.sql.execution.benchmark.ExtractBenchmark.main(ExtractBenchmark.scala)
> java.lang.RuntimeException: Nonzero exit code returned from runner: 1
> at scala.sys.package$.error(package.scala:27)
> [trace] Stack trace suppressed: run last sql/test:runMain for the full output.
> [error] (sql/test:runMain) Nonzero exit code returned from runner: 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org