You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by SRK <sw...@gmail.com> on 2016/02/29 03:09:30 UTC

Error when trying to overwrite a partition dynamically in Spark SQL

Hi,

I am getting an error when trying to overwrite a partition dynamically.
Following is the code and the error. Any idea as to why this is happening?



test.write.partitionBy("dtPtn","idPtn").mode(SaveMode.Overwrite).format("parquet").save("/user/test/sessRecs")

16/02/28 18:02:55 ERROR input.FileInputFormat: Exception while trying to
createSplits
java.util.concurrent.ExecutionException:
java.lang.ArrayIndexOutOfBoundsException: -1
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:188)
	at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:337)



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Error-when-trying-to-overwrite-a-partition-dynamically-in-Spark-SQL-tp26355.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org