You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by David Sabater Dinter <da...@gmail.com> on 2015/07/01 12:57:05 UTC

Issues when saving dataframe in Spark 1.4 with parquet format

Hi chaps,
It seems there is an issue while saving dataframes in Spark 1.4.

The default file extension inside Hive warehouse folder is now
part-r-X.gz.parquet but while running queries from SparkSQL Thriftserver is
still looking for part-r-X.parquet.

Is there any config parameter we can use as workaround? Is there any Jira
opened about the same? Am I missing anything if I upgraded from Spark 1.3
to 1.4?

The only similar reference I have seen is this:
http://mail-archives.apache.org/mod_mbox/spark-user/201506.mbox/%3CCAHP0wa+JApfVj+Pc2mzWOmzB+MmdOZfbR-XaXDBKOppe68tFww@mail.gmail.com%3E



Thanks.