You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Daniel Haviv <da...@gmail.com> on 2014/11/20 05:54:41 UTC

insertIntoTable failure deleted pre-existing _metadata file

Hello,
I'm loading and saving json files into an existing directory with parquet files using the insertIntoTable method.
If the method fails for some reason (differences in the schema in my case), the _metadata file of the parquet dir is automatically deleted, rendering the existing parquet files useless.

Looks like a bug to me.

Daniel