You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Karlson <ks...@siberie.de> on 2015/03/04 16:14:41 UTC

Save and read parquet from the same path

Hi all,

what would happen if I save a RDD via saveAsParquetFile to the same path 
that RDD is originally read from? Is that a safe thing to do in Pyspark?

Thanks!


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Save and read parquet from the same path

Posted by Michael Armbrust <mi...@databricks.com>.

No, this is not safe to do.

On Wed, Mar 4, 2015 at 7:14 AM, Karlson <ks...@siberie.de> wrote:

> Hi all,
>
> what would happen if I save a RDD via saveAsParquetFile to the same path
> that RDD is originally read from? Is that a safe thing to do in Pyspark?
>
> Thanks!
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>