You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by pseudo oduesp <ps...@gmail.com> on 2016/06/16 14:43:02 UTC

difference between dataframe and dataframwrite

hi,

what is difference between dataframe and dataframwrite ?

Re: difference between dataframe and dataframwrite

Posted by Richard Catlin <ri...@gmail.com>.

I believe it depends on your Spark application.

To write to Hive, use 
dataframe.saveAsTable

To write to S3, use
dataframe.write.parquet(“s3://<bucket>”)

Hope this helps.
Richard

> On Jun 16, 2016, at 9:54 AM, Natu Lauchande <nl...@gmail.com> wrote:
> 
> Does

RE: difference between dataframe and dataframwrite

Posted by Natu Lauchande <nl...@gmail.com>.

Hi

Does anyone know wich one aws emr uses by default?

Thanks,
Natu
On Jun 16, 2016 5:12 PM, "David Newberger" <da...@wandcorp.com>
wrote:

> DataFrame is a collection of data which is organized into named columns.
>
> DataFrame.write is an interface for saving the contents of a DataFrame to
> external storage.
>
>
>
> Hope this helps
>
>
>
> *David Newberger*
>
>
>
>
>
> *From:* pseudo oduesp [mailto:pseudo20140@gmail.com]
> *Sent:* Thursday, June 16, 2016 9:43 AM
> *To:* user@spark.apache.org
> *Subject:* difference between dataframe and dataframwrite
>
>
>
> hi,
>
>
>
> what is difference between dataframe and dataframwrite ?
>
>
>

RE: difference between dataframe and dataframwrite

Posted by David Newberger <da...@wandcorp.com>.

DataFrame is a collection of data which is organized into named columns.
DataFrame.write is an interface for saving the contents of a DataFrame to external storage.

Hope this helps

David Newberger


From: pseudo oduesp [mailto:pseudo20140@gmail.com]
Sent: Thursday, June 16, 2016 9:43 AM
To: user@spark.apache.org
Subject: difference between dataframe and dataframwrite

hi,

what is difference between dataframe and dataframwrite ?