You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by George Obama <fj...@gmail.com> on 2017/03/29 18:58:21 UTC

Returning DataFrame for text file

Hi,

I saw that the API, either R or Scala, we are returning DataFrame for sparkSession.read.text()

What’s the rational behind this? 

Regards,
George

Re: Returning DataFrame for text file

Posted by Jacek Laskowski <ja...@japila.pl>.

Hi,

What's the alternative? Dataset? You've got textFile then.

It's an older API from the ages when Dataset was merely experimental.

Jacek

On 29 Mar 2017 8:58 p.m., "George Obama" <fj...@gmail.com> wrote:

> Hi,
>
> I saw that the API, either R or Scala, we are returning DataFrame for
> sparkSession.read.text()
>
> What’s the rational behind this?
>
> Regards,
> George
>

Re: Returning DataFrame for text file

Posted by "颜发才 (Yan Facai)" <fa...@gmail.com>.

SparkSession.read returns a DataFrameReader.
DataFrameReader supports a series of format, such as csv, json, text as you
mentioned.

check API to find more details:
+ http://spark.apache.org/docs/latest/api/scala/index.html#org
.apache.spark.sql.SparkSession
+ http://spark.apache.org/docs/latest/api/scala/index.html#org
.apache.spark.sql.DataFrameReader

On Thu, Mar 30, 2017 at 2:58 AM, George Obama <fj...@gmail.com> wrote:

> Hi,
>
> I saw that the API, either R or Scala, we are returning DataFrame for
> sparkSession.read.text()
>
> What’s the rational behind this?
>
> Regards,
> George
>