Convert Dataframe to Dataset in pyspark

[Selvam Raman <se...@gmail.com>]

In Scala,
val ds = sqlContext.read.text("/home/spark/1.6/lines").as[String]

what is the equivalent code in pyspark?

-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Re: Convert Dataframe to Dataset in pyspark

[Michael Armbrust <mi...@databricks.com>]

You don't need encoders in python since its all dynamically typed anyway.
You can just do the following if you want the data as a string.

sqlContext.read.text("/home/spark/1.6/lines").rdd.map(lambda row: row.value)

2017-04-01 5:36 GMT-07:00 Selvam Raman <se...@gmail.com>:

> In Scala,
> val ds = sqlContext.read.text("/home/spark/1.6/lines").as[String]
>
> what is the equivalent code in pyspark?
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>