You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Naveen Kumar Pokala <np...@spcapitaliq.com> on 2014/11/17 11:33:44 UTC
HDFS read text file
Hi,
JavaRDD<Instrument> studentsData = sc.parallelize(list);--list is Student Info List<Student>
studentsData.saveAsTextFile("hdfs://master/data/spark/instruments.txt");
above statements saved the students information in the HDFS as a text file. Each object is a line in text file as below.
[cid:image001.png@01D0027F.FB321550]
How to read that file, I mean each line as Object of student.
-Naveen
Re: HDFS read text file
Posted by Hlib Mykhailenko <hl...@inria.fr>.
Hello Naveen,
I think you should first override "toString" method of your sample.spark.test.Student class.
--
Cordialement,
Hlib Mykhailenko
Doctorant à INRIA Sophia-Antipolis Méditerranée
2004 Route des Lucioles BP93
06902 SOPHIA ANTIPOLIS cedex
----- Original Message -----
> From: "Naveen Kumar Pokala" <np...@spcapitaliq.com>
> To: user@spark.apache.org
> Sent: Monday, November 17, 2014 11:33:44 AM
> Subject: HDFS read text file
> Hi,
> JavaRDD<Instrument> studentsData = sc .parallelize( list );--list is Student
> Info List<Student>
> studentsData .saveAsTextFile( "hdfs://master/data/spark/instruments.txt" );
> above statements saved the students information in the HDFS as a text file.
> Each object is a line in text file as below.
> How to read that file, I mean each line as Object of student.
> -Naveen
Re: HDFS read text file
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
You can use the sc.objectFile
<https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext>
to read it. It will be RDD[Student] type.
Thanks
Best Regards
On Mon, Nov 17, 2014 at 4:03 PM, Naveen Kumar Pokala <
npokala@spcapitaliq.com> wrote:
> Hi,
>
>
>
>
>
> JavaRDD<Instrument> studentsData = sc.parallelize(list);--list is Student
> Info List<Student>
>
>
>
> studentsData.saveAsTextFile("hdfs://master/data/spark/instruments.txt");
>
>
>
> above statements saved the students information in the HDFS as a text
> file. Each object is a line in text file as below.
>
>
>
>
>
> How to read that file, I mean each line as Object of student.
>
>
>
> -Naveen
>