You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Divya Gehlot <di...@gmail.com> on 2016/02/23 04:34:55 UTC

[Example] : Save dataframes with different schema + Spark 1.5.2 and Dataframe + Spark-CSV package

Hi,
My usecase :
Have  two datsets1 like below :
year make model comment blank Carname
2012 Tesla S No comment
1997 Ford E350 Go get one
 now they are going fast MyFord
2015 Chevy Volt
2016 Mercedes

Datset2
carowner year make model
John 2012 Tesla S
David
Peter 1997 Ford E350
Paul 2015 Chevy Volt

My output should be like
carowner year make model comment blank Carname
John 2012 Tesla S No comment null
David null
Peter 1997 Ford E350 Go get one
 now they are going fast MyFord
Paul 2015 Chevy Volt null
null 2016 Mercedes null null null

How can I achieve this using dataframe and spark CSV packages .
Would really appreciate the help.


Thanks,
Divya