You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Mahebub Sayyed <ma...@gmail.com> on 2016/09/21 08:01:34 UTC
How to write multiple outputs in avro format in spark(java)?
Hello,
Currently I am writing multiple text files based on keys.
Code :
public class RDDMultipleTextOutputFormat extends
MultipleTextOutputFormat<String, String> {
@Override
protected String generateFileNameForKeyValue(String key, String value,
String name) {
return key;
}}
rddPairs.saveAsHadoopFile(args[1], String.class, String.class,
RDDMultipleTextOutputFormat.class);
How to write multiple output in avro format.
Also I have checked *"Avro Data Source for Apache Spark" *by *databricks*
Is it possible to write Multiple output using this library?
--
*Regards,*
*Mahebub Sayyed*