You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Mahebub Sayyed <ma...@gmail.com> on 2016/09/21 08:01:34 UTC

How to write multiple outputs in avro format in spark(java)?

Hello,

Currently I am writing multiple text files based on keys.
Code :

public class RDDMultipleTextOutputFormat extends
MultipleTextOutputFormat<String, String> {

    @Override
    protected String generateFileNameForKeyValue(String key, String value,
            String name) {
        return key;
    }}

rddPairs.saveAsHadoopFile(args[1], String.class, String.class,
RDDMultipleTextOutputFormat.class);

How to write multiple output in avro format.

Also I have checked *"Avro Data Source for Apache Spark" *by *databricks*

Is it possible to write Multiple output using this library?

-- 
*Regards,*
*Mahebub Sayyed*