[jira] [Created] (BEAM-881) Provide a PTransform in IOs providing a "standard" Avro IndexedRecord

["Jean-Baptiste Onofré (JIRA)" <ji...@apache.org>]

Jean-Baptiste Onofré created BEAM-881:
-----------------------------------------

             Summary: Provide a PTransform in IOs providing a "standard" Avro IndexedRecord
                 Key: BEAM-881
                 URL: https://issues.apache.org/jira/browse/BEAM-881
             Project: Beam
          Issue Type: New Feature
          Components: sdk-java-extensions
            Reporter: Jean-Baptiste Onofré
            Assignee: Jean-Baptiste Onofré


Now, each IO is using a different data format. For instance, the {{JmsIO.Read}} provides a {{PCollection}} of {{JmsRecord}} (and {{JmsIO.Write}} expects also a {{JmsRecord}}), {{KafkaIO.Read}} provides a {{PCollection}} of {{KafkaRecord}}.

It could appear a bit "complex" for users to manipulate such kind of data format: some users may expect kind of standard format.

Without modifying the existing IO, we could add a {{PTransform}} (as part of the IO) that an user can optionally use. This transform will convert the IO data format (let say {{JmsRecord}} for instance) to a standard Avro {{IndexedRecord}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)