[jira] [Commented] (FLINK-11427) Protobuf parquet writer implementation

["Jingsong Lee (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108072#comment-17108072 ] 

Jingsong Lee commented on FLINK-11427:
--------------------------------------

I'm not sure if we need to add this code. Because it will bring a lot of dependencies (parquet-proto) to the parquet project, which will not be used most of the time.

Considering the simplicity of the code, I think it's enough to just add documentation. 
{code:java}
public static <T extends Message> ParquetWriterFactory create(Class<T> type) {
   class Builder extends ParquetWriter.Builder<T, Builder> {
      private Builder(OutputFile path) {
         super(path);
      }

      @Override
      protected Builder self() {
         return this;
      }

      @Override
      protected WriteSupport<T> getWriteSupport(Configuration conf) {
         return new ProtoWriteSupport<>(type);
      }
   }
   return new ParquetWriterFactory<>(out -> new Builder(out).build());
}{code}
Just add this example to documentation.

> Protobuf parquet writer implementation
> --------------------------------------
>
>                 Key: FLINK-11427
>                 URL: https://issues.apache.org/jira/browse/FLINK-11427
>             Project: Flink
>          Issue Type: Improvement
>          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>            Reporter: Guang Hu
>            Assignee: Yun Gao
>            Priority: Trivial
>              Labels: pull-request-available, usability
>             Fix For: 1.11.0
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Right now only ParquetAvroWriters exist to create ParquetWriterFactory. We want to implement a protobuf ParquetProtoWriters to create ParquetWriterFactory.  I am happy to submit a PR if this approach sounds good . 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)