You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/05/25 12:41:46 UTC

[GitHub] [flink] gaoyunhaii opened a new pull request #12321: Add document for writing Avro files with StreamingFileSink

gaoyunhaii opened a new pull request #12321:
URL: https://github.com/apache/flink/pull/12321


   ## What is the purpose of the change
   
   This PR add documents for the newly added `Avro` writers introduced in [Flink-11395](https://issues.apache.org/jira/browse/FLINK-11395).
   
   ## Brief change log
   
   - 499e54cffce1ed49287fa5688d228279007282b9 fixes the existing html error in the streaming file sink doc which makes readers unable to switch between `Java` and `Scala` codes.
   - 56bdc3a61f65cb30b48cf7f932520be02ebed734 adds the docs for the `Avro` writers.
   
   
   ## Verifying this change
   
   This change is a trivial work without any test coverage.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): **no**
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
     - The serializers: **no**
     - The runtime per-record code paths (performance sensitive): **no**
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: **no**
     - The S3 file system connector:**no**
   
   ## Documentation
   
     - Does this pull request introduce a new feature? **no**
     - If yes, how is the feature documented? **not applicable**
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] JingsongLi commented on a change in pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
JingsongLi commented on a change in pull request #12321:
URL: https://github.com/apache/flink/pull/12321#discussion_r436469905



##########
File path: docs/dev/connectors/streamfile_sink.md
##########
@@ -204,6 +205,109 @@ input.addSink(sink)
 </div>
 </div>
 
+#### Avro format
+
+Flink also provides built-in support for writing data into Avro files. A list of convenience methods to create
+Avro writer factories and their associated documentation can be found in the 
+[AvroWriters]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroWriters.html) class.
+
+To use the Avro writers in your application you need to add the following dependency:
+
+{% highlight xml %}
+<dependency>
+  <groupId>org.apache.flink</groupId>
+  <artifactId>flink-avro</artifactId>
+  <version>{{ site.version }}</version>
+</dependency>
+{% endhighlight %}
+
+A StreamingFileSink that writes data to Avro files can be created like this:
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+{% highlight java %}
+import org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink;
+import org.apache.flink.formats.avro.AvroWriters;
+import org.apache.avro.Schema;
+
+
+Schema schema = ...;
+DataStream<GenericRecord> stream = ...;
+
+final StreamingFileSink<GenericRecord> sink = StreamingFileSink
+	.forBulkFormat(outputBasePath, AvroWriters.forGenericRecord(schema))
+	.build();
+
+input.addSink(sink);
+
+{% endhighlight %}
+</div>
+<div data-lang="scala" markdown="1">
+{% highlight scala %}
+import org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink
+import org.apache.flink.formats.avro.AvroWriters
+import org.apache.avro.Schema
+
+val schema: Schema = ...
+val input: DataStream[GenericRecord] = ...
+
+val sink: StreamingFileSink[GenericRecord] = StreamingFileSink
+    .forBulkFormat(outputBasePath, AvroWriters.forGenericRecord(schema))
+    .build()
+
+input.addSink(sink)
+
+{% endhighlight %}
+</div>
+</div>
+
+For creating customized Avro writers, e.g. enabling compression, users need to create the `AvroWriterFactory`
+with a custom implementation of the [AvroBuilder]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroBuilder.html) interface:
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+{% highlight java %}
+AvroWriterFactory<?> factory = new AvroWriterFactory<>(new AvroBuilder<Address>() {
+	@Override
+	public DataFileWriter<Address> createWriter(OutputStream out) throws IOException {

Review comment:
       Better to use lambda, because lambda is serializable, but inner class is not.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zhuzhurk commented on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
zhuzhurk commented on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-640383192


   @JingsongLi Would you take another look, especially the code example? I am not too familiar with `AvroWriter` so mainly checked the writing.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zhuzhurk commented on a change in pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
zhuzhurk commented on a change in pull request #12321:
URL: https://github.com/apache/flink/pull/12321#discussion_r434360735



##########
File path: docs/dev/connectors/streamfile_sink.md
##########
@@ -204,6 +205,65 @@ input.addSink(sink)
 </div>
 </div>
 
+#### Avro format
+
+Flink also provides built-in support for writing data into Avro files. A list of convenience methods to create
+Avro writer factories and their associated documentation can be found in the 
+[AvroWriters]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroWriters.html) class.
+
+For creating customized Avro writers like enabling compression, users need to create the `AvroWriterFactory`

Review comment:
       maybe "For creating customized Avro writers, e.g. enabling compression, ..." ?
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zhuzhurk commented on a change in pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
zhuzhurk commented on a change in pull request #12321:
URL: https://github.com/apache/flink/pull/12321#discussion_r434380806



##########
File path: docs/dev/connectors/streamfile_sink.md
##########
@@ -204,6 +205,65 @@ input.addSink(sink)
 </div>
 </div>
 
+#### Avro format
+
+Flink also provides built-in support for writing data into Avro files. A list of convenience methods to create
+Avro writer factories and their associated documentation can be found in the 
+[AvroWriters]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroWriters.html) class.
+
+For creating customized Avro writers like enabling compression, users need to create the `AvroWriterFactory`
+with a custom implementation of the [AvroBuilder]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroBuilder.html) interface.

Review comment:
       Would it make sense to add an example on creating customized Avro writers?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633558255


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143",
       "triggerID" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "triggerType" : "PUSH"
     }, {
       "hash" : "23a939d18a7e3b989e521963e0399b73c775165d",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2869",
       "triggerID" : "23a939d18a7e3b989e521963e0399b73c775165d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56bdc3a61f65cb30b48cf7f932520be02ebed734 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143) 
   * 23a939d18a7e3b989e521963e0399b73c775165d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2869) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] JingsongLi commented on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
JingsongLi commented on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633835029


   CC: @aljoscha and @kl0u 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] JingsongLi commented on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
JingsongLi commented on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-641235888


   Merging...


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633558255


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143",
       "triggerID" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "triggerType" : "PUSH"
     }, {
       "hash" : "23a939d18a7e3b989e521963e0399b73c775165d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "23a939d18a7e3b989e521963e0399b73c775165d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56bdc3a61f65cb30b48cf7f932520be02ebed734 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143) 
   * 23a939d18a7e3b989e521963e0399b73c775165d UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot commented on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633554264


   Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress of the review.
   
   
   ## Automated Checks
   Last check on commit 56bdc3a61f65cb30b48cf7f932520be02ebed734 (Mon May 25 12:44:12 UTC 2020)
   
   **Warnings:**
    * **Invalid pull request title: No valid Jira ID provided**
   
   
   <sub>Mention the bot in a comment to re-run the automated checks.</sub>
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
    The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
    - `@flinkbot approve all` to approve all aspects
    - `@flinkbot approve-until architecture` to approve everything until `architecture`
    - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
    - `@flinkbot disapprove architecture` to remove an approval you gave earlier
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] gaoyunhaii commented on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
gaoyunhaii commented on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-640179377


   Very thanks @JingsongLi @zhuzhurk for the reviewing! I have updated the PR according to the comments. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot commented on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633558255


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56bdc3a61f65cb30b48cf7f932520be02ebed734 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633558255


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143",
       "triggerID" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56bdc3a61f65cb30b48cf7f932520be02ebed734 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633558255






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633558255


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143",
       "triggerID" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "triggerType" : "PUSH"
     }, {
       "hash" : "23a939d18a7e3b989e521963e0399b73c775165d",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2869",
       "triggerID" : "23a939d18a7e3b989e521963e0399b73c775165d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 23a939d18a7e3b989e521963e0399b73c775165d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2869) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] JingsongLi commented on a change in pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
JingsongLi commented on a change in pull request #12321:
URL: https://github.com/apache/flink/pull/12321#discussion_r436469905



##########
File path: docs/dev/connectors/streamfile_sink.md
##########
@@ -204,6 +205,109 @@ input.addSink(sink)
 </div>
 </div>
 
+#### Avro format
+
+Flink also provides built-in support for writing data into Avro files. A list of convenience methods to create
+Avro writer factories and their associated documentation can be found in the 
+[AvroWriters]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroWriters.html) class.
+
+To use the Avro writers in your application you need to add the following dependency:
+
+{% highlight xml %}
+<dependency>
+  <groupId>org.apache.flink</groupId>
+  <artifactId>flink-avro</artifactId>
+  <version>{{ site.version }}</version>
+</dependency>
+{% endhighlight %}
+
+A StreamingFileSink that writes data to Avro files can be created like this:
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+{% highlight java %}
+import org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink;
+import org.apache.flink.formats.avro.AvroWriters;
+import org.apache.avro.Schema;
+
+
+Schema schema = ...;
+DataStream<GenericRecord> stream = ...;
+
+final StreamingFileSink<GenericRecord> sink = StreamingFileSink
+	.forBulkFormat(outputBasePath, AvroWriters.forGenericRecord(schema))
+	.build();
+
+input.addSink(sink);
+
+{% endhighlight %}
+</div>
+<div data-lang="scala" markdown="1">
+{% highlight scala %}
+import org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink
+import org.apache.flink.formats.avro.AvroWriters
+import org.apache.avro.Schema
+
+val schema: Schema = ...
+val input: DataStream[GenericRecord] = ...
+
+val sink: StreamingFileSink[GenericRecord] = StreamingFileSink
+    .forBulkFormat(outputBasePath, AvroWriters.forGenericRecord(schema))
+    .build()
+
+input.addSink(sink)
+
+{% endhighlight %}
+</div>
+</div>
+
+For creating customized Avro writers, e.g. enabling compression, users need to create the `AvroWriterFactory`
+with a custom implementation of the [AvroBuilder]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroBuilder.html) interface:
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+{% highlight java %}
+AvroWriterFactory<?> factory = new AvroWriterFactory<>(new AvroBuilder<Address>() {
+	@Override
+	public DataFileWriter<Address> createWriter(OutputStream out) throws IOException {

Review comment:
       Better to use lambda, because lambda is serializable, but inner class depends on outer class.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] gaoyunhaii commented on a change in pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
gaoyunhaii commented on a change in pull request #12321:
URL: https://github.com/apache/flink/pull/12321#discussion_r436340532



##########
File path: docs/dev/connectors/streamfile_sink.md
##########
@@ -204,6 +205,65 @@ input.addSink(sink)
 </div>
 </div>
 
+#### Avro format
+
+Flink also provides built-in support for writing data into Avro files. A list of convenience methods to create
+Avro writer factories and their associated documentation can be found in the 
+[AvroWriters]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroWriters.html) class.
+
+For creating customized Avro writers like enabling compression, users need to create the `AvroWriterFactory`
+with a custom implementation of the [AvroBuilder]({{ site.javadocs_baseurl }}/api/java/org/apache/flink/formats/avro/AvroBuilder.html) interface.

Review comment:
       I have added the example for the customized Avro writers.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12321:
URL: https://github.com/apache/flink/pull/12321#issuecomment-633558255


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143",
       "triggerID" : "56bdc3a61f65cb30b48cf7f932520be02ebed734",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56bdc3a61f65cb30b48cf7f932520be02ebed734 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2143) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] JingsongLi merged pull request #12321: Add document for writing Avro files with StreamingFileSink

Posted by GitBox <gi...@apache.org>.
JingsongLi merged pull request #12321:
URL: https://github.com/apache/flink/pull/12321


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org