[jira] [Resolved] (AVRO-2052) Remove org.apache.avro.file.DataFileWriter Double Buffering

["BELUGA BEHR (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AVRO-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

BELUGA BEHR resolved AVRO-2052.
-------------------------------
    Resolution: Not A Problem

[~cutting] I just ran the Perf test.  Indeed, the {{directBinaryEncoder}} is slower than the {{binaryEncoder}} even when writing into a buffered OutputStream.  Closing the ticket.

> Remove org.apache.avro.file.DataFileWriter Double Buffering
> -----------------------------------------------------------
>
>                 Key: AVRO-2052
>                 URL: https://issues.apache.org/jira/browse/AVRO-2052
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.7, 1.8.2
>            Reporter: BELUGA BEHR
>            Assignee: BELUGA BEHR
>            Priority: Trivial
>         Attachments: AVRO-2052.1.patch
>
>
> {code:title=org.apache.avro.file.DataFileWriter}
>   private void init(OutputStream outs) throws IOException {
>     this.underlyingStream = outs;
>     this.out = new BufferedFileOutputStream(outs);
>     EncoderFactory efactory = new EncoderFactory();
>     this.vout = efactory.binaryEncoder(out, null);
>     dout.setSchema(schema);
>     buffer = new NonCopyingByteArrayOutputStream(
>         Math.min((int)(syncInterval * 1.25), Integer.MAX_VALUE/2 -1));
>     this.bufOut = efactory.binaryEncoder(buffer, null);
>     if (this.codec == null) {
>       this.codec = CodecFactory.nullCodec().createInstance();
>     }
>     this.isOpen = true;
>   }
> {code}
> It's clear here that both streams are writing to a buffered destination, {{ BufferedFileOutputStream}} and {{ByteArrayOutputStream}} therefore there is no reason to need a buffered encoder and instead, write directly to the buffered streams with {{directBinaryEncoder}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)