[jira] [Resolved] (AVRO-4031) Writer::append_to_with_codec cannot take block_size so it falls back to default even if original Writer had a different size

["Martin Tzvetanov Grigorov (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AVRO-4031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martin Tzvetanov Grigorov resolved AVRO-4031.
---------------------------------------------
    Fix Version/s: 0.18.0
         Assignee: Marco Lugo
       Resolution: Fixed

> Writer::append_to_with_codec cannot take block_size so it falls back to default even if original Writer had a different size
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AVRO-4031
>                 URL: https://issues.apache.org/jira/browse/AVRO-4031
>             Project: Apache Avro
>          Issue Type: Improvement
>          Components: rust
>            Reporter: Marco Lugo
>            Assignee: Marco Lugo
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.18.0
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Writer::append_to_with_codec is great to continue a writer but it can be inconsistent with the original writer in terms of block_size. If the original writer used a custom block_size then this new writer will fall back to DEFAULT_BLOCK_SIZE. This can be important when considering compression among other things.
>  
> I am not sure what the proper API should be here:
> - Change append_to_with_codec's signature ro reflect this, which would be a breaking change;
> - Add a new method, something like append_to_with_codec_and_block_size, but this does not feel very future proof;
> - Allow the Writer builder to set has_header to true. If I'm understanding this correctly, this would be future proof and would not break backwards compatibility;
> - Other?
>  
> From the little context I have, I would be inclined to choose the 3rd option.
>  
> What would be the preferred option, if any? I'd be happy to open a PR once there is consensus on the way forward.
> Thank you,
> Marco



--
This message was sent by Atlassian Jira
(v8.20.10#820010)