You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Parth Desai <pa...@google.com> on 2020/04/28 16:31:57 UTC
Support for Avro 1.9.X
Hi,
The latest version of beam currently only supports Avro 1.8.2, however, my
customer (Charles Schwab) is heavily using Avro 1.9.x which handles date
fields much better. Is there any way they can upgrade from 1.8.2 to 1.9.x
in Dataflow/beam?
Thanks,
Parth Desai
Customer Engineer, Data Analytics Specialist - Google Cloud
500 W 2nd St, Austin, TX 78701
Mobile: 404-637-6674
Re: Support for Avro 1.9.X
Posted by Brian Hulette <bh...@google.com>.
There's no way to do this now. There was an attempt to upgrade to 1.9.1
here [1]. The hangup was just that the default datetime type was switched
to java time (from joda time). My memory is we could get that PR working
just by setting some configuration.. somewhere.. to make sure we use joda
time for Avro in Beam.
That approach limits us to 1.9 though as joda support is dropped in 1.10
[2]. Also you note that 1.9.x handles date fields much better - is that
*because* it uses java time by default?
Brian
[1] https://github.com/apache/beam/pull/9779
[2] https://github.com/apache/beam/pull/9779#discussion_r334400709
On Tue, Apr 28, 2020 at 9:34 AM Parth Desai <pa...@google.com> wrote:
> Hi,
>
> The latest version of beam currently only supports Avro 1.8.2, however, my
> customer (Charles Schwab) is heavily using Avro 1.9.x which handles date
> fields much better. Is there any way they can upgrade from 1.8.2 to 1.9.x
> in Dataflow/beam?
>
> Thanks,
>
>
>
> Parth Desai
>
> Customer Engineer, Data Analytics Specialist - Google Cloud
>
> 500 W 2nd St, Austin, TX 78701
>
> Mobile: 404-637-6674 <(404)%20637-6674>
>
>
>
>