You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Ryan Skraba <rs...@apache.org> on 2021/03/09 17:55:30 UTC

[VOTE] Release Apache Avro 1.10.2 RC2

Hi everyone,

I'd like to propose the following RC2 to be released as the official Apache
Avro 1.10.2 release.

There have been two commits since RC1, addressing the following issues:
- [AVRO-3072] ZSTD Performance improvements
- [AVRO-3073] Potential vulnerability with velocity

The commit id is 8111cdc35430ff68dcb644306362859de40999d9
* This corresponds to the tag: release-1.10.2-rc2
* https://github.com/apache/avro/releases/tag/release-1.10.2-rc2

The release tarball, signature, and checksums are here (revision 46543):
* https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/

You can find the KEYS file here:
* https://dist.apache.org/repos/dist/dev/avro/KEYS

Binary artifacts for Java are staged in Nexus here:
*
https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/

This release includes ~30 Jira issues:
*
https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2

Some interesting highlights:

Avro specification
- [AVRO-3028] Clarify that records encode values even if they equal their
default

C#
- [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings

C++
- [AVRO-3031] avrocppgen does not generate correct C++ code when the schema
contains
  C++ reserved words

Java
- [AVRO-2471] Java code generation doesn't add conversion for
timestamp-micros
- [AVRO-2860] More Closely Adhere to ASF Parent POM
- [AVRO-2944] DataFileReader has incorrect logic reading magic header
- [AVRO-3024] Bump Jackson to 2.12.1
- [AVRO-3060] Support ZSTD level and BufferPool options
- [AVRO-3049] BinaryDecoder lacks checks on bytes array length
- [AVRO-3072] ZSTD Performance improvements

Python
- [AVRO-3006] Update PyPI documentation to deprecate avro-python3
  *** The avro package supports Python 3, and avro-python3 will be removed
in
  the next major release ***

Ruby
- [AVRO-2984] Unnecessary memory allocations during serialization
- [AVRO-2998] Records with symbol keys fail validation
- [AVRO-2999] Optimize Ruby union serialization
- [AVRO-3000] Avoid unnecessary schema compatibility checks
- [AVRO-3023] Validate with Ruby 3

* Upgrade dependencies to latest versions, including CVE fixes.
* Multiple fixes, better documentation and more...

Avro 1.10 is still using Travis, but the status isn't necessarily reflected
on the branch in github:
* https://travis-ci.com/github/apache/avro/builds/219514042

Please download, verify, and test. This vote will remain open for at least
72 hours. Given sufficient votes, I would like to close after the weekend on
18h00 UTC Friday, March 12th, 2021

[ ] +1 Release this as Apache Avro 1.10.2
[ ] +0
[ ] -1 Do not release this because...

Best regards,
Ryan Skraba

Re: [VOTE] Release Apache Avro 1.10.2 RC2

Posted by Ryan Skraba <ry...@skraba.com>.
Hello everyone, I'm please to announce that the vote has passed to release
Avro 1.10.2.

I will start releasing artifacts to the various package managers mid-week.

Thanks everyone for your contributions! Ryan

On Thu, Mar 11, 2021, 22:47 Ismaël Mejía <ie...@gmail.com> wrote:

> +1 (binding)
>
> Validated signatures and hashes
> Built distribution from source for Java
> Tried with downstream projects (Spark/Parquet) with success
>
> On Thu, Mar 11, 2021 at 7:09 PM Daniel Kulp <dk...@apache.org> wrote:
> >
> >
> > +1
> >
> > Most of my testing was with RC1, but some followup with RC2 doesn’t
> reveal anything problematic.
> >
> > Dan
> >
> >
> >
> >
> > > On Mar 9, 2021, at 12:55 PM, Ryan Skraba <rs...@apache.org> wrote:
> > >
> > > Hi everyone,
> > >
> > > I'd like to propose the following RC2 to be released as the official
> Apache
> > > Avro 1.10.2 release.
> > >
> > > There have been two commits since RC1, addressing the following issues:
> > > - [AVRO-3072] ZSTD Performance improvements
> > > - [AVRO-3073] Potential vulnerability with velocity
> > >
> > > The commit id is 8111cdc35430ff68dcb644306362859de40999d9
> > > * This corresponds to the tag: release-1.10.2-rc2
> > > * https://github.com/apache/avro/releases/tag/release-1.10.2-rc2
> > >
> > > The release tarball, signature, and checksums are here (revision
> 46543):
> > > * https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/
> > >
> > > You can find the KEYS file here:
> > > * https://dist.apache.org/repos/dist/dev/avro/KEYS
> > >
> > > Binary artifacts for Java are staged in Nexus here:
> > > *
> > >
> https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/
> > >
> > > This release includes ~30 Jira issues:
> > > *
> > >
> https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2
> > >
> > > Some interesting highlights:
> > >
> > > Avro specification
> > > - [AVRO-3028] Clarify that records encode values even if they equal
> their
> > > default
> > >
> > > C#
> > > - [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings
> > >
> > > C++
> > > - [AVRO-3031] avrocppgen does not generate correct C++ code when the
> schema
> > > contains
> > >  C++ reserved words
> > >
> > > Java
> > > - [AVRO-2471] Java code generation doesn't add conversion for
> > > timestamp-micros
> > > - [AVRO-2860] More Closely Adhere to ASF Parent POM
> > > - [AVRO-2944] DataFileReader has incorrect logic reading magic header
> > > - [AVRO-3024] Bump Jackson to 2.12.1
> > > - [AVRO-3060] Support ZSTD level and BufferPool options
> > > - [AVRO-3049] BinaryDecoder lacks checks on bytes array length
> > > - [AVRO-3072] ZSTD Performance improvements
> > >
> > > Python
> > > - [AVRO-3006] Update PyPI documentation to deprecate avro-python3
> > >  *** The avro package supports Python 3, and avro-python3 will be
> removed
> > > in
> > >  the next major release ***
> > >
> > > Ruby
> > > - [AVRO-2984] Unnecessary memory allocations during serialization
> > > - [AVRO-2998] Records with symbol keys fail validation
> > > - [AVRO-2999] Optimize Ruby union serialization
> > > - [AVRO-3000] Avoid unnecessary schema compatibility checks
> > > - [AVRO-3023] Validate with Ruby 3
> > >
> > > * Upgrade dependencies to latest versions, including CVE fixes.
> > > * Multiple fixes, better documentation and more...
> > >
> > > Avro 1.10 is still using Travis, but the status isn't necessarily
> reflected
> > > on the branch in github:
> > > * https://travis-ci.com/github/apache/avro/builds/219514042
> > >
> > > Please download, verify, and test. This vote will remain open for at
> least
> > > 72 hours. Given sufficient votes, I would like to close after the
> weekend on
> > > 18h00 UTC Friday, March 12th, 2021
> > >
> > > [ ] +1 Release this as Apache Avro 1.10.2
> > > [ ] +0
> > > [ ] -1 Do not release this because...
> > >
> > > Best regards,
> > > Ryan Skraba
> >
> > --
> > Daniel Kulp
> > dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <
> http://dankulp.com/blog>
> > Talend - http://talend.com <http://coders.talend.com/>
>

Re: [VOTE] Release Apache Avro 1.10.2 RC2

Posted by Ismaël Mejía <ie...@gmail.com>.
+1 (binding)

Validated signatures and hashes
Built distribution from source for Java
Tried with downstream projects (Spark/Parquet) with success

On Thu, Mar 11, 2021 at 7:09 PM Daniel Kulp <dk...@apache.org> wrote:
>
>
> +1
>
> Most of my testing was with RC1, but some followup with RC2 doesn’t reveal anything problematic.
>
> Dan
>
>
>
>
> > On Mar 9, 2021, at 12:55 PM, Ryan Skraba <rs...@apache.org> wrote:
> >
> > Hi everyone,
> >
> > I'd like to propose the following RC2 to be released as the official Apache
> > Avro 1.10.2 release.
> >
> > There have been two commits since RC1, addressing the following issues:
> > - [AVRO-3072] ZSTD Performance improvements
> > - [AVRO-3073] Potential vulnerability with velocity
> >
> > The commit id is 8111cdc35430ff68dcb644306362859de40999d9
> > * This corresponds to the tag: release-1.10.2-rc2
> > * https://github.com/apache/avro/releases/tag/release-1.10.2-rc2
> >
> > The release tarball, signature, and checksums are here (revision 46543):
> > * https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/
> >
> > You can find the KEYS file here:
> > * https://dist.apache.org/repos/dist/dev/avro/KEYS
> >
> > Binary artifacts for Java are staged in Nexus here:
> > *
> > https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/
> >
> > This release includes ~30 Jira issues:
> > *
> > https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2
> >
> > Some interesting highlights:
> >
> > Avro specification
> > - [AVRO-3028] Clarify that records encode values even if they equal their
> > default
> >
> > C#
> > - [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings
> >
> > C++
> > - [AVRO-3031] avrocppgen does not generate correct C++ code when the schema
> > contains
> >  C++ reserved words
> >
> > Java
> > - [AVRO-2471] Java code generation doesn't add conversion for
> > timestamp-micros
> > - [AVRO-2860] More Closely Adhere to ASF Parent POM
> > - [AVRO-2944] DataFileReader has incorrect logic reading magic header
> > - [AVRO-3024] Bump Jackson to 2.12.1
> > - [AVRO-3060] Support ZSTD level and BufferPool options
> > - [AVRO-3049] BinaryDecoder lacks checks on bytes array length
> > - [AVRO-3072] ZSTD Performance improvements
> >
> > Python
> > - [AVRO-3006] Update PyPI documentation to deprecate avro-python3
> >  *** The avro package supports Python 3, and avro-python3 will be removed
> > in
> >  the next major release ***
> >
> > Ruby
> > - [AVRO-2984] Unnecessary memory allocations during serialization
> > - [AVRO-2998] Records with symbol keys fail validation
> > - [AVRO-2999] Optimize Ruby union serialization
> > - [AVRO-3000] Avoid unnecessary schema compatibility checks
> > - [AVRO-3023] Validate with Ruby 3
> >
> > * Upgrade dependencies to latest versions, including CVE fixes.
> > * Multiple fixes, better documentation and more...
> >
> > Avro 1.10 is still using Travis, but the status isn't necessarily reflected
> > on the branch in github:
> > * https://travis-ci.com/github/apache/avro/builds/219514042
> >
> > Please download, verify, and test. This vote will remain open for at least
> > 72 hours. Given sufficient votes, I would like to close after the weekend on
> > 18h00 UTC Friday, March 12th, 2021
> >
> > [ ] +1 Release this as Apache Avro 1.10.2
> > [ ] +0
> > [ ] -1 Do not release this because...
> >
> > Best regards,
> > Ryan Skraba
>
> --
> Daniel Kulp
> dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <http://dankulp.com/blog>
> Talend - http://talend.com <http://coders.talend.com/>

Re: [VOTE] Release Apache Avro 1.10.2 RC2

Posted by Daniel Kulp <dk...@apache.org>.
+1 

Most of my testing was with RC1, but some followup with RC2 doesn’t reveal anything problematic.

Dan




> On Mar 9, 2021, at 12:55 PM, Ryan Skraba <rs...@apache.org> wrote:
> 
> Hi everyone,
> 
> I'd like to propose the following RC2 to be released as the official Apache
> Avro 1.10.2 release.
> 
> There have been two commits since RC1, addressing the following issues:
> - [AVRO-3072] ZSTD Performance improvements
> - [AVRO-3073] Potential vulnerability with velocity
> 
> The commit id is 8111cdc35430ff68dcb644306362859de40999d9
> * This corresponds to the tag: release-1.10.2-rc2
> * https://github.com/apache/avro/releases/tag/release-1.10.2-rc2
> 
> The release tarball, signature, and checksums are here (revision 46543):
> * https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/
> 
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/avro/KEYS
> 
> Binary artifacts for Java are staged in Nexus here:
> *
> https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/
> 
> This release includes ~30 Jira issues:
> *
> https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2
> 
> Some interesting highlights:
> 
> Avro specification
> - [AVRO-3028] Clarify that records encode values even if they equal their
> default
> 
> C#
> - [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings
> 
> C++
> - [AVRO-3031] avrocppgen does not generate correct C++ code when the schema
> contains
>  C++ reserved words
> 
> Java
> - [AVRO-2471] Java code generation doesn't add conversion for
> timestamp-micros
> - [AVRO-2860] More Closely Adhere to ASF Parent POM
> - [AVRO-2944] DataFileReader has incorrect logic reading magic header
> - [AVRO-3024] Bump Jackson to 2.12.1
> - [AVRO-3060] Support ZSTD level and BufferPool options
> - [AVRO-3049] BinaryDecoder lacks checks on bytes array length
> - [AVRO-3072] ZSTD Performance improvements
> 
> Python
> - [AVRO-3006] Update PyPI documentation to deprecate avro-python3
>  *** The avro package supports Python 3, and avro-python3 will be removed
> in
>  the next major release ***
> 
> Ruby
> - [AVRO-2984] Unnecessary memory allocations during serialization
> - [AVRO-2998] Records with symbol keys fail validation
> - [AVRO-2999] Optimize Ruby union serialization
> - [AVRO-3000] Avoid unnecessary schema compatibility checks
> - [AVRO-3023] Validate with Ruby 3
> 
> * Upgrade dependencies to latest versions, including CVE fixes.
> * Multiple fixes, better documentation and more...
> 
> Avro 1.10 is still using Travis, but the status isn't necessarily reflected
> on the branch in github:
> * https://travis-ci.com/github/apache/avro/builds/219514042
> 
> Please download, verify, and test. This vote will remain open for at least
> 72 hours. Given sufficient votes, I would like to close after the weekend on
> 18h00 UTC Friday, March 12th, 2021
> 
> [ ] +1 Release this as Apache Avro 1.10.2
> [ ] +0
> [ ] -1 Do not release this because...
> 
> Best regards,
> Ryan Skraba

-- 
Daniel Kulp
dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <http://dankulp.com/blog>
Talend - http://talend.com <http://coders.talend.com/>

Re: [VOTE] Release Apache Avro 1.10.2 RC2

Posted by Michael Heuer <he...@gmail.com>.
Hello Ismaël,

Thank you for the clarification and work on these issues!

I submitted a similar issue for commons-compress [1]; though it appears they are still on 1.4.8-7 [2] hopefully they'll also be on  1.4.9-1 before the next release.

   michael

[1] https://issues.apache.org/jira/browse/COMPRESS-564 <https://issues.apache.org/jira/browse/COMPRESS-564>
[2] https://github.com/apache/commons-compress/pull/174 <https://github.com/apache/commons-compress/pull/174>


> On Mar 10, 2021, at 1:25 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> 
> Thanks for bringing this subject Michael, we have vastly improved our
> communication with downstream projects (Spark/Parquet/Hive) and most
> of the issues you and other users have had will be available soon in
> upcoming versions of those projects.
> 
> Spark finally merged the upgrade to Avro 1.10.1 recently [1], the
> Parquet upgrade will be there too. Dongjoon (one Spark committer) has
> been contributing to the zstd upgrades in the three projects
> (Avro/Parquet/Spark) so we should be good to go. Actually the 1.10.2
> release is motivated in part because of the needs of Spark [2] I
> expect to open the PR with the Avro upgrade once published so it will
> be included in Spark 3.2.
> 
> Regards,
> Ismaël
> 
> [1] https://github.com/apache/spark/pull/31232
> [2] https://github.com/apache/avro/pull/1129
> 
> On Wed, Mar 10, 2021 at 7:28 PM Michael Heuer <he...@gmail.com> wrote:
>> 
>> Hello,
>> 
>> The zstd-jni dependency version has been moving quickly lately, and there are API changes (although hopefully all compatible).  Any worry about version conflicts between Avro, Parquet, Spark, commons-compress, and other projects that all have a dependency on zstd-jni?
>> 
>> I would test this RC with our stack, but we're still stuck on Avro 1.8.2 due to other version conflicts between Avro and Parquet in Spark (which I believe have been resolved in the master branch towards a future Spark 3.2 release).
>> 
>> Sorry I can't be more help other than raising a possible concern.
>> 
>>   michael
>> 
>> 
>>> On Mar 10, 2021, at 12:17 PM, Ryan Skraba <ry...@skraba.com> wrote:
>>> 
>>> Hello!
>>> 
>>> I finished some validation and I'm pleased to add my own vote on the
>>> release candidate!
>>> 
>>> I checked the Avro 1.10.2-rc2 artifacts by:
>>> 
>>> * Running a large set of internal Java code and unit tests (already on
>>> 1.10) with these candidates.
>>> * Building parquet-mr with the candidates.
>>> * We have some internal python code and unit tests that I ran on versions
>>> python3.5 - python3.9.
>>> * Checked that all of the artifacts were signed and verified my own
>>> signature (FWIW)
>>> * Checked that the artifacts in the staging nexus were signed by my own
>>> signature (FWIW)
>>> 
>>> It looks good for me!
>>> 
>>> [X] +1 Release this as Apache Avro 1.10.2
>>> [ ] +0
>>> [ ] -1 Do not release this because...
>>> 
>>> All my best, Ryan
>>> 
>>> 
>>> On Tue, Mar 9, 2021 at 6:55 PM Ryan Skraba <rs...@apache.org> wrote:
>>> 
>>>> Hi everyone,
>>>> 
>>>> I'd like to propose the following RC2 to be released as the official Apache
>>>> Avro 1.10.2 release.
>>>> 
>>>> There have been two commits since RC1, addressing the following issues:
>>>> - [AVRO-3072] ZSTD Performance improvements
>>>> - [AVRO-3073] Potential vulnerability with velocity
>>>> 
>>>> The commit id is 8111cdc35430ff68dcb644306362859de40999d9
>>>> * This corresponds to the tag: release-1.10.2-rc2
>>>> * https://github.com/apache/avro/releases/tag/release-1.10.2-rc2
>>>> 
>>>> The release tarball, signature, and checksums are here (revision 46543):
>>>> * https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/
>>>> 
>>>> You can find the KEYS file here:
>>>> * https://dist.apache.org/repos/dist/dev/avro/KEYS
>>>> 
>>>> Binary artifacts for Java are staged in Nexus here:
>>>> *
>>>> https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/
>>>> 
>>>> This release includes ~30 Jira issues:
>>>> *
>>>> https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2
>>>> 
>>>> Some interesting highlights:
>>>> 
>>>> Avro specification
>>>> - [AVRO-3028] Clarify that records encode values even if they equal their
>>>> default
>>>> 
>>>> C#
>>>> - [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings
>>>> 
>>>> C++
>>>> - [AVRO-3031] avrocppgen does not generate correct C++ code when the
>>>> schema contains
>>>> C++ reserved words
>>>> 
>>>> Java
>>>> - [AVRO-2471] Java code generation doesn't add conversion for
>>>> timestamp-micros
>>>> - [AVRO-2860] More Closely Adhere to ASF Parent POM
>>>> - [AVRO-2944] DataFileReader has incorrect logic reading magic header
>>>> - [AVRO-3024] Bump Jackson to 2.12.1
>>>> - [AVRO-3060] Support ZSTD level and BufferPool options
>>>> - [AVRO-3049] BinaryDecoder lacks checks on bytes array length
>>>> - [AVRO-3072] ZSTD Performance improvements
>>>> 
>>>> Python
>>>> - [AVRO-3006] Update PyPI documentation to deprecate avro-python3
>>>> *** The avro package supports Python 3, and avro-python3 will be removed
>>>> in
>>>> the next major release ***
>>>> 
>>>> Ruby
>>>> - [AVRO-2984] Unnecessary memory allocations during serialization
>>>> - [AVRO-2998] Records with symbol keys fail validation
>>>> - [AVRO-2999] Optimize Ruby union serialization
>>>> - [AVRO-3000] Avoid unnecessary schema compatibility checks
>>>> - [AVRO-3023] Validate with Ruby 3
>>>> 
>>>> * Upgrade dependencies to latest versions, including CVE fixes.
>>>> * Multiple fixes, better documentation and more...
>>>> 
>>>> Avro 1.10 is still using Travis, but the status isn't necessarily
>>>> reflected on the branch in github:
>>>> * https://travis-ci.com/github/apache/avro/builds/219514042
>>>> 
>>>> Please download, verify, and test. This vote will remain open for at least
>>>> 72 hours. Given sufficient votes, I would like to close after the weekend
>>>> on
>>>> 18h00 UTC Friday, March 12th, 2021
>>>> 
>>>> [ ] +1 Release this as Apache Avro 1.10.2
>>>> [ ] +0
>>>> [ ] -1 Do not release this because...
>>>> 
>>>> Best regards,
>>>> Ryan Skraba
>>>> 
>>

Re: [VOTE] Release Apache Avro 1.10.2 RC2

Posted by Ismaël Mejía <ie...@gmail.com>.
Thanks for bringing this subject Michael, we have vastly improved our
communication with downstream projects (Spark/Parquet/Hive) and most
of the issues you and other users have had will be available soon in
upcoming versions of those projects.

Spark finally merged the upgrade to Avro 1.10.1 recently [1], the
Parquet upgrade will be there too. Dongjoon (one Spark committer) has
been contributing to the zstd upgrades in the three projects
(Avro/Parquet/Spark) so we should be good to go. Actually the 1.10.2
release is motivated in part because of the needs of Spark [2] I
expect to open the PR with the Avro upgrade once published so it will
be included in Spark 3.2.

Regards,
Ismaël

[1] https://github.com/apache/spark/pull/31232
[2] https://github.com/apache/avro/pull/1129

On Wed, Mar 10, 2021 at 7:28 PM Michael Heuer <he...@gmail.com> wrote:
>
> Hello,
>
> The zstd-jni dependency version has been moving quickly lately, and there are API changes (although hopefully all compatible).  Any worry about version conflicts between Avro, Parquet, Spark, commons-compress, and other projects that all have a dependency on zstd-jni?
>
> I would test this RC with our stack, but we're still stuck on Avro 1.8.2 due to other version conflicts between Avro and Parquet in Spark (which I believe have been resolved in the master branch towards a future Spark 3.2 release).
>
> Sorry I can't be more help other than raising a possible concern.
>
>    michael
>
>
> > On Mar 10, 2021, at 12:17 PM, Ryan Skraba <ry...@skraba.com> wrote:
> >
> > Hello!
> >
> > I finished some validation and I'm pleased to add my own vote on the
> > release candidate!
> >
> > I checked the Avro 1.10.2-rc2 artifacts by:
> >
> > * Running a large set of internal Java code and unit tests (already on
> > 1.10) with these candidates.
> > * Building parquet-mr with the candidates.
> > * We have some internal python code and unit tests that I ran on versions
> > python3.5 - python3.9.
> > * Checked that all of the artifacts were signed and verified my own
> > signature (FWIW)
> > * Checked that the artifacts in the staging nexus were signed by my own
> > signature (FWIW)
> >
> > It looks good for me!
> >
> > [X] +1 Release this as Apache Avro 1.10.2
> > [ ] +0
> > [ ] -1 Do not release this because...
> >
> > All my best, Ryan
> >
> >
> > On Tue, Mar 9, 2021 at 6:55 PM Ryan Skraba <rs...@apache.org> wrote:
> >
> >> Hi everyone,
> >>
> >> I'd like to propose the following RC2 to be released as the official Apache
> >> Avro 1.10.2 release.
> >>
> >> There have been two commits since RC1, addressing the following issues:
> >> - [AVRO-3072] ZSTD Performance improvements
> >> - [AVRO-3073] Potential vulnerability with velocity
> >>
> >> The commit id is 8111cdc35430ff68dcb644306362859de40999d9
> >> * This corresponds to the tag: release-1.10.2-rc2
> >> * https://github.com/apache/avro/releases/tag/release-1.10.2-rc2
> >>
> >> The release tarball, signature, and checksums are here (revision 46543):
> >> * https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/
> >>
> >> You can find the KEYS file here:
> >> * https://dist.apache.org/repos/dist/dev/avro/KEYS
> >>
> >> Binary artifacts for Java are staged in Nexus here:
> >> *
> >> https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/
> >>
> >> This release includes ~30 Jira issues:
> >> *
> >> https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2
> >>
> >> Some interesting highlights:
> >>
> >> Avro specification
> >> - [AVRO-3028] Clarify that records encode values even if they equal their
> >> default
> >>
> >> C#
> >> - [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings
> >>
> >> C++
> >> - [AVRO-3031] avrocppgen does not generate correct C++ code when the
> >> schema contains
> >>  C++ reserved words
> >>
> >> Java
> >> - [AVRO-2471] Java code generation doesn't add conversion for
> >> timestamp-micros
> >> - [AVRO-2860] More Closely Adhere to ASF Parent POM
> >> - [AVRO-2944] DataFileReader has incorrect logic reading magic header
> >> - [AVRO-3024] Bump Jackson to 2.12.1
> >> - [AVRO-3060] Support ZSTD level and BufferPool options
> >> - [AVRO-3049] BinaryDecoder lacks checks on bytes array length
> >> - [AVRO-3072] ZSTD Performance improvements
> >>
> >> Python
> >> - [AVRO-3006] Update PyPI documentation to deprecate avro-python3
> >>  *** The avro package supports Python 3, and avro-python3 will be removed
> >> in
> >>  the next major release ***
> >>
> >> Ruby
> >> - [AVRO-2984] Unnecessary memory allocations during serialization
> >> - [AVRO-2998] Records with symbol keys fail validation
> >> - [AVRO-2999] Optimize Ruby union serialization
> >> - [AVRO-3000] Avoid unnecessary schema compatibility checks
> >> - [AVRO-3023] Validate with Ruby 3
> >>
> >> * Upgrade dependencies to latest versions, including CVE fixes.
> >> * Multiple fixes, better documentation and more...
> >>
> >> Avro 1.10 is still using Travis, but the status isn't necessarily
> >> reflected on the branch in github:
> >> * https://travis-ci.com/github/apache/avro/builds/219514042
> >>
> >> Please download, verify, and test. This vote will remain open for at least
> >> 72 hours. Given sufficient votes, I would like to close after the weekend
> >> on
> >> 18h00 UTC Friday, March 12th, 2021
> >>
> >> [ ] +1 Release this as Apache Avro 1.10.2
> >> [ ] +0
> >> [ ] -1 Do not release this because...
> >>
> >> Best regards,
> >> Ryan Skraba
> >>
>

Re: [VOTE] Release Apache Avro 1.10.2 RC2

Posted by Michael Heuer <he...@gmail.com>.
Hello,

The zstd-jni dependency version has been moving quickly lately, and there are API changes (although hopefully all compatible).  Any worry about version conflicts between Avro, Parquet, Spark, commons-compress, and other projects that all have a dependency on zstd-jni?

I would test this RC with our stack, but we're still stuck on Avro 1.8.2 due to other version conflicts between Avro and Parquet in Spark (which I believe have been resolved in the master branch towards a future Spark 3.2 release).

Sorry I can't be more help other than raising a possible concern.

   michael


> On Mar 10, 2021, at 12:17 PM, Ryan Skraba <ry...@skraba.com> wrote:
> 
> Hello!
> 
> I finished some validation and I'm pleased to add my own vote on the
> release candidate!
> 
> I checked the Avro 1.10.2-rc2 artifacts by:
> 
> * Running a large set of internal Java code and unit tests (already on
> 1.10) with these candidates.
> * Building parquet-mr with the candidates.
> * We have some internal python code and unit tests that I ran on versions
> python3.5 - python3.9.
> * Checked that all of the artifacts were signed and verified my own
> signature (FWIW)
> * Checked that the artifacts in the staging nexus were signed by my own
> signature (FWIW)
> 
> It looks good for me!
> 
> [X] +1 Release this as Apache Avro 1.10.2
> [ ] +0
> [ ] -1 Do not release this because...
> 
> All my best, Ryan
> 
> 
> On Tue, Mar 9, 2021 at 6:55 PM Ryan Skraba <rs...@apache.org> wrote:
> 
>> Hi everyone,
>> 
>> I'd like to propose the following RC2 to be released as the official Apache
>> Avro 1.10.2 release.
>> 
>> There have been two commits since RC1, addressing the following issues:
>> - [AVRO-3072] ZSTD Performance improvements
>> - [AVRO-3073] Potential vulnerability with velocity
>> 
>> The commit id is 8111cdc35430ff68dcb644306362859de40999d9
>> * This corresponds to the tag: release-1.10.2-rc2
>> * https://github.com/apache/avro/releases/tag/release-1.10.2-rc2
>> 
>> The release tarball, signature, and checksums are here (revision 46543):
>> * https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/
>> 
>> You can find the KEYS file here:
>> * https://dist.apache.org/repos/dist/dev/avro/KEYS
>> 
>> Binary artifacts for Java are staged in Nexus here:
>> *
>> https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/
>> 
>> This release includes ~30 Jira issues:
>> *
>> https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2
>> 
>> Some interesting highlights:
>> 
>> Avro specification
>> - [AVRO-3028] Clarify that records encode values even if they equal their
>> default
>> 
>> C#
>> - [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings
>> 
>> C++
>> - [AVRO-3031] avrocppgen does not generate correct C++ code when the
>> schema contains
>>  C++ reserved words
>> 
>> Java
>> - [AVRO-2471] Java code generation doesn't add conversion for
>> timestamp-micros
>> - [AVRO-2860] More Closely Adhere to ASF Parent POM
>> - [AVRO-2944] DataFileReader has incorrect logic reading magic header
>> - [AVRO-3024] Bump Jackson to 2.12.1
>> - [AVRO-3060] Support ZSTD level and BufferPool options
>> - [AVRO-3049] BinaryDecoder lacks checks on bytes array length
>> - [AVRO-3072] ZSTD Performance improvements
>> 
>> Python
>> - [AVRO-3006] Update PyPI documentation to deprecate avro-python3
>>  *** The avro package supports Python 3, and avro-python3 will be removed
>> in
>>  the next major release ***
>> 
>> Ruby
>> - [AVRO-2984] Unnecessary memory allocations during serialization
>> - [AVRO-2998] Records with symbol keys fail validation
>> - [AVRO-2999] Optimize Ruby union serialization
>> - [AVRO-3000] Avoid unnecessary schema compatibility checks
>> - [AVRO-3023] Validate with Ruby 3
>> 
>> * Upgrade dependencies to latest versions, including CVE fixes.
>> * Multiple fixes, better documentation and more...
>> 
>> Avro 1.10 is still using Travis, but the status isn't necessarily
>> reflected on the branch in github:
>> * https://travis-ci.com/github/apache/avro/builds/219514042
>> 
>> Please download, verify, and test. This vote will remain open for at least
>> 72 hours. Given sufficient votes, I would like to close after the weekend
>> on
>> 18h00 UTC Friday, March 12th, 2021
>> 
>> [ ] +1 Release this as Apache Avro 1.10.2
>> [ ] +0
>> [ ] -1 Do not release this because...
>> 
>> Best regards,
>> Ryan Skraba
>>

Re: [VOTE] Release Apache Avro 1.10.2 RC2

Posted by Ryan Skraba <ry...@skraba.com>.
Hello!

I finished some validation and I'm pleased to add my own vote on the
release candidate!

I checked the Avro 1.10.2-rc2 artifacts by:

* Running a large set of internal Java code and unit tests (already on
1.10) with these candidates.
* Building parquet-mr with the candidates.
* We have some internal python code and unit tests that I ran on versions
python3.5 - python3.9.
* Checked that all of the artifacts were signed and verified my own
signature (FWIW)
* Checked that the artifacts in the staging nexus were signed by my own
signature (FWIW)

It looks good for me!

[X] +1 Release this as Apache Avro 1.10.2
[ ] +0
[ ] -1 Do not release this because...

All my best, Ryan


On Tue, Mar 9, 2021 at 6:55 PM Ryan Skraba <rs...@apache.org> wrote:

> Hi everyone,
>
> I'd like to propose the following RC2 to be released as the official Apache
> Avro 1.10.2 release.
>
> There have been two commits since RC1, addressing the following issues:
> - [AVRO-3072] ZSTD Performance improvements
> - [AVRO-3073] Potential vulnerability with velocity
>
> The commit id is 8111cdc35430ff68dcb644306362859de40999d9
> * This corresponds to the tag: release-1.10.2-rc2
> * https://github.com/apache/avro/releases/tag/release-1.10.2-rc2
>
> The release tarball, signature, and checksums are here (revision 46543):
> * https://dist.apache.org/repos/dist/dev/avro/avro-1.10.2-rc2/
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/avro/KEYS
>
> Binary artifacts for Java are staged in Nexus here:
> *
> https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.10.2/
>
> This release includes ~30 Jira issues:
> *
> https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.10.2
>
> Some interesting highlights:
>
> Avro specification
> - [AVRO-3028] Clarify that records encode values even if they equal their
> default
>
> C#
> - [AVRO-3005][AVRO-2983] BinaryDecoder fails to read large strings
>
> C++
> - [AVRO-3031] avrocppgen does not generate correct C++ code when the
> schema contains
>   C++ reserved words
>
> Java
> - [AVRO-2471] Java code generation doesn't add conversion for
> timestamp-micros
> - [AVRO-2860] More Closely Adhere to ASF Parent POM
> - [AVRO-2944] DataFileReader has incorrect logic reading magic header
> - [AVRO-3024] Bump Jackson to 2.12.1
> - [AVRO-3060] Support ZSTD level and BufferPool options
> - [AVRO-3049] BinaryDecoder lacks checks on bytes array length
> - [AVRO-3072] ZSTD Performance improvements
>
> Python
> - [AVRO-3006] Update PyPI documentation to deprecate avro-python3
>   *** The avro package supports Python 3, and avro-python3 will be removed
> in
>   the next major release ***
>
> Ruby
> - [AVRO-2984] Unnecessary memory allocations during serialization
> - [AVRO-2998] Records with symbol keys fail validation
> - [AVRO-2999] Optimize Ruby union serialization
> - [AVRO-3000] Avoid unnecessary schema compatibility checks
> - [AVRO-3023] Validate with Ruby 3
>
> * Upgrade dependencies to latest versions, including CVE fixes.
> * Multiple fixes, better documentation and more...
>
> Avro 1.10 is still using Travis, but the status isn't necessarily
> reflected on the branch in github:
> * https://travis-ci.com/github/apache/avro/builds/219514042
>
> Please download, verify, and test. This vote will remain open for at least
> 72 hours. Given sufficient votes, I would like to close after the weekend
> on
> 18h00 UTC Friday, March 12th, 2021
>
> [ ] +1 Release this as Apache Avro 1.10.2
> [ ] +0
> [ ] -1 Do not release this because...
>
> Best regards,
> Ryan Skraba
>