You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Gabor Szadovszky <ga...@apache.org> on 2021/02/25 11:26:50 UTC
[VOTE] Release Apache Parquet 1.12.0 RC2
Hi everyone,
I propose the following RC to be released as the official Apache Parquet
1.12.0 release.
The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
* This corresponds to the tag: apache-parquet-1.12.0-rc2
*
https://github.com/apache/parquet-mr/tree/d859b4435d139f1979f46cf63d2afdf32935ad51
The release tarball, signature, and checksums are here:
* https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.12.0-rc2
You can find the KEYS file here:
* https://downloads.apache.org/parquet/KEYS
Binary artifacts are staged in Nexus here:
* https://repository.apache.org/content/groups/staging/org/apache/parquet/
For release content see:
*
https://github.com/apache/parquet-mr/blob/apache-parquet-1.12.0-rc2/CHANGES.md
Please download, verify, and test.
Please vote in the next 72 hours.
[ ] +1 Release this as Apache Parquet 1.12.0
[ ] +0
[ ] -1 Do not release this because...
[RESULT] Release Apache Parquet 1.12.0 RC2
Posted by Gabor Szadovszky <ga...@apache.org>.
Since there were no replies about the tarball issue I give a -1 (binding)
to be on the safe side. As a result this vote is FAILED.
Because the issue is already fixed I'll initiate a new RC soon.
Cheers,
Gabor
On Mon, Mar 1, 2021 at 11:47 AM Gabor Szadovszky <ga...@apache.org> wrote:
> Thanks Junjie, for testing.
>
> The issue is that some encryption interop tests uses parquet files from a
> separate git repository (parquet-testing). To achieve that the
> maven-exec-plugin is configured to use the "git submodule update" command
> for downloading the related files. Unfortunately, we did not think about
> this issue during implementing this idea.
> It all means that the tarball is not 100% standalone.
>
> I've created the jira PARQUET-1992
> <https://issues.apache.org/jira/browse/PARQUET-1992> about this topic.
> Any comments about the solution of this issue are welcomed.
>
> I am currently unsure if this issue is a show stopper or not. By editing
> the pom.xml file and removing related exec plugin the binaries are
> reproducible from the tarball. All the tests are also executable excluding
> the one that requires these parquet files. What are the exact requirements
> from a tarball?
>
> Thanks a lot,
> Gabor
>
> On Mon, Mar 1, 2021 at 11:19 AM Junjie Chen <ch...@gmail.com>
> wrote:
>
>> Hi
>> I downloaded the package and ran mvn clean install, it failed with
>> message:
>> [ERROR] Failed to execute goal
>> org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule update) on
>> project parquet-format-structures: Command
>> execution failed.: Process exited with an error: 128 (Exit value: 128) ->
>> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule updat
>> e) on project parquet-format-structures: Command execution failed.
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:215)
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:156)
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:148)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
>> (LifecycleModuleBuilder.java:117)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
>> (LifecycleModuleBuilder.java:81)
>> at
>>
>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
>> (SingleThreadedBuilder.java:56)
>> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
>> (LifecycleStarter.java:128)
>> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
>> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
>> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
>> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
>> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
>> at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke
>> (NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke
>> (DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke (Method.java:498)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
>> (Launcher.java:282)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
>> (Launcher.java:225)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
>> (Launcher.java:406)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.main
>> (Launcher.java:347)
>>
>> Should I install something before running the maven build?
>>
>> On Sun, Feb 28, 2021 at 11:50 AM Wang, Yuming <yu...@ebay.com.invalid>
>> wrote:
>>
>> > +1. Thank you Gabor. It works.
>> >
>> > On 2021/2/26, 19:50, "Gabor Szadovszky" <ga...@apache.org> wrote:
>> >
>> > External Email
>> >
>> > Thanks for testing. It seems that the previous (rc1) was stuck on
>> the
>> > staging maven repo. I've had to manually drop the previous upload. I
>> > don't
>> > remember we've had to do such things before. Anyway, the referenced
>> > staging
>> > maven repository now contains the rc2 build. Could you please try it
>> > again?
>> >
>> > Thanks a lot,
>> > Gabor
>> >
>> > On Fri, Feb 26, 2021 at 1:30 AM Wang, Yuming
>> <yumwang@ebay.com.invalid
>> > >
>> > wrote:
>> >
>> > > -1. It still throws incompatible exceptions:
>> > > Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
>> > > java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
>> > > at
>> > >
>> >
>> org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
>> > > at
>> > >
>> >
>> org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
>> > > at
>> > >
>> >
>> org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
>> > > at
>> > >
>> org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
>> > > at
>> > >
>> >
>> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
>> > > at
>> > >
>> >
>> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
>> > > ... 9 more
>> > >
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F31649&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zm%2FlQaid9CeDDxig3RdWFzJ065lqOoT0s3T72hqFwc4%3D&reserved=0
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fruns%2F1983121842%3Fcheck_suite_focus%3Dtrue&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BV9GaP1SCYjkVyH42Rs1PbpFxLvS3AY9nVLJLXZL8tE%3D&reserved=0
>> > >
>> > >
>> > > On 2021/2/25, 19:27, "Gabor Szadovszky" <ga...@apache.org> wrote:
>> > >
>> > > External Email
>> > >
>> > > Hi everyone,
>> > >
>> > > I propose the following RC to be released as the official
>> Apache
>> > > Parquet
>> > > 1.12.0 release.
>> > >
>> > > The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
>> > > * This corresponds to the tag: apache-parquet-1.12.0-rc2
>> > > *
>> > >
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S91VddHb1xV5XG5gMXSWU29UeF9H2yVbEjKv9koovYw%3D&reserved=0
>> > >
>> > > The release tarball, signature, and checksums are here:
>> > > *
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z6gftDJWkIdxqJH0LB6zbvEkf9Pb25uXOScmwQ%2BwwOM%3D&reserved=0
>> > >
>> > > You can find the KEYS file here:
>> > > *
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7gMa%2Fw%2F0PUujCwTBV%2F34F0s4JZwwIy%2FWKaO8pLsXfkw%3D&reserved=0
>> > >
>> > > Binary artifacts are staged in Nexus here:
>> > > *
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tNdrkiwxcx3lxkLr1wup%2FI8ohuEhoEuQJHjJ6gMJE9Q%3D&reserved=0
>> > >
>> > > For release content see:
>> > > *
>> > >
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BjQTQq5iRekn2Cs7hk9b%2BcJy%2BpBOLLTI3Se5iSYYwbM%3D&reserved=0
>> > >
>> > > Please download, verify, and test.
>> > >
>> > > Please vote in the next 72 hours.
>> > >
>> > > [ ] +1 Release this as Apache Parquet 1.12.0
>> > > [ ] +0
>> > > [ ] -1 Do not release this because...
>> > >
>> > >
>> >
>> >
>>
>> --
>> Best Regards
>>
>
Re: [VOTE] Release Apache Parquet 1.12.0 RC2
Posted by Gabor Szadovszky <ga...@apache.org>.
Thanks Junjie, for testing.
The issue is that some encryption interop tests uses parquet files from a
separate git repository (parquet-testing). To achieve that the
maven-exec-plugin is configured to use the "git submodule update" command
for downloading the related files. Unfortunately, we did not think about
this issue during implementing this idea.
It all means that the tarball is not 100% standalone.
I've created the jira PARQUET-1992
<https://issues.apache.org/jira/browse/PARQUET-1992> about this topic. Any
comments about the solution of this issue are welcomed.
I am currently unsure if this issue is a show stopper or not. By editing
the pom.xml file and removing related exec plugin the binaries are
reproducible from the tarball. All the tests are also executable excluding
the one that requires these parquet files. What are the exact requirements
from a tarball?
Thanks a lot,
Gabor
On Mon, Mar 1, 2021 at 11:19 AM Junjie Chen <ch...@gmail.com>
wrote:
> Hi
> I downloaded the package and ran mvn clean install, it failed with message:
> [ERROR] Failed to execute goal
> org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule update) on
> project parquet-format-structures: Command
> execution failed.: Process exited with an error: 128 (Exit value: 128) ->
> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule updat
> e) on project parquet-format-structures: Command execution failed.
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:215)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)
> at
>
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
>
> Should I install something before running the maven build?
>
> On Sun, Feb 28, 2021 at 11:50 AM Wang, Yuming <yu...@ebay.com.invalid>
> wrote:
>
> > +1. Thank you Gabor. It works.
> >
> > On 2021/2/26, 19:50, "Gabor Szadovszky" <ga...@apache.org> wrote:
> >
> > External Email
> >
> > Thanks for testing. It seems that the previous (rc1) was stuck on the
> > staging maven repo. I've had to manually drop the previous upload. I
> > don't
> > remember we've had to do such things before. Anyway, the referenced
> > staging
> > maven repository now contains the rc2 build. Could you please try it
> > again?
> >
> > Thanks a lot,
> > Gabor
> >
> > On Fri, Feb 26, 2021 at 1:30 AM Wang, Yuming
> <yumwang@ebay.com.invalid
> > >
> > wrote:
> >
> > > -1. It still throws incompatible exceptions:
> > > Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
> > > java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
> > > at
> > >
> >
> org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
> > > at
> > >
> >
> org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
> > > at
> > >
> >
> org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
> > > at
> > >
> org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
> > > at
> > >
> >
> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
> > > at
> > >
> >
> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
> > > at
> > >
> >
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
> > > at
> > >
> >
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
> > > at
> > >
> >
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
> > > at
> > >
> >
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
> > > at
> > >
> >
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
> > > ... 9 more
> > >
> > >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F31649&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zm%2FlQaid9CeDDxig3RdWFzJ065lqOoT0s3T72hqFwc4%3D&reserved=0
> > >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fruns%2F1983121842%3Fcheck_suite_focus%3Dtrue&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BV9GaP1SCYjkVyH42Rs1PbpFxLvS3AY9nVLJLXZL8tE%3D&reserved=0
> > >
> > >
> > > On 2021/2/25, 19:27, "Gabor Szadovszky" <ga...@apache.org> wrote:
> > >
> > > External Email
> > >
> > > Hi everyone,
> > >
> > > I propose the following RC to be released as the official
> Apache
> > > Parquet
> > > 1.12.0 release.
> > >
> > > The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
> > > * This corresponds to the tag: apache-parquet-1.12.0-rc2
> > > *
> > >
> > >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S91VddHb1xV5XG5gMXSWU29UeF9H2yVbEjKv9koovYw%3D&reserved=0
> > >
> > > The release tarball, signature, and checksums are here:
> > > *
> > >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z6gftDJWkIdxqJH0LB6zbvEkf9Pb25uXOScmwQ%2BwwOM%3D&reserved=0
> > >
> > > You can find the KEYS file here:
> > > *
> > >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7gMa%2Fw%2F0PUujCwTBV%2F34F0s4JZwwIy%2FWKaO8pLsXfkw%3D&reserved=0
> > >
> > > Binary artifacts are staged in Nexus here:
> > > *
> > >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tNdrkiwxcx3lxkLr1wup%2FI8ohuEhoEuQJHjJ6gMJE9Q%3D&reserved=0
> > >
> > > For release content see:
> > > *
> > >
> > >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BjQTQq5iRekn2Cs7hk9b%2BcJy%2BpBOLLTI3Se5iSYYwbM%3D&reserved=0
> > >
> > > Please download, verify, and test.
> > >
> > > Please vote in the next 72 hours.
> > >
> > > [ ] +1 Release this as Apache Parquet 1.12.0
> > > [ ] +0
> > > [ ] -1 Do not release this because...
> > >
> > >
> >
> >
>
> --
> Best Regards
>
Re: [VOTE] Release Apache Parquet 1.12.0 RC2
Posted by Junjie Chen <ch...@gmail.com>.
Hi
I downloaded the package and ran mvn clean install, it failed with message:
[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule update) on
project parquet-format-structures: Command
execution failed.: Process exited with an error: 128 (Exit value: 128) ->
[Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule updat
e) on project parquet-format-structures: Command execution failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:148)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:117)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:81)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
(SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
(Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch
(Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
(Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main
(Launcher.java:347)
Should I install something before running the maven build?
On Sun, Feb 28, 2021 at 11:50 AM Wang, Yuming <yu...@ebay.com.invalid>
wrote:
> +1. Thank you Gabor. It works.
>
> On 2021/2/26, 19:50, "Gabor Szadovszky" <ga...@apache.org> wrote:
>
> External Email
>
> Thanks for testing. It seems that the previous (rc1) was stuck on the
> staging maven repo. I've had to manually drop the previous upload. I
> don't
> remember we've had to do such things before. Anyway, the referenced
> staging
> maven repository now contains the rc2 build. Could you please try it
> again?
>
> Thanks a lot,
> Gabor
>
> On Fri, Feb 26, 2021 at 1:30 AM Wang, Yuming <yumwang@ebay.com.invalid
> >
> wrote:
>
> > -1. It still throws incompatible exceptions:
> > Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
> > java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
> > at
> >
> org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
> > at
> >
> org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
> > at
> >
> org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
> > at
> > org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
> > at
> >
> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
> > at
> >
> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
> > at
> >
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
> > at
> >
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
> > at
> >
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
> > at
> >
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
> > at
> >
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
> > ... 9 more
> >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F31649&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zm%2FlQaid9CeDDxig3RdWFzJ065lqOoT0s3T72hqFwc4%3D&reserved=0
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fruns%2F1983121842%3Fcheck_suite_focus%3Dtrue&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BV9GaP1SCYjkVyH42Rs1PbpFxLvS3AY9nVLJLXZL8tE%3D&reserved=0
> >
> >
> > On 2021/2/25, 19:27, "Gabor Szadovszky" <ga...@apache.org> wrote:
> >
> > External Email
> >
> > Hi everyone,
> >
> > I propose the following RC to be released as the official Apache
> > Parquet
> > 1.12.0 release.
> >
> > The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
> > * This corresponds to the tag: apache-parquet-1.12.0-rc2
> > *
> >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S91VddHb1xV5XG5gMXSWU29UeF9H2yVbEjKv9koovYw%3D&reserved=0
> >
> > The release tarball, signature, and checksums are here:
> > *
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z6gftDJWkIdxqJH0LB6zbvEkf9Pb25uXOScmwQ%2BwwOM%3D&reserved=0
> >
> > You can find the KEYS file here:
> > *
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7gMa%2Fw%2F0PUujCwTBV%2F34F0s4JZwwIy%2FWKaO8pLsXfkw%3D&reserved=0
> >
> > Binary artifacts are staged in Nexus here:
> > *
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tNdrkiwxcx3lxkLr1wup%2FI8ohuEhoEuQJHjJ6gMJE9Q%3D&reserved=0
> >
> > For release content see:
> > *
> >
> >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BjQTQq5iRekn2Cs7hk9b%2BcJy%2BpBOLLTI3Se5iSYYwbM%3D&reserved=0
> >
> > Please download, verify, and test.
> >
> > Please vote in the next 72 hours.
> >
> > [ ] +1 Release this as Apache Parquet 1.12.0
> > [ ] +0
> > [ ] -1 Do not release this because...
> >
> >
>
>
--
Best Regards
Re: [VOTE] Release Apache Parquet 1.12.0 RC2
Posted by "Wang, Yuming" <yu...@ebay.com.INVALID>.
+1. Thank you Gabor. It works.
On 2021/2/26, 19:50, "Gabor Szadovszky" <ga...@apache.org> wrote:
External Email
Thanks for testing. It seems that the previous (rc1) was stuck on the
staging maven repo. I've had to manually drop the previous upload. I don't
remember we've had to do such things before. Anyway, the referenced staging
maven repository now contains the rc2 build. Could you please try it again?
Thanks a lot,
Gabor
On Fri, Feb 26, 2021 at 1:30 AM Wang, Yuming <yu...@ebay.com.invalid>
wrote:
> -1. It still throws incompatible exceptions:
> Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
> java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
> at
> org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
> at
> org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
> at
> org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
> at
> org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
> at
> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
> at
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
> ... 9 more
>
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F31649&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zm%2FlQaid9CeDDxig3RdWFzJ065lqOoT0s3T72hqFwc4%3D&reserved=0
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fruns%2F1983121842%3Fcheck_suite_focus%3Dtrue&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BV9GaP1SCYjkVyH42Rs1PbpFxLvS3AY9nVLJLXZL8tE%3D&reserved=0
>
>
> On 2021/2/25, 19:27, "Gabor Szadovszky" <ga...@apache.org> wrote:
>
> External Email
>
> Hi everyone,
>
> I propose the following RC to be released as the official Apache
> Parquet
> 1.12.0 release.
>
> The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
> * This corresponds to the tag: apache-parquet-1.12.0-rc2
> *
>
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S91VddHb1xV5XG5gMXSWU29UeF9H2yVbEjKv9koovYw%3D&reserved=0
>
> The release tarball, signature, and checksums are here:
> *
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z6gftDJWkIdxqJH0LB6zbvEkf9Pb25uXOScmwQ%2BwwOM%3D&reserved=0
>
> You can find the KEYS file here:
> *
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7gMa%2Fw%2F0PUujCwTBV%2F34F0s4JZwwIy%2FWKaO8pLsXfkw%3D&reserved=0
>
> Binary artifacts are staged in Nexus here:
> *
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tNdrkiwxcx3lxkLr1wup%2FI8ohuEhoEuQJHjJ6gMJE9Q%3D&reserved=0
>
> For release content see:
> *
>
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BjQTQq5iRekn2Cs7hk9b%2BcJy%2BpBOLLTI3Se5iSYYwbM%3D&reserved=0
>
> Please download, verify, and test.
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Parquet 1.12.0
> [ ] +0
> [ ] -1 Do not release this because...
>
>
Re: [VOTE] Release Apache Parquet 1.12.0 RC2
Posted by Gabor Szadovszky <ga...@apache.org>.
Thanks for testing. It seems that the previous (rc1) was stuck on the
staging maven repo. I've had to manually drop the previous upload. I don't
remember we've had to do such things before. Anyway, the referenced staging
maven repository now contains the rc2 build. Could you please try it again?
Thanks a lot,
Gabor
On Fri, Feb 26, 2021 at 1:30 AM Wang, Yuming <yu...@ebay.com.invalid>
wrote:
> -1. It still throws incompatible exceptions:
> Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
> java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
> at
> org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
> at
> org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
> at
> org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
> at
> org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
> at
> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
> at
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
> ... 9 more
>
> https://github.com/apache/spark/pull/31649
> https://github.com/apache/spark/runs/1983121842?check_suite_focus=true
>
>
> On 2021/2/25, 19:27, "Gabor Szadovszky" <ga...@apache.org> wrote:
>
> External Email
>
> Hi everyone,
>
> I propose the following RC to be released as the official Apache
> Parquet
> 1.12.0 release.
>
> The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
> * This corresponds to the tag: apache-parquet-1.12.0-rc2
> *
>
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BRDSPrv9JNQVY3FEgNd%2B3v%2FUNj3nYnYwRFxH7P%2BeqfI%3D&reserved=0
>
> The release tarball, signature, and checksums are here:
> *
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6I9cfSlW0hzN5vFtMv6%2FnTb11v%2BMolcoENGod0pwuGk%3D&reserved=0
>
> You can find the KEYS file here:
> *
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XBD6DvGcFxKVpQ0%2Bfq8aF6j0krO4rJjLqR76QYfjhNc%3D&reserved=0
>
> Binary artifacts are staged in Nexus here:
> *
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=J05VEyDHfWax42zwNiIUaakX%2BEWBZvRdbfIm%2B4Ku51A%3D&reserved=0
>
> For release content see:
> *
>
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yxOGzFI5DLpgekoNTGkDX4tYL6NLbLUrzZWS7X8LFTM%3D&reserved=0
>
> Please download, verify, and test.
>
> Please vote in the next 72 hours.
>
> [ ] +1 Release this as Apache Parquet 1.12.0
> [ ] +0
> [ ] -1 Do not release this because...
>
>
Re: [VOTE] Release Apache Parquet 1.12.0 RC2
Posted by "Wang, Yuming" <yu...@ebay.com.INVALID>.
-1. It still throws incompatible exceptions:
Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
at org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
at org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
at org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
at org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
at org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
at org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
at org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
at org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
at org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
at org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
... 9 more
https://github.com/apache/spark/pull/31649
https://github.com/apache/spark/runs/1983121842?check_suite_focus=true
On 2021/2/25, 19:27, "Gabor Szadovszky" <ga...@apache.org> wrote:
External Email
Hi everyone,
I propose the following RC to be released as the official Apache Parquet
1.12.0 release.
The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
* This corresponds to the tag: apache-parquet-1.12.0-rc2
*
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BRDSPrv9JNQVY3FEgNd%2B3v%2FUNj3nYnYwRFxH7P%2BeqfI%3D&reserved=0
The release tarball, signature, and checksums are here:
* https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6I9cfSlW0hzN5vFtMv6%2FnTb11v%2BMolcoENGod0pwuGk%3D&reserved=0
You can find the KEYS file here:
* https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XBD6DvGcFxKVpQ0%2Bfq8aF6j0krO4rJjLqR76QYfjhNc%3D&reserved=0
Binary artifacts are staged in Nexus here:
* https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=J05VEyDHfWax42zwNiIUaakX%2BEWBZvRdbfIm%2B4Ku51A%3D&reserved=0
For release content see:
*
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yxOGzFI5DLpgekoNTGkDX4tYL6NLbLUrzZWS7X8LFTM%3D&reserved=0
Please download, verify, and test.
Please vote in the next 72 hours.
[ ] +1 Release this as Apache Parquet 1.12.0
[ ] +0
[ ] -1 Do not release this because...