You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Gabor Szadovszky <ga...@apache.org> on 2021/03/11 15:53:37 UTC
[RESULT] Release Apache Parquet 1.12.0 RC2
Since there were no replies about the tarball issue I give a -1 (binding)
to be on the safe side. As a result this vote is FAILED.
Because the issue is already fixed I'll initiate a new RC soon.
Cheers,
Gabor
On Mon, Mar 1, 2021 at 11:47 AM Gabor Szadovszky <ga...@apache.org> wrote:
> Thanks Junjie, for testing.
>
> The issue is that some encryption interop tests uses parquet files from a
> separate git repository (parquet-testing). To achieve that the
> maven-exec-plugin is configured to use the "git submodule update" command
> for downloading the related files. Unfortunately, we did not think about
> this issue during implementing this idea.
> It all means that the tarball is not 100% standalone.
>
> I've created the jira PARQUET-1992
> <https://issues.apache.org/jira/browse/PARQUET-1992> about this topic.
> Any comments about the solution of this issue are welcomed.
>
> I am currently unsure if this issue is a show stopper or not. By editing
> the pom.xml file and removing related exec plugin the binaries are
> reproducible from the tarball. All the tests are also executable excluding
> the one that requires these parquet files. What are the exact requirements
> from a tarball?
>
> Thanks a lot,
> Gabor
>
> On Mon, Mar 1, 2021 at 11:19 AM Junjie Chen <ch...@gmail.com>
> wrote:
>
>> Hi
>> I downloaded the package and ran mvn clean install, it failed with
>> message:
>> [ERROR] Failed to execute goal
>> org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule update) on
>> project parquet-format-structures: Command
>> execution failed.: Process exited with an error: 128 (Exit value: 128) ->
>> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (git submodule updat
>> e) on project parquet-format-structures: Command execution failed.
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:215)
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:156)
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:148)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
>> (LifecycleModuleBuilder.java:117)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
>> (LifecycleModuleBuilder.java:81)
>> at
>>
>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
>> (SingleThreadedBuilder.java:56)
>> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
>> (LifecycleStarter.java:128)
>> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
>> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
>> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
>> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
>> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
>> at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke
>> (NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke
>> (DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke (Method.java:498)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
>> (Launcher.java:282)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
>> (Launcher.java:225)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
>> (Launcher.java:406)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.main
>> (Launcher.java:347)
>>
>> Should I install something before running the maven build?
>>
>> On Sun, Feb 28, 2021 at 11:50 AM Wang, Yuming <yu...@ebay.com.invalid>
>> wrote:
>>
>> > +1. Thank you Gabor. It works.
>> >
>> > On 2021/2/26, 19:50, "Gabor Szadovszky" <ga...@apache.org> wrote:
>> >
>> > External Email
>> >
>> > Thanks for testing. It seems that the previous (rc1) was stuck on
>> the
>> > staging maven repo. I've had to manually drop the previous upload. I
>> > don't
>> > remember we've had to do such things before. Anyway, the referenced
>> > staging
>> > maven repository now contains the rc2 build. Could you please try it
>> > again?
>> >
>> > Thanks a lot,
>> > Gabor
>> >
>> > On Fri, Feb 26, 2021 at 1:30 AM Wang, Yuming
>> <yumwang@ebay.com.invalid
>> > >
>> > wrote:
>> >
>> > > -1. It still throws incompatible exceptions:
>> > > Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
>> > > java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
>> > > at
>> > >
>> >
>> org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
>> > > at
>> > >
>> >
>> org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
>> > > at
>> > >
>> >
>> org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
>> > > at
>> > >
>> org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
>> > > at
>> > >
>> >
>> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
>> > > at
>> > >
>> >
>> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
>> > > at
>> > >
>> >
>> org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
>> > > ... 9 more
>> > >
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F31649&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zm%2FlQaid9CeDDxig3RdWFzJ065lqOoT0s3T72hqFwc4%3D&reserved=0
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fruns%2F1983121842%3Fcheck_suite_focus%3Dtrue&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BV9GaP1SCYjkVyH42Rs1PbpFxLvS3AY9nVLJLXZL8tE%3D&reserved=0
>> > >
>> > >
>> > > On 2021/2/25, 19:27, "Gabor Szadovszky" <ga...@apache.org> wrote:
>> > >
>> > > External Email
>> > >
>> > > Hi everyone,
>> > >
>> > > I propose the following RC to be released as the official
>> Apache
>> > > Parquet
>> > > 1.12.0 release.
>> > >
>> > > The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
>> > > * This corresponds to the tag: apache-parquet-1.12.0-rc2
>> > > *
>> > >
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S91VddHb1xV5XG5gMXSWU29UeF9H2yVbEjKv9koovYw%3D&reserved=0
>> > >
>> > > The release tarball, signature, and checksums are here:
>> > > *
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z6gftDJWkIdxqJH0LB6zbvEkf9Pb25uXOScmwQ%2BwwOM%3D&reserved=0
>> > >
>> > > You can find the KEYS file here:
>> > > *
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7gMa%2Fw%2F0PUujCwTBV%2F34F0s4JZwwIy%2FWKaO8pLsXfkw%3D&reserved=0
>> > >
>> > > Binary artifacts are staged in Nexus here:
>> > > *
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tNdrkiwxcx3lxkLr1wup%2FI8ohuEhoEuQJHjJ6gMJE9Q%3D&reserved=0
>> > >
>> > > For release content see:
>> > > *
>> > >
>> > >
>> >
>> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584049885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BjQTQq5iRekn2Cs7hk9b%2BcJy%2BpBOLLTI3Se5iSYYwbM%3D&reserved=0
>> > >
>> > > Please download, verify, and test.
>> > >
>> > > Please vote in the next 72 hours.
>> > >
>> > > [ ] +1 Release this as Apache Parquet 1.12.0
>> > > [ ] +0
>> > > [ ] -1 Do not release this because...
>> > >
>> > >
>> >
>> >
>>
>> --
>> Best Regards
>>
>