You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Ryan Blue <bl...@apache.org> on 2016/10/13 17:53:37 UTC
[VOTE] Release Apache Parquet 1.9.0 RC1
Hi everyone,
I propose the following RC to be released as official Apache Parquet 1.9.0
release.
The commit id is 2a99abf784cb6e76160d49506ea87581a2256021
* This corresponds to the tag: apache-parquet-1.9.0
* https://github.com/apache/parquet-mr/tree/2a99abf7
*
https://git-wip-us.apache.org/repos/asf/projects/repo?p=parquet-mr.git&a=commit&h=2a99abf7
The release tarball, signature, and checksums are here:
* https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.9.0-rc1
You can find the KEYS file here:
* https://dist.apache.org/repos/dist/dev/parquet/KEYS
Binary artifacts are staged in Nexus here:
*
https://repository.apache.org/content/groups/staging/org/apache/parquet/parquet/
This release includes:
* Support for Hadoop's ByteBuffer read API
* Dictionary-based row group filters
* Cascading3 integration
* Predicate push-down for Pig
* Decimal, timestamp, and date support in Avro
* Numerous bug fixes
Please verify, test, and vote by Monday, 17 October 2016.
[ ] +1 Release this as Apache Parquet 1.9.0
[ ] +0
[ ] -1 Do not release this because...
--
Ryan Blue
Re: [VOTE] Release Apache Parquet 1.9.0 RC1
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
Uwe,
Thanks for checking the candidate! Looks like the way to do it is `java
-Duser.country=US -Duser.language=en . . .` (according to [a question on
stack overflow](
http://stackoverflow.com/questions/8809098/how-do-i-set-the-default-locale-for-my-jvm)
).
Others should also run the tests, and as long as this is the only failure
(due to locale), I don't think this should be a reason to fail the
candidate.
rb
On Fri, Oct 14, 2016 at 2:09 AM, Uwe Korn <uw...@xhochy.com> wrote:
> Hello Ryan,
>
> sadly I have failing tests with the RC. Seems like they are locale
> dependent ("," vs "."). Rerunning with LANG=en_US.UTF-8 did sadly not solve
> this, is there some other magic I need to provide to switch JVM locals?
>
> % cat parquet-column/target/surefire-reports/org.apache.parquet.
> column.statistics.TestStatistics.txt
> ------------------------------------------------------------
> -------------------
> Test set: org.apache.parquet.column.statistics.TestStatistics
> ------------------------------------------------------------
> -------------------
> Tests run: 9, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec
> <<< FAILURE!
> testFloatMinMax(org.apache.parquet.column.statistics.TestStatistics) Time
> elapsed: 0.01 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<min: 0[,00010, max: 553,]59998,
> num_nulls: 0> but was:<min: 0[.00010, max: 553.]59998, num_nulls: 0>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at org.apache.parquet.column.statistics.TestStatistics.testFloa
> tMinMax(TestStatistics.java:235)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> ssorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(Refl
> ectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
> ameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(
> InvokeMethod.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
> 4ClassRunner.java:68)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
> 4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit
> 4TestSet.java:53)
> at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestS
> et(JUnit4Provider.java:123)
> at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit
> 4Provider.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> ssorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodW
> ithArray(ReflectionUtils.java:164)
> at org.apache.maven.surefire.booter.ProviderFactory$ProviderPro
> xy.invoke(ProviderFactory.java:110)
> at org.apache.maven.surefire.booter.SurefireStarter.invokeProvi
> der(SurefireStarter.java:175)
> at org.apache.maven.surefire.booter.SurefireStarter.runSuitesIn
> ProcessWhenForked(SurefireStarter.java:107)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBoo
> ter.java:68)
> testDoubleMinMax(org.apache.parquet.column.statistics.TestStatistics)
> Time elapsed: 0 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<min: 0[,00001, max: 944,]50000,
> num_nulls: 0> but was:<min: 0[.00001, max: 944.]50000, num_nulls: 0>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at org.apache.parquet.column.statistics.TestStatistics.testDoub
> leMinMax(TestStatistics.java:296)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> ssorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> FrameworkMethod.java:45)
> at org.junit.internal.runners.model.ReflectiveCallable.run(Refl
> ectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
> ameworkMethod.java:42)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(
> InvokeMethod.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
> 4ClassRunner.java:68)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
> 4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit
> 4TestSet.java:53)
> at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestS
> et(JUnit4Provider.java:123)
> at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit
> 4Provider.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> ssorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodW
> ithArray(ReflectionUtils.java:164)
> at org.apache.maven.surefire.booter.ProviderFactory$ProviderPro
> xy.invoke(ProviderFactory.java:110)
> at org.apache.maven.surefire.booter.SurefireStarter.invokeProvi
> der(SurefireStarter.java:175)
> at org.apache.maven.surefire.booter.SurefireStarter.runSuitesIn
> ProcessWhenForked(SurefireStarter.java:107)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBoo
> ter.java:68)
>
>
>
> On 13.10.16 19:53, Ryan Blue wrote:
>
>> Hi everyone,
>>
>> I propose the following RC to be released as official Apache Parquet 1.9.0
>> release.
>>
>> The commit id is 2a99abf784cb6e76160d49506ea87581a2256021
>> * This corresponds to the tag: apache-parquet-1.9.0
>> * https://github.com/apache/parquet-mr/tree/2a99abf7
>> *
>> https://git-wip-us.apache.org/repos/asf/projects/repo?p=parq
>> uet-mr.git&a=commit&h=2a99abf7
>>
>> The release tarball, signature, and checksums are here:
>> * https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.9.0-rc1
>>
>> You can find the KEYS file here:
>> * https://dist.apache.org/repos/dist/dev/parquet/KEYS
>>
>> Binary artifacts are staged in Nexus here:
>> *
>> https://repository.apache.org/content/groups/staging/org/apa
>> che/parquet/parquet/
>>
>> This release includes:
>> * Support for Hadoop's ByteBuffer read API
>> * Dictionary-based row group filters
>> * Cascading3 integration
>> * Predicate push-down for Pig
>> * Decimal, timestamp, and date support in Avro
>> * Numerous bug fixes
>>
>> Please verify, test, and vote by Monday, 17 October 2016.
>>
>> [ ] +1 Release this as Apache Parquet 1.9.0
>> [ ] +0
>> [ ] -1 Do not release this because...
>>
>>
>>
>
--
Ryan Blue
Software Engineer
Netflix
Re: [VOTE] Release Apache Parquet 1.9.0 RC1
Posted by Uwe Korn <uw...@xhochy.com>.
Hello Ryan,
sadly I have failing tests with the RC. Seems like they are locale
dependent ("," vs "."). Rerunning with LANG=en_US.UTF-8 did sadly not
solve this, is there some other magic I need to provide to switch JVM
locals?
% cat
parquet-column/target/surefire-reports/org.apache.parquet.column.statistics.TestStatistics.txt
-------------------------------------------------------------------------------
Test set: org.apache.parquet.column.statistics.TestStatistics
-------------------------------------------------------------------------------
Tests run: 9, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.024
sec <<< FAILURE!
testFloatMinMax(org.apache.parquet.column.statistics.TestStatistics)
Time elapsed: 0.01 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<min: 0[,00010, max: 553,]59998,
num_nulls: 0> but was:<min: 0[.00010, max: 553.]59998, num_nulls: 0>
at org.junit.Assert.assertEquals(Assert.java:125)
at org.junit.Assert.assertEquals(Assert.java:147)
at
org.apache.parquet.column.statistics.TestStatistics.testFloatMinMax(TestStatistics.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
testDoubleMinMax(org.apache.parquet.column.statistics.TestStatistics)
Time elapsed: 0 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<min: 0[,00001, max: 944,]50000,
num_nulls: 0> but was:<min: 0[.00001, max: 944.]50000, num_nulls: 0>
at org.junit.Assert.assertEquals(Assert.java:125)
at org.junit.Assert.assertEquals(Assert.java:147)
at
org.apache.parquet.column.statistics.TestStatistics.testDoubleMinMax(TestStatistics.java:296)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
On 13.10.16 19:53, Ryan Blue wrote:
> Hi everyone,
>
> I propose the following RC to be released as official Apache Parquet 1.9.0
> release.
>
> The commit id is 2a99abf784cb6e76160d49506ea87581a2256021
> * This corresponds to the tag: apache-parquet-1.9.0
> * https://github.com/apache/parquet-mr/tree/2a99abf7
> *
> https://git-wip-us.apache.org/repos/asf/projects/repo?p=parquet-mr.git&a=commit&h=2a99abf7
>
> The release tarball, signature, and checksums are here:
> * https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.9.0-rc1
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/dev/parquet/KEYS
>
> Binary artifacts are staged in Nexus here:
> *
> https://repository.apache.org/content/groups/staging/org/apache/parquet/parquet/
>
> This release includes:
> * Support for Hadoop's ByteBuffer read API
> * Dictionary-based row group filters
> * Cascading3 integration
> * Predicate push-down for Pig
> * Decimal, timestamp, and date support in Avro
> * Numerous bug fixes
>
> Please verify, test, and vote by Monday, 17 October 2016.
>
> [ ] +1 Release this as Apache Parquet 1.9.0
> [ ] +0
> [ ] -1 Do not release this because...
>
>