You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Mass Dosage <ma...@gmail.com> on 2020/06/05 12:13:10 UTC
CI for Iceberg
Hello all,
I just wanted to know if there is any CI set up for Iceberg? I noticed that
if I pull the current master branch I get failing tests (see below for
stack traces, Ryan - we talked about this last night but it's still
happening). So this made me wonder why there isn't some CI set up to check
that every PR actually successfully passes the build. I noticed there is a
.travis.yml but this doesn't seem to run the gradle build. Should we add
that or create a GitHub action to do this? I think it would be a really
good safety net to have to reduce the likelihood of broken code and tests
getting into master.
Below are the two tests which are currently failing for me:
> Task :iceberg-data:test
org.apache.iceberg.data.TestLocalScan > testFilterWithDateAndTimestamp[1]
FAILED
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.iceberg.data.TestLocalScan.testFilterWithDateAndTimestamp(TestLocalScan.java:486)
org.apache.iceberg.data.TestMetricsRowGroupFilterTypes > testEq[20] FAILED
java.lang.AssertionError: Should read: value is in the row group:
2018-06-29
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.iceberg.data.TestMetricsRowGroupFilterTypes.testEq(TestMetricsRowGroupFilterTypes.java:284)
211 tests completed, 2 failed, 6 skipped
> Task :iceberg-data:test FAILED
Thanks,
Adrian
Re: CI for Iceberg
Posted by Mass Dosage <ma...@gmail.com>.
Ok, that makes sense.
I pulled master a few hours ago and was still getting those errors.
Christine also had one of them fail. Could it be timezone related? I
haven't had a chance to look into the failures in detail yet, will try
again on Monday.
On Fri, 5 Jun 2020, 17:15 Ryan Blue, <rb...@netflix.com.invalid> wrote:
> These can creep in because pull requests are tested against the current
> master when the test starts and not re-tested when master changes. So you
> can merge two passing pull requests and end up with a failure in master. We
> generally fix those pretty quickly because all subsequent pull requests
> will fail.
>
> I think what you ran into here was an issue with a new test that pushed
> down timestamp filters and an ORC timestamp correctness bug in stats. That
> should be fixed now. The current master builds are green.
>
> On Fri, Jun 5, 2020 at 5:38 AM Mass Dosage <ma...@gmail.com> wrote:
>
>> I now looked better and see that the Travis file does actually build
>> Iceberg ;) I'm still curious how something managed to get merged into
>> master while failing the tests though?
>>
>> On Fri, 5 Jun 2020 at 13:13, Mass Dosage <ma...@gmail.com> wrote:
>>
>>> Hello all,
>>>
>>> I just wanted to know if there is any CI set up for Iceberg? I noticed
>>> that if I pull the current master branch I get failing tests (see below for
>>> stack traces, Ryan - we talked about this last night but it's still
>>> happening). So this made me wonder why there isn't some CI set up to check
>>> that every PR actually successfully passes the build. I noticed there is a
>>> .travis.yml but this doesn't seem to run the gradle build. Should we add
>>> that or create a GitHub action to do this? I think it would be a really
>>> good safety net to have to reduce the likelihood of broken code and tests
>>> getting into master.
>>>
>>> Below are the two tests which are currently failing for me:
>>>
>>> > Task :iceberg-data:test
>>>
>>> org.apache.iceberg.data.TestLocalScan >
>>> testFilterWithDateAndTimestamp[1] FAILED
>>> java.lang.AssertionError
>>> at org.junit.Assert.fail(Assert.java:86)
>>> at org.junit.Assert.assertTrue(Assert.java:41)
>>> at org.junit.Assert.assertTrue(Assert.java:52)
>>> at
>>> org.apache.iceberg.data.TestLocalScan.testFilterWithDateAndTimestamp(TestLocalScan.java:486)
>>>
>>> org.apache.iceberg.data.TestMetricsRowGroupFilterTypes > testEq[20]
>>> FAILED
>>> java.lang.AssertionError: Should read: value is in the row group:
>>> 2018-06-29
>>> at org.junit.Assert.fail(Assert.java:88)
>>> at org.junit.Assert.assertTrue(Assert.java:41)
>>> at
>>> org.apache.iceberg.data.TestMetricsRowGroupFilterTypes.testEq(TestMetricsRowGroupFilterTypes.java:284)
>>>
>>> 211 tests completed, 2 failed, 6 skipped
>>>
>>> > Task :iceberg-data:test FAILED
>>>
>>> Thanks,
>>>
>>> Adrian
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
Re: CI for Iceberg
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
These can creep in because pull requests are tested against the current
master when the test starts and not re-tested when master changes. So you
can merge two passing pull requests and end up with a failure in master. We
generally fix those pretty quickly because all subsequent pull requests
will fail.
I think what you ran into here was an issue with a new test that pushed
down timestamp filters and an ORC timestamp correctness bug in stats. That
should be fixed now. The current master builds are green.
On Fri, Jun 5, 2020 at 5:38 AM Mass Dosage <ma...@gmail.com> wrote:
> I now looked better and see that the Travis file does actually build
> Iceberg ;) I'm still curious how something managed to get merged into
> master while failing the tests though?
>
> On Fri, 5 Jun 2020 at 13:13, Mass Dosage <ma...@gmail.com> wrote:
>
>> Hello all,
>>
>> I just wanted to know if there is any CI set up for Iceberg? I noticed
>> that if I pull the current master branch I get failing tests (see below for
>> stack traces, Ryan - we talked about this last night but it's still
>> happening). So this made me wonder why there isn't some CI set up to check
>> that every PR actually successfully passes the build. I noticed there is a
>> .travis.yml but this doesn't seem to run the gradle build. Should we add
>> that or create a GitHub action to do this? I think it would be a really
>> good safety net to have to reduce the likelihood of broken code and tests
>> getting into master.
>>
>> Below are the two tests which are currently failing for me:
>>
>> > Task :iceberg-data:test
>>
>> org.apache.iceberg.data.TestLocalScan > testFilterWithDateAndTimestamp[1]
>> FAILED
>> java.lang.AssertionError
>> at org.junit.Assert.fail(Assert.java:86)
>> at org.junit.Assert.assertTrue(Assert.java:41)
>> at org.junit.Assert.assertTrue(Assert.java:52)
>> at
>> org.apache.iceberg.data.TestLocalScan.testFilterWithDateAndTimestamp(TestLocalScan.java:486)
>>
>> org.apache.iceberg.data.TestMetricsRowGroupFilterTypes > testEq[20] FAILED
>> java.lang.AssertionError: Should read: value is in the row group:
>> 2018-06-29
>> at org.junit.Assert.fail(Assert.java:88)
>> at org.junit.Assert.assertTrue(Assert.java:41)
>> at
>> org.apache.iceberg.data.TestMetricsRowGroupFilterTypes.testEq(TestMetricsRowGroupFilterTypes.java:284)
>>
>> 211 tests completed, 2 failed, 6 skipped
>>
>> > Task :iceberg-data:test FAILED
>>
>> Thanks,
>>
>> Adrian
>>
>
--
Ryan Blue
Software Engineer
Netflix
Re: CI for Iceberg
Posted by Mass Dosage <ma...@gmail.com>.
I now looked better and see that the Travis file does actually build
Iceberg ;) I'm still curious how something managed to get merged into
master while failing the tests though?
On Fri, 5 Jun 2020 at 13:13, Mass Dosage <ma...@gmail.com> wrote:
> Hello all,
>
> I just wanted to know if there is any CI set up for Iceberg? I noticed
> that if I pull the current master branch I get failing tests (see below for
> stack traces, Ryan - we talked about this last night but it's still
> happening). So this made me wonder why there isn't some CI set up to check
> that every PR actually successfully passes the build. I noticed there is a
> .travis.yml but this doesn't seem to run the gradle build. Should we add
> that or create a GitHub action to do this? I think it would be a really
> good safety net to have to reduce the likelihood of broken code and tests
> getting into master.
>
> Below are the two tests which are currently failing for me:
>
> > Task :iceberg-data:test
>
> org.apache.iceberg.data.TestLocalScan > testFilterWithDateAndTimestamp[1]
> FAILED
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.iceberg.data.TestLocalScan.testFilterWithDateAndTimestamp(TestLocalScan.java:486)
>
> org.apache.iceberg.data.TestMetricsRowGroupFilterTypes > testEq[20] FAILED
> java.lang.AssertionError: Should read: value is in the row group:
> 2018-06-29
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.iceberg.data.TestMetricsRowGroupFilterTypes.testEq(TestMetricsRowGroupFilterTypes.java:284)
>
> 211 tests completed, 2 failed, 6 skipped
>
> > Task :iceberg-data:test FAILED
>
> Thanks,
>
> Adrian
>