You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Chinmay Kulkarni <ch...@gmail.com> on 2019/11/06 18:21:39 UTC
Precommit Jenkins build is broken
The Precommit build has been broken for a while now. Recent builds are
failing with
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on
project phoenix-core: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.20:test failed:
java.lang.NoClassDefFoundError: java/sql/SQLException: java.sql.SQLException
I am unaware of how to fix this or even get access to the build
configurations/build machine itself. We are committing patches without
getting green builds and that is not a good idea (but we don't have an
option until the build is fixed either).
Does anyone know how to debug this?
--
Chinmay Kulkarni
Re: Precommit Jenkins build is broken
Posted by Chinmay Kulkarni <ch...@gmail.com>.
Thanks Josh! I agree that backporting all relevant JDK11 related Jiras
seems like extra work we don't immediate need to do at the moment. I have
opened PHOENIX-5569 <https://issues.apache.org/jira/browse/PHOENIX-5569> to
just bump up the maven surefire plugin version in 4.x branches.
On Sun, Nov 10, 2019 at 6:01 AM Josh Elser <el...@apache.org> wrote:
> Great digging, Chinmay! This is an awesome summary email. I really
> appreciate you taking the time to write it up.
>
> Definitely, bumping up surefire is the right way to go about fixing this.
>
> I think backporting all of Istvan's JDK11 precursor work should depend
> on someone with an interest to run Phoenix4.x on HBase1x. with JDK11. If
> no one has aspirations to do this, it just potentially creates more work :)
>
> On 11/7/19 5:03 PM, Chinmay Kulkarni wrote:
> > Okay, I finally got some time to debug this. This is what seems to be the
> > problem:
> >
> > 1. In our master branch, the maven-surefire plugin version is 2.22.2
> > <
> https://github.com/apache/phoenix/blob/70baf4c6805eb46420dd1fbaf71a093ab7d84a8b/pom.xml#L126
> >
> > and
> > in our 4.x branches, it is 2.20
> > <
> https://github.com/apache/phoenix/blob/69448a845e843628558e17032287fe9e2bb6c5cf/pom.xml#L127
> >.
> > Looks like PHOENIX-5459
> > <https://issues.apache.org/jira/browse/PHOENIX-5459> was not
> applied to
> > 4.x branches. Was this just a miss, or did we not apply to 4.x
> branches on
> > purpose?
> > 2. In some of our build hosts, we use the OpenJDK Runtime Environment
> > (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3), and in others we use
> > Java(TM) SE Runtime Environment (build 1.8.0_191-b12). See
> JDK-11-build
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull>
> > vs JDK-8-build
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull>
> > .
> > 3. The "java.lang.NoClassDefFoundError: java/sql/SQLException:
> > java.sql.SQLException" was a known issue (SUREFIRE-1265
> > <https://issues.apache.org/jira/browse/SUREFIRE-1265>) for Java
> runtime
> > 1.9+ and was fixed in version 2.20.1.
> > 4. The builds that fail with the above error are only for 4.x
> branches +
> > JDK 11. Here are recent builds satisfying this combination (old
> surefire
> > and new Java version) and failing with this error: 3095
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3095/consoleFull>,
> > 3094
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3094/consoleFull>,
> > 3093
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3093/consoleFull>,
> > 3091
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull>.
> > 5. Builds on the master branch (surefire plugin 2.22.2) + JDK 11, do
> not
> > fail with this error, for ex: 3089
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3089/consoleFull>,
> > 3088
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3088/consoleFull>,
> > 3092
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3092/consoleFull> .
> > Also, builds on 4.x branches (surefire plugin 2.20) + JDK 8 also do
> not
> > fail with this error, for ex: 3100
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull>,
> > 3099
> > <
> https://builds.apache.org/job/PreCommit-PHOENIX-Build/3099/consoleFull>
> >
> > It looks like the build issue can be fixed if we apply PHOENIX-5459 to
> 4.x
> > branches since that would bump up the surefire plugin version + support
> > running tests on JDK11. If PHOENIX-5459 is not be included in 4.x
> branches
> > for some reason, we should at least bump up the surefire plugin version
> on
> > 4.x branches.
> > Thoughts?
> >
> > On Thu, Nov 7, 2019 at 9:35 AM Josh Elser <el...@apache.org> wrote:
> >
> >> What have you been able to find so far, Chinmay?
> >>
> >> For example,
> >>
> >> * Does every PreCommit build fail? If not, are we limited to a certain
> >> host(s) that causes failure?
> >> * Do local builds still work?
> >> * What's different (maven+java versions) from the build hosts to local
> >> builds?
> >>
> >> With coordination from Infra, I think it is possible (or was possible in
> >> the past) to get SSH access to the build workers, but it's rare that
> >> this is actually required.
> >>
> >> On 11/6/19 1:21 PM, Chinmay Kulkarni wrote:
> >>> The Precommit build has been broken for a while now. Recent builds are
> >>> failing with
> >>>
> >>> [ERROR] Failed to execute goal
> >>> org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test)
> >> on
> >>> project phoenix-core: Execution default-test of goal
> >>> org.apache.maven.plugins:maven-surefire-plugin:2.20:test failed:
> >>> java.lang.NoClassDefFoundError: java/sql/SQLException:
> >> java.sql.SQLException
> >>>
> >>> I am unaware of how to fix this or even get access to the build
> >>> configurations/build machine itself. We are committing patches without
> >>> getting green builds and that is not a good idea (but we don't have an
> >>> option until the build is fixed either).
> >>>
> >>> Does anyone know how to debug this?
> >>>
> >>
> >
> >
>
--
Chinmay Kulkarni
Re: Precommit Jenkins build is broken
Posted by Josh Elser <el...@apache.org>.
Great digging, Chinmay! This is an awesome summary email. I really
appreciate you taking the time to write it up.
Definitely, bumping up surefire is the right way to go about fixing this.
I think backporting all of Istvan's JDK11 precursor work should depend
on someone with an interest to run Phoenix4.x on HBase1x. with JDK11. If
no one has aspirations to do this, it just potentially creates more work :)
On 11/7/19 5:03 PM, Chinmay Kulkarni wrote:
> Okay, I finally got some time to debug this. This is what seems to be the
> problem:
>
> 1. In our master branch, the maven-surefire plugin version is 2.22.2
> <https://github.com/apache/phoenix/blob/70baf4c6805eb46420dd1fbaf71a093ab7d84a8b/pom.xml#L126>
> and
> in our 4.x branches, it is 2.20
> <https://github.com/apache/phoenix/blob/69448a845e843628558e17032287fe9e2bb6c5cf/pom.xml#L127>.
> Looks like PHOENIX-5459
> <https://issues.apache.org/jira/browse/PHOENIX-5459> was not applied to
> 4.x branches. Was this just a miss, or did we not apply to 4.x branches on
> purpose?
> 2. In some of our build hosts, we use the OpenJDK Runtime Environment
> (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3), and in others we use
> Java(TM) SE Runtime Environment (build 1.8.0_191-b12). See JDK-11-build
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull>
> vs JDK-8-build
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull>
> .
> 3. The "java.lang.NoClassDefFoundError: java/sql/SQLException:
> java.sql.SQLException" was a known issue (SUREFIRE-1265
> <https://issues.apache.org/jira/browse/SUREFIRE-1265>) for Java runtime
> 1.9+ and was fixed in version 2.20.1.
> 4. The builds that fail with the above error are only for 4.x branches +
> JDK 11. Here are recent builds satisfying this combination (old surefire
> and new Java version) and failing with this error: 3095
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3095/consoleFull>,
> 3094
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3094/consoleFull>,
> 3093
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3093/consoleFull>,
> 3091
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull>.
> 5. Builds on the master branch (surefire plugin 2.22.2) + JDK 11, do not
> fail with this error, for ex: 3089
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3089/consoleFull>,
> 3088
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3088/consoleFull>,
> 3092
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3092/consoleFull> .
> Also, builds on 4.x branches (surefire plugin 2.20) + JDK 8 also do not
> fail with this error, for ex: 3100
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull>,
> 3099
> <https://builds.apache.org/job/PreCommit-PHOENIX-Build/3099/consoleFull>
>
> It looks like the build issue can be fixed if we apply PHOENIX-5459 to 4.x
> branches since that would bump up the surefire plugin version + support
> running tests on JDK11. If PHOENIX-5459 is not be included in 4.x branches
> for some reason, we should at least bump up the surefire plugin version on
> 4.x branches.
> Thoughts?
>
> On Thu, Nov 7, 2019 at 9:35 AM Josh Elser <el...@apache.org> wrote:
>
>> What have you been able to find so far, Chinmay?
>>
>> For example,
>>
>> * Does every PreCommit build fail? If not, are we limited to a certain
>> host(s) that causes failure?
>> * Do local builds still work?
>> * What's different (maven+java versions) from the build hosts to local
>> builds?
>>
>> With coordination from Infra, I think it is possible (or was possible in
>> the past) to get SSH access to the build workers, but it's rare that
>> this is actually required.
>>
>> On 11/6/19 1:21 PM, Chinmay Kulkarni wrote:
>>> The Precommit build has been broken for a while now. Recent builds are
>>> failing with
>>>
>>> [ERROR] Failed to execute goal
>>> org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test)
>> on
>>> project phoenix-core: Execution default-test of goal
>>> org.apache.maven.plugins:maven-surefire-plugin:2.20:test failed:
>>> java.lang.NoClassDefFoundError: java/sql/SQLException:
>> java.sql.SQLException
>>>
>>> I am unaware of how to fix this or even get access to the build
>>> configurations/build machine itself. We are committing patches without
>>> getting green builds and that is not a good idea (but we don't have an
>>> option until the build is fixed either).
>>>
>>> Does anyone know how to debug this?
>>>
>>
>
>
Re: Precommit Jenkins build is broken
Posted by Chinmay Kulkarni <ch...@gmail.com>.
Okay, I finally got some time to debug this. This is what seems to be the
problem:
1. In our master branch, the maven-surefire plugin version is 2.22.2
<https://github.com/apache/phoenix/blob/70baf4c6805eb46420dd1fbaf71a093ab7d84a8b/pom.xml#L126>
and
in our 4.x branches, it is 2.20
<https://github.com/apache/phoenix/blob/69448a845e843628558e17032287fe9e2bb6c5cf/pom.xml#L127>.
Looks like PHOENIX-5459
<https://issues.apache.org/jira/browse/PHOENIX-5459> was not applied to
4.x branches. Was this just a miss, or did we not apply to 4.x branches on
purpose?
2. In some of our build hosts, we use the OpenJDK Runtime Environment
(build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3), and in others we use
Java(TM) SE Runtime Environment (build 1.8.0_191-b12). See JDK-11-build
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull>
vs JDK-8-build
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull>
.
3. The "java.lang.NoClassDefFoundError: java/sql/SQLException:
java.sql.SQLException" was a known issue (SUREFIRE-1265
<https://issues.apache.org/jira/browse/SUREFIRE-1265>) for Java runtime
1.9+ and was fixed in version 2.20.1.
4. The builds that fail with the above error are only for 4.x branches +
JDK 11. Here are recent builds satisfying this combination (old surefire
and new Java version) and failing with this error: 3095
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3095/consoleFull>,
3094
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3094/consoleFull>,
3093
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3093/consoleFull>,
3091
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3091/consoleFull>.
5. Builds on the master branch (surefire plugin 2.22.2) + JDK 11, do not
fail with this error, for ex: 3089
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3089/consoleFull>,
3088
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3088/consoleFull>,
3092
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3092/consoleFull> .
Also, builds on 4.x branches (surefire plugin 2.20) + JDK 8 also do not
fail with this error, for ex: 3100
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100/consoleFull>,
3099
<https://builds.apache.org/job/PreCommit-PHOENIX-Build/3099/consoleFull>
It looks like the build issue can be fixed if we apply PHOENIX-5459 to 4.x
branches since that would bump up the surefire plugin version + support
running tests on JDK11. If PHOENIX-5459 is not be included in 4.x branches
for some reason, we should at least bump up the surefire plugin version on
4.x branches.
Thoughts?
On Thu, Nov 7, 2019 at 9:35 AM Josh Elser <el...@apache.org> wrote:
> What have you been able to find so far, Chinmay?
>
> For example,
>
> * Does every PreCommit build fail? If not, are we limited to a certain
> host(s) that causes failure?
> * Do local builds still work?
> * What's different (maven+java versions) from the build hosts to local
> builds?
>
> With coordination from Infra, I think it is possible (or was possible in
> the past) to get SSH access to the build workers, but it's rare that
> this is actually required.
>
> On 11/6/19 1:21 PM, Chinmay Kulkarni wrote:
> > The Precommit build has been broken for a while now. Recent builds are
> > failing with
> >
> > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test)
> on
> > project phoenix-core: Execution default-test of goal
> > org.apache.maven.plugins:maven-surefire-plugin:2.20:test failed:
> > java.lang.NoClassDefFoundError: java/sql/SQLException:
> java.sql.SQLException
> >
> > I am unaware of how to fix this or even get access to the build
> > configurations/build machine itself. We are committing patches without
> > getting green builds and that is not a good idea (but we don't have an
> > option until the build is fixed either).
> >
> > Does anyone know how to debug this?
> >
>
--
Chinmay Kulkarni
Re: Precommit Jenkins build is broken
Posted by Josh Elser <el...@apache.org>.
What have you been able to find so far, Chinmay?
For example,
* Does every PreCommit build fail? If not, are we limited to a certain
host(s) that causes failure?
* Do local builds still work?
* What's different (maven+java versions) from the build hosts to local
builds?
With coordination from Infra, I think it is possible (or was possible in
the past) to get SSH access to the build workers, but it's rare that
this is actually required.
On 11/6/19 1:21 PM, Chinmay Kulkarni wrote:
> The Precommit build has been broken for a while now. Recent builds are
> failing with
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on
> project phoenix-core: Execution default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20:test failed:
> java.lang.NoClassDefFoundError: java/sql/SQLException: java.sql.SQLException
>
> I am unaware of how to fix this or even get access to the build
> configurations/build machine itself. We are committing patches without
> getting green builds and that is not a good idea (but we don't have an
> option until the build is fixed either).
>
> Does anyone know how to debug this?
>