You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Andrew Musselman <an...@gmail.com> on 2017/04/25 01:46:53 UTC
Failed building hadoop-deb
From master on Ubuntu 16.04; is there an issue using the ubuntu-14.04
docker image?
$ sudo docker run -u jenkins --rm --workdir /ws -v `pwd`:/ws
bigtop/slaves:trunk-ubuntu-14.04 bash -l -c './gradlew hadoop-deb
bigtop-utils-deb zookeeper-deb'
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [
20.159 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [
2.572 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [
2.685 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [
3.150 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [
0.077 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [
1.082 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [
4.444 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [
44.601 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [
19.104 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [
2.294 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [01:16
min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [
2.472 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [
12.127 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [
0.028 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [01:09
min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [
16.642 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [
7.189 s]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [
2.218 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [
0.020 s]
[INFO] hadoop-yarn ........................................ SUCCESS [
0.017 s]
[INFO] hadoop-yarn-api .................................... SUCCESS [
15.536 s]
[INFO] hadoop-yarn-common ................................. SUCCESS [
26.133 s]
[INFO] hadoop-yarn-server ................................. SUCCESS [
0.018 s]
[INFO] hadoop-yarn-server-common .......................... SUCCESS [
4.366 s]
[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [
7.952 s]
[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [
1.665 s]
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [
2.927 s]
[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [
8.022 s]
[INFO] hadoop-yarn-server-tests ........................... FAILURE [
4.129 s]
[INFO] hadoop-yarn-client ................................. SKIPPED
[INFO] hadoop-yarn-server-sharedcachemanager .............. SKIPPED
[INFO] hadoop-yarn-applications ........................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
[INFO] hadoop-yarn-site ................................... SKIPPED
[INFO] hadoop-yarn-registry ............................... SKIPPED
[INFO] hadoop-yarn-project ................................ SKIPPED
[INFO] hadoop-mapreduce-client ............................ SKIPPED
[INFO] hadoop-mapreduce-client-core ....................... SKIPPED
[INFO] hadoop-mapreduce-client-common ..................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
[INFO] hadoop-mapreduce-client-app ........................ SKIPPED
[INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
[INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] hadoop-mapreduce ................................... SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Azure support ........................ SKIPPED
[INFO] Apache Hadoop Client ............................... SKIPPED
[INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 06:04 min
[INFO] Finished at: 2017-04-25T01:41:27+00:00
[INFO] Final Memory: 214M/3445M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (depcheck) on
project hadoop-yarn-server-tests: Some Enforcer rules have failed. Look
above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR] mvn <goals> -rf :hadoop-yarn-server-tests
make[1]: *** [override_dh_auto_build] Error 1
make[1]: Leaving directory `/ws/output/hadoop/hadoop-2.7.3'
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2
debuild: fatal error at line 1364:
dpkg-buildpackage -rfakeroot -D -us -uc -b
failed
:hadoop-deb FAILED
FAILURE: Build failed with an exception.
* Where:
Script '/ws/packages.gradle' line: 341
* What went wrong:
Execution failed for task ':hadoop-deb'.
> Process 'command 'debuild'' finished with non-zero exit value 29
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or
--debug option to get more log output.
BUILD FAILED
Total time: 6 mins 44.819 secs
Re: Failed building hadoop-deb
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
Strictly speaking we no longer support 14.04
Thanks,
Roman.
On Mon, Apr 24, 2017 at 6:46 PM, Andrew Musselman
<an...@gmail.com> wrote:
> From master on Ubuntu 16.04; is there an issue using the ubuntu-14.04
> docker image?
>
> $ sudo docker run -u jenkins --rm --workdir /ws -v `pwd`:/ws
> bigtop/slaves:trunk-ubuntu-14.04 bash -l -c './gradlew hadoop-deb
> bigtop-utils-deb zookeeper-deb'
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................. SUCCESS [
> 20.159 s]
> [INFO] Apache Hadoop Build Tools .......................... SUCCESS [
> 2.572 s]
> [INFO] Apache Hadoop Project POM .......................... SUCCESS [
> 2.685 s]
> [INFO] Apache Hadoop Annotations .......................... SUCCESS [
> 3.150 s]
> [INFO] Apache Hadoop Assemblies ........................... SUCCESS [
> 0.077 s]
> [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [
> 1.082 s]
> [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [
> 4.444 s]
> [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [
> 44.601 s]
> [INFO] Apache Hadoop Auth ................................. SUCCESS [
> 19.104 s]
> [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [
> 2.294 s]
> [INFO] Apache Hadoop Common ............................... SUCCESS [01:16
> min]
> [INFO] Apache Hadoop NFS .................................. SUCCESS [
> 2.472 s]
> [INFO] Apache Hadoop KMS .................................. SUCCESS [
> 12.127 s]
> [INFO] Apache Hadoop Common Project ....................... SUCCESS [
> 0.028 s]
> [INFO] Apache Hadoop HDFS ................................. SUCCESS [01:09
> min]
> [INFO] Apache Hadoop HttpFS ............................... SUCCESS [
> 16.642 s]
> [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [
> 7.189 s]
> [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [
> 2.218 s]
> [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [
> 0.020 s]
> [INFO] hadoop-yarn ........................................ SUCCESS [
> 0.017 s]
> [INFO] hadoop-yarn-api .................................... SUCCESS [
> 15.536 s]
> [INFO] hadoop-yarn-common ................................. SUCCESS [
> 26.133 s]
> [INFO] hadoop-yarn-server ................................. SUCCESS [
> 0.018 s]
> [INFO] hadoop-yarn-server-common .......................... SUCCESS [
> 4.366 s]
> [INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [
> 7.952 s]
> [INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [
> 1.665 s]
> [INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [
> 2.927 s]
> [INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [
> 8.022 s]
> [INFO] hadoop-yarn-server-tests ........................... FAILURE [
> 4.129 s]
> [INFO] hadoop-yarn-client ................................. SKIPPED
> [INFO] hadoop-yarn-server-sharedcachemanager .............. SKIPPED
> [INFO] hadoop-yarn-applications ........................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
> [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
> [INFO] hadoop-yarn-site ................................... SKIPPED
> [INFO] hadoop-yarn-registry ............................... SKIPPED
> [INFO] hadoop-yarn-project ................................ SKIPPED
> [INFO] hadoop-mapreduce-client ............................ SKIPPED
> [INFO] hadoop-mapreduce-client-core ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-common ..................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
> [INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
> [INFO] hadoop-mapreduce ................................... SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
> [INFO] Apache Hadoop Archives ............................. SKIPPED
> [INFO] Apache Hadoop Rumen ................................ SKIPPED
> [INFO] Apache Hadoop Gridmix .............................. SKIPPED
> [INFO] Apache Hadoop Data Join ............................ SKIPPED
> [INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
> [INFO] Apache Hadoop Extras ............................... SKIPPED
> [INFO] Apache Hadoop Pipes ................................ SKIPPED
> [INFO] Apache Hadoop OpenStack support .................... SKIPPED
> [INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
> [INFO] Apache Hadoop Azure support ........................ SKIPPED
> [INFO] Apache Hadoop Client ............................... SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
> [INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
> [INFO] Apache Hadoop Tools Dist ........................... SKIPPED
> [INFO] Apache Hadoop Tools ................................ SKIPPED
> [INFO] Apache Hadoop Distribution ......................... SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 06:04 min
> [INFO] Finished at: 2017-04-25T01:41:27+00:00
> [INFO] Final Memory: 214M/3445M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (depcheck) on
> project hadoop-yarn-server-tests: Some Enforcer rules have failed. Look
> above for specific messages explaining why the rule failed. -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :hadoop-yarn-server-tests
> make[1]: *** [override_dh_auto_build] Error 1
> make[1]: Leaving directory `/ws/output/hadoop/hadoop-2.7.3'
> make: *** [build] Error 2
> dpkg-buildpackage: error: debian/rules build gave error exit status 2
> debuild: fatal error at line 1364:
> dpkg-buildpackage -rfakeroot -D -us -uc -b
> failed
> :hadoop-deb FAILED
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Script '/ws/packages.gradle' line: 341
>
> * What went wrong:
> Execution failed for task ':hadoop-deb'.
>> Process 'command 'debuild'' finished with non-zero exit value 29
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output.
>
> BUILD FAILED
>
> Total time: 6 mins 44.819 secs
Re: Failed building hadoop-deb
Posted by Konstantin Boudnik <co...@apache.org>.
I would hesitant as well. A part of the initial design of the Bigtop
was to guarantee the dependencies consistency through out the stack.
Yes, it didn't work all the time because different components were
doing funny business and bringing in a lot of transitional shit.
However, we never did this on purpose. So, I'd rather don't have this option.
--
Take care,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Sat, Apr 29, 2017 at 10:04 PM, Olaf Flebbe <of...@oflebbe.de> wrote:
> Hi Roman,
>
> you remember we are symlinking jars? esp. zookeeper. It was feeling weird to me at that time, now it will be desaster area. (because of netty dependency changes in zookeeper).
>
> If we use old client we have to remove the symlink functionality in hadoop build code as well.
>
> Right now i am hesitant to have different zookeeper versions in bigtop (client and server).
>
> olaf
>
>> Am 30.04.2017 um 04:33 schrieb Roman Shaposhnik <ro...@shaposhnik.org>:
>>
>>> On Tue, Apr 25, 2017 at 11:07 PM, Olaf Flebbe <of...@oflebbe.de> wrote:
>>> Hi,
>>>
>>> There may be issues with netty and zookeeper within hadoop depending on each
>>> other.
>>> https://issues.apache.org/jira/browse/HADOOP-12928
>>> https://issues.apache.org/jira/browse/HADOOP-13413
>>>
>>> We have a dependency on curator as well
>>> https://issues.apache.org/jira/browse/HADOOP-14187 ( only in hadoop 3 alpha)
>>>
>>> I didn't see all the mess when I initialy reviewed b00ac0936
>>>
>>> I think we should roll back commit b00ac0936 to zookeeper-3.4.6 T I have
>>> to revoke my LGTM ;-(
>>
>> One option is to have Zookeeper 3.4.6 as a service, but relax Hadoop dependency
>> back to the default. Typically old clients work pretty well with new Zookeeper
>> servers.
>>
>> What do y'all think?
>>
>> Thanks,
>> Roman.
Re: Failed building hadoop-deb
Posted by Evans Ye <ev...@apache.org>.
Olaf:
Thanks for spotting on the CI issue.
Roman:
Although that's doable, I think that adds complexity and hidden knowledge
in our pipeline. I'm a +1 to roll it back until upstream fixed the problem.
Regarding to CI, another problem is if it's always failing, then no body
will look into it. That's why I'd like to separate out ARCH64. I'll try to
tweak the sleep time and see if it gets around with PPC64LE. Meanwhile, I
think we also need to add deployment matrix in CI as well, so that the
issue like HBase won't start on PPC64LE won't happened again. I'll spend
some effort on that.
What's your plan on the CI improvements? I'd love to know.
Evans
2017-04-30 13:04 GMT+08:00 Olaf Flebbe <of...@oflebbe.de>:
> Hi Roman,
>
> you remember we are symlinking jars? esp. zookeeper. It was feeling weird
> to me at that time, now it will be desaster area. (because of netty
> dependency changes in zookeeper).
>
> If we use old client we have to remove the symlink functionality in hadoop
> build code as well.
>
> Right now i am hesitant to have different zookeeper versions in bigtop
> (client and server).
>
> olaf
>
> > Am 30.04.2017 um 04:33 schrieb Roman Shaposhnik <ro...@shaposhnik.org>:
> >
> >> On Tue, Apr 25, 2017 at 11:07 PM, Olaf Flebbe <of...@oflebbe.de> wrote:
> >> Hi,
> >>
> >> There may be issues with netty and zookeeper within hadoop depending on
> each
> >> other.
> >> https://issues.apache.org/jira/browse/HADOOP-12928
> >> https://issues.apache.org/jira/browse/HADOOP-13413
> >>
> >> We have a dependency on curator as well
> >> https://issues.apache.org/jira/browse/HADOOP-14187 ( only in hadoop 3
> alpha)
> >>
> >> I didn't see all the mess when I initialy reviewed b00ac0936
> >>
> >> I think we should roll back commit b00ac0936 to zookeeper-3.4.6 T I
> have
> >> to revoke my LGTM ;-(
> >
> > One option is to have Zookeeper 3.4.6 as a service, but relax Hadoop
> dependency
> > back to the default. Typically old clients work pretty well with new
> Zookeeper
> > servers.
> >
> > What do y'all think?
> >
> > Thanks,
> > Roman.
>
Re: Failed building hadoop-deb
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi Roman,
you remember we are symlinking jars? esp. zookeeper. It was feeling weird to me at that time, now it will be desaster area. (because of netty dependency changes in zookeeper).
If we use old client we have to remove the symlink functionality in hadoop build code as well.
Right now i am hesitant to have different zookeeper versions in bigtop (client and server).
olaf
> Am 30.04.2017 um 04:33 schrieb Roman Shaposhnik <ro...@shaposhnik.org>:
>
>> On Tue, Apr 25, 2017 at 11:07 PM, Olaf Flebbe <of...@oflebbe.de> wrote:
>> Hi,
>>
>> There may be issues with netty and zookeeper within hadoop depending on each
>> other.
>> https://issues.apache.org/jira/browse/HADOOP-12928
>> https://issues.apache.org/jira/browse/HADOOP-13413
>>
>> We have a dependency on curator as well
>> https://issues.apache.org/jira/browse/HADOOP-14187 ( only in hadoop 3 alpha)
>>
>> I didn't see all the mess when I initialy reviewed b00ac0936
>>
>> I think we should roll back commit b00ac0936 to zookeeper-3.4.6 T I have
>> to revoke my LGTM ;-(
>
> One option is to have Zookeeper 3.4.6 as a service, but relax Hadoop dependency
> back to the default. Typically old clients work pretty well with new Zookeeper
> servers.
>
> What do y'all think?
>
> Thanks,
> Roman.
Re: Failed building hadoop-deb
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Tue, Apr 25, 2017 at 11:07 PM, Olaf Flebbe <of...@oflebbe.de> wrote:
> Hi,
>
> There may be issues with netty and zookeeper within hadoop depending on each
> other.
> https://issues.apache.org/jira/browse/HADOOP-12928
> https://issues.apache.org/jira/browse/HADOOP-13413
>
> We have a dependency on curator as well
> https://issues.apache.org/jira/browse/HADOOP-14187 ( only in hadoop 3 alpha)
>
> I didn't see all the mess when I initialy reviewed b00ac0936
>
> I think we should roll back commit b00ac0936 to zookeeper-3.4.6 T I have
> to revoke my LGTM ;-(
One option is to have Zookeeper 3.4.6 as a service, but relax Hadoop dependency
back to the default. Typically old clients work pretty well with new Zookeeper
servers.
What do y'all think?
Thanks,
Roman.
Re: Failed building hadoop-deb
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi,
There may be issues with netty and zookeeper within hadoop depending on each other.
https://issues.apache.org/jira/browse/HADOOP-12928 <https://issues.apache.org/jira/browse/HADOOP-12928>
https://issues.apache.org/jira/browse/HADOOP-13413 <https://issues.apache.org/jira/browse/HADOOP-13413>
We have a dependency on curator as well
https://issues.apache.org/jira/browse/HADOOP-14187 <https://issues.apache.org/jira/browse/HADOOP-14187> ( only in hadoop 3 alpha)
I didn't see all the mess when I initialy reviewed b00ac0936
I think we should roll back commit b00ac0936 to zookeeper-3.4.6 T I have to revoke my LGTM ;-(
Olaf
> Am 25.04.2017 um 17:44 schrieb Andrew Musselman <an...@gmail.com>:
>
> I'm trying to confirm revving Mahout to 0.13.0 works in master for the
> 1.2.1 release. Should I file a bug ticket?
>
> On Tue, Apr 25, 2017 at 2:17 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>
>> i am a little bit nervous patching these enforcements. who validates this
>> is actually working for all eventualities?
>>
>> olaf
>>
>>> Am 25.04.2017 um 11:11 schrieb Arnaud Launay <as...@launay.org>:
>>>
>>> Le Tue, Apr 25, 2017 at 07:49:05AM +0200, Olaf Flebbe a écrit:
>>>> Dependency convergence error for org.apache.zookeeper:zookeeper:3.4.10
>> paths to dependency are:
>>>> +-org.apache.hadoop:hadoop-yarn-server-tests:2.7.3
>>>> +-org.apache.hadoop:hadoop-common:2.7.3
>>>> +-org.apache.hadoop:hadoop-auth:2.7.3
>>>> +-org.apache.zookeeper:zookeeper:3.4.10
>>>>
>>>> Seems like hadoop checks and enforces zookeeper version.
>>>
>>> I believe this is the patch you're looking for:
>>>
>>>
>> https://github.com/alaunay/bigtop/commit/c14d72ebb5f789150e7c0d7310db0f0d552d0d52
>>>
>>>
>>> Arnaud.
>>
Re: Failed building hadoop-deb
Posted by Andrew Musselman <an...@gmail.com>.
I'm trying to confirm revving Mahout to 0.13.0 works in master for the
1.2.1 release. Should I file a bug ticket?
On Tue, Apr 25, 2017 at 2:17 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> i am a little bit nervous patching these enforcements. who validates this
> is actually working for all eventualities?
>
> olaf
>
> > Am 25.04.2017 um 11:11 schrieb Arnaud Launay <as...@launay.org>:
> >
> > Le Tue, Apr 25, 2017 at 07:49:05AM +0200, Olaf Flebbe a écrit:
> >> Dependency convergence error for org.apache.zookeeper:zookeeper:3.4.10
> paths to dependency are:
> >> +-org.apache.hadoop:hadoop-yarn-server-tests:2.7.3
> >> +-org.apache.hadoop:hadoop-common:2.7.3
> >> +-org.apache.hadoop:hadoop-auth:2.7.3
> >> +-org.apache.zookeeper:zookeeper:3.4.10
> >>
> >> Seems like hadoop checks and enforces zookeeper version.
> >
> > I believe this is the patch you're looking for:
> >
> >
> https://github.com/alaunay/bigtop/commit/c14d72ebb5f789150e7c0d7310db0f0d552d0d52
> >
> >
> > Arnaud.
>
Re: Failed building hadoop-deb
Posted by Olaf Flebbe <of...@oflebbe.de>.
i am a little bit nervous patching these enforcements. who validates this is actually working for all eventualities?
olaf
> Am 25.04.2017 um 11:11 schrieb Arnaud Launay <as...@launay.org>:
>
> Le Tue, Apr 25, 2017 at 07:49:05AM +0200, Olaf Flebbe a écrit:
>> Dependency convergence error for org.apache.zookeeper:zookeeper:3.4.10 paths to dependency are:
>> +-org.apache.hadoop:hadoop-yarn-server-tests:2.7.3
>> +-org.apache.hadoop:hadoop-common:2.7.3
>> +-org.apache.hadoop:hadoop-auth:2.7.3
>> +-org.apache.zookeeper:zookeeper:3.4.10
>>
>> Seems like hadoop checks and enforces zookeeper version.
>
> I believe this is the patch you're looking for:
>
> https://github.com/alaunay/bigtop/commit/c14d72ebb5f789150e7c0d7310db0f0d552d0d52
>
>
> Arnaud.
Re: Failed building hadoop-deb
Posted by Arnaud Launay <as...@launay.org>.
Le Tue, Apr 25, 2017 at 07:49:05AM +0200, Olaf Flebbe a écrit:
> Dependency convergence error for org.apache.zookeeper:zookeeper:3.4.10 paths to dependency are:
> +-org.apache.hadoop:hadoop-yarn-server-tests:2.7.3
> +-org.apache.hadoop:hadoop-common:2.7.3
> +-org.apache.hadoop:hadoop-auth:2.7.3
> +-org.apache.zookeeper:zookeeper:3.4.10
>
> Seems like hadoop checks and enforces zookeeper version.
I believe this is the patch you're looking for:
https://github.com/alaunay/bigtop/commit/c14d72ebb5f789150e7c0d7310db0f0d552d0d52
Arnaud.
Re: Failed building hadoop-deb
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Mon, Apr 24, 2017 at 10:49 PM, Olaf Flebbe <of...@oflebbe.de> wrote:
> Hi,
>
> When nobody fixes our CI these kind of errors got unnoticed.
Agree. This is pretty embarrassing :-(
I was planning to spend some time on 1.2.1 and do some CI improvements
as part of that -- but it looks like it only will be possible in the
2nd half of May :-(
> I changed the CI job from
> ./gradlew allclean $CONFIGURE_NEXUS $COMPONENTS-pkg ; ./gradlew repo'
> to
> ./gradlew allclean $CONFIGURE_NEXUS $COMPONENTS-pkg && ./gradlew repo'
>
> This will destroy the all-blue CI.
>
>
> Back to the error:
>
>
> https://ci.bigtop.apache.org/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=ubuntu-16.04/46/console
>
> The error can clearly be seen in the logfile
>
> Dependency convergence error for org.apache.zookeeper:zookeeper:3.4.10 paths
> to dependency are:
> +-org.apache.hadoop:hadoop-yarn-server-tests:2.7.3
> +-org.apache.hadoop:hadoop-common:2.7.3
> +-org.apache.hadoop:hadoop-auth:2.7.3
> +-org.apache.zookeeper:zookeeper:3.4.10
>
>
> Seems like hadoop checks and enforces zookeeper version.
Great catch!
Thanks,
Roman.
Re: Failed building hadoop-deb
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi,
When nobody fixes our CI these kind of errors got unnoticed.
I changed the CI job from
./gradlew allclean $CONFIGURE_NEXUS $COMPONENTS-pkg ; ./gradlew repo'
to
./gradlew allclean $CONFIGURE_NEXUS $COMPONENTS-pkg && ./gradlew repo'
This will destroy the all-blue CI.
Back to the error:
https://ci.bigtop.apache.org/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=ubuntu-16.04/46/console <https://ci.bigtop.apache.org/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=ubuntu-16.04/46/console>
The error can clearly be seen in the logfile
Dependency convergence error for org.apache.zookeeper:zookeeper:3.4.10 paths to dependency are:
+-org.apache.hadoop:hadoop-yarn-server-tests:2.7.3
+-org.apache.hadoop:hadoop-common:2.7.3
+-org.apache.hadoop:hadoop-auth:2.7.3
+-org.apache.zookeeper:zookeeper:3.4.10
Seems like hadoop checks and enforces zookeeper version.
Olaf
> Am 25.04.2017 um 04:01 schrieb Andrew Musselman <an...@gmail.com>:
>
> Getting the same error using the ubuntu-16.04 image; is this a known issue
> or is there something to look for?
>
> On Mon, Apr 24, 2017 at 6:46 PM, Andrew Musselman <
> andrew.musselman@gmail.com> wrote:
>
>> From master on Ubuntu 16.04; is there an issue using the ubuntu-14.04
>> docker image?
>>
>> $ sudo docker run -u jenkins --rm --workdir /ws -v `pwd`:/ws
>> bigtop/slaves:trunk-ubuntu-14.04 bash -l -c './gradlew hadoop-deb
>> bigtop-utils-deb zookeeper-deb'
>>
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................. SUCCESS [
>> 20.159 s]
>> [INFO] Apache Hadoop Build Tools .......................... SUCCESS [
>> 2.572 s]
>> [INFO] Apache Hadoop Project POM .......................... SUCCESS [
>> 2.685 s]
>> [INFO] Apache Hadoop Annotations .......................... SUCCESS [
>> 3.150 s]
>> [INFO] Apache Hadoop Assemblies ........................... SUCCESS [
>> 0.077 s]
>> [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [
>> 1.082 s]
>> [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [
>> 4.444 s]
>> [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [
>> 44.601 s]
>> [INFO] Apache Hadoop Auth ................................. SUCCESS [
>> 19.104 s]
>> [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [
>> 2.294 s]
>> [INFO] Apache Hadoop Common ............................... SUCCESS
>> [01:16 min]
>> [INFO] Apache Hadoop NFS .................................. SUCCESS [
>> 2.472 s]
>> [INFO] Apache Hadoop KMS .................................. SUCCESS [
>> 12.127 s]
>> [INFO] Apache Hadoop Common Project ....................... SUCCESS [
>> 0.028 s]
>> [INFO] Apache Hadoop HDFS ................................. SUCCESS
>> [01:09 min]
>> [INFO] Apache Hadoop HttpFS ............................... SUCCESS [
>> 16.642 s]
>> [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [
>> 7.189 s]
>> [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [
>> 2.218 s]
>> [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [
>> 0.020 s]
>> [INFO] hadoop-yarn ........................................ SUCCESS [
>> 0.017 s]
>> [INFO] hadoop-yarn-api .................................... SUCCESS [
>> 15.536 s]
>> [INFO] hadoop-yarn-common ................................. SUCCESS [
>> 26.133 s]
>> [INFO] hadoop-yarn-server ................................. SUCCESS [
>> 0.018 s]
>> [INFO] hadoop-yarn-server-common .......................... SUCCESS [
>> 4.366 s]
>> [INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [
>> 7.952 s]
>> [INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [
>> 1.665 s]
>> [INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [
>> 2.927 s]
>> [INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [
>> 8.022 s]
>> [INFO] hadoop-yarn-server-tests ........................... FAILURE [
>> 4.129 s]
>> [INFO] hadoop-yarn-client ................................. SKIPPED
>> [INFO] hadoop-yarn-server-sharedcachemanager .............. SKIPPED
>> [INFO] hadoop-yarn-applications ........................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
>> [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
>> [INFO] hadoop-yarn-site ................................... SKIPPED
>> [INFO] hadoop-yarn-registry ............................... SKIPPED
>> [INFO] hadoop-yarn-project ................................ SKIPPED
>> [INFO] hadoop-mapreduce-client ............................ SKIPPED
>> [INFO] hadoop-mapreduce-client-core ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-common ..................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
>> [INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
>> [INFO] hadoop-mapreduce ................................... SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................. SKIPPED
>> [INFO] Apache Hadoop Rumen ................................ SKIPPED
>> [INFO] Apache Hadoop Gridmix .............................. SKIPPED
>> [INFO] Apache Hadoop Data Join ............................ SKIPPED
>> [INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
>> [INFO] Apache Hadoop Extras ............................... SKIPPED
>> [INFO] Apache Hadoop Pipes ................................ SKIPPED
>> [INFO] Apache Hadoop OpenStack support .................... SKIPPED
>> [INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
>> [INFO] Apache Hadoop Azure support ........................ SKIPPED
>> [INFO] Apache Hadoop Client ............................... SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
>> [INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
>> [INFO] Apache Hadoop Tools Dist ........................... SKIPPED
>> [INFO] Apache Hadoop Tools ................................ SKIPPED
>> [INFO] Apache Hadoop Distribution ......................... SKIPPED
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] BUILD FAILURE
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Total time: 06:04 min
>> [INFO] Finished at: 2017-04-25T01:41:27+00:00
>> [INFO] Final Memory: 214M/3445M
>> [INFO] ------------------------------------------------------------
>> ------------
>> [ERROR] Failed to execute goal org.apache.maven.plugins:
>> maven-enforcer-plugin:1.3.1:enforce (depcheck) on project
>> hadoop-yarn-server-tests: Some Enforcer rules have failed. Look above for
>> specific messages explaining why the rule failed. -> [Help 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>> -e switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/
>> MojoExecutionException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :hadoop-yarn-server-tests
>> make[1]: *** [override_dh_auto_build] Error 1
>> make[1]: Leaving directory `/ws/output/hadoop/hadoop-2.7.3'
>> make: *** [build] Error 2
>> dpkg-buildpackage: error: debian/rules build gave error exit status 2
>> debuild: fatal error at line 1364:
>> dpkg-buildpackage -rfakeroot -D -us -uc -b
>> failed
>> :hadoop-deb FAILED
>>
>> FAILURE: Build failed with an exception.
>>
>> * Where:
>> Script '/ws/packages.gradle' line: 341
>>
>> * What went wrong:
>> Execution failed for task ':hadoop-deb'.
>>> Process 'command 'debuild'' finished with non-zero exit value 29
>>
>> * Try:
>> Run with --stacktrace option to get the stack trace. Run with --info or
>> --debug option to get more log output.
>>
>> BUILD FAILED
>>
>> Total time: 6 mins 44.819 secs
>>
>>
Re: Failed building hadoop-deb
Posted by Andrew Musselman <an...@gmail.com>.
Getting the same error using the ubuntu-16.04 image; is this a known issue
or is there something to look for?
On Mon, Apr 24, 2017 at 6:46 PM, Andrew Musselman <
andrew.musselman@gmail.com> wrote:
> From master on Ubuntu 16.04; is there an issue using the ubuntu-14.04
> docker image?
>
> $ sudo docker run -u jenkins --rm --workdir /ws -v `pwd`:/ws
> bigtop/slaves:trunk-ubuntu-14.04 bash -l -c './gradlew hadoop-deb
> bigtop-utils-deb zookeeper-deb'
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................. SUCCESS [
> 20.159 s]
> [INFO] Apache Hadoop Build Tools .......................... SUCCESS [
> 2.572 s]
> [INFO] Apache Hadoop Project POM .......................... SUCCESS [
> 2.685 s]
> [INFO] Apache Hadoop Annotations .......................... SUCCESS [
> 3.150 s]
> [INFO] Apache Hadoop Assemblies ........................... SUCCESS [
> 0.077 s]
> [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [
> 1.082 s]
> [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [
> 4.444 s]
> [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [
> 44.601 s]
> [INFO] Apache Hadoop Auth ................................. SUCCESS [
> 19.104 s]
> [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [
> 2.294 s]
> [INFO] Apache Hadoop Common ............................... SUCCESS
> [01:16 min]
> [INFO] Apache Hadoop NFS .................................. SUCCESS [
> 2.472 s]
> [INFO] Apache Hadoop KMS .................................. SUCCESS [
> 12.127 s]
> [INFO] Apache Hadoop Common Project ....................... SUCCESS [
> 0.028 s]
> [INFO] Apache Hadoop HDFS ................................. SUCCESS
> [01:09 min]
> [INFO] Apache Hadoop HttpFS ............................... SUCCESS [
> 16.642 s]
> [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [
> 7.189 s]
> [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [
> 2.218 s]
> [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [
> 0.020 s]
> [INFO] hadoop-yarn ........................................ SUCCESS [
> 0.017 s]
> [INFO] hadoop-yarn-api .................................... SUCCESS [
> 15.536 s]
> [INFO] hadoop-yarn-common ................................. SUCCESS [
> 26.133 s]
> [INFO] hadoop-yarn-server ................................. SUCCESS [
> 0.018 s]
> [INFO] hadoop-yarn-server-common .......................... SUCCESS [
> 4.366 s]
> [INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [
> 7.952 s]
> [INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [
> 1.665 s]
> [INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [
> 2.927 s]
> [INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [
> 8.022 s]
> [INFO] hadoop-yarn-server-tests ........................... FAILURE [
> 4.129 s]
> [INFO] hadoop-yarn-client ................................. SKIPPED
> [INFO] hadoop-yarn-server-sharedcachemanager .............. SKIPPED
> [INFO] hadoop-yarn-applications ........................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
> [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
> [INFO] hadoop-yarn-site ................................... SKIPPED
> [INFO] hadoop-yarn-registry ............................... SKIPPED
> [INFO] hadoop-yarn-project ................................ SKIPPED
> [INFO] hadoop-mapreduce-client ............................ SKIPPED
> [INFO] hadoop-mapreduce-client-core ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-common ..................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
> [INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
> [INFO] hadoop-mapreduce ................................... SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
> [INFO] Apache Hadoop Archives ............................. SKIPPED
> [INFO] Apache Hadoop Rumen ................................ SKIPPED
> [INFO] Apache Hadoop Gridmix .............................. SKIPPED
> [INFO] Apache Hadoop Data Join ............................ SKIPPED
> [INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
> [INFO] Apache Hadoop Extras ............................... SKIPPED
> [INFO] Apache Hadoop Pipes ................................ SKIPPED
> [INFO] Apache Hadoop OpenStack support .................... SKIPPED
> [INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
> [INFO] Apache Hadoop Azure support ........................ SKIPPED
> [INFO] Apache Hadoop Client ............................... SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
> [INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
> [INFO] Apache Hadoop Tools Dist ........................... SKIPPED
> [INFO] Apache Hadoop Tools ................................ SKIPPED
> [INFO] Apache Hadoop Distribution ......................... SKIPPED
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Total time: 06:04 min
> [INFO] Finished at: 2017-04-25T01:41:27+00:00
> [INFO] Final Memory: 214M/3445M
> [INFO] ------------------------------------------------------------
> ------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:
> maven-enforcer-plugin:1.3.1:enforce (depcheck) on project
> hadoop-yarn-server-tests: Some Enforcer rules have failed. Look above for
> specific messages explaining why the rule failed. -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/
> MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :hadoop-yarn-server-tests
> make[1]: *** [override_dh_auto_build] Error 1
> make[1]: Leaving directory `/ws/output/hadoop/hadoop-2.7.3'
> make: *** [build] Error 2
> dpkg-buildpackage: error: debian/rules build gave error exit status 2
> debuild: fatal error at line 1364:
> dpkg-buildpackage -rfakeroot -D -us -uc -b
> failed
> :hadoop-deb FAILED
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Script '/ws/packages.gradle' line: 341
>
> * What went wrong:
> Execution failed for task ':hadoop-deb'.
> > Process 'command 'debuild'' finished with non-zero exit value 29
>
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output.
>
> BUILD FAILED
>
> Total time: 6 mins 44.819 secs
>
>