You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Alexander Frolushkin <ia...@gmail.com> on 2020/01/22 04:59:57 UTC
Bigtop error on Centos 7
Hello everyone.
Looks like Bigtop mailing lists is not very active, but I still hope
someone could help me.
We are trying to employ BigTop to build own hadoop repository (./gradlew
hadoop-rpm), but after all efforts still getting error on zookeeper version
dependency:
-org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
it wants -org.apache.zookeeper:zookeeper:3.4.6
while all other conponents depends on -org.apache.zookeeper:zookeeper:3.4.13
so we got:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (depcheck) on
project hadoop-yarn-server-tests
Please advise me some solution for this show stopper, if possible. Thanks!
Re: Bigtop error on Centos 7
Posted by Konstantin Boudnik <co...@apache.org>.
Thanks, Alexander, for bringing this up.
I don't see anything wrong with the way you're starting the build,
except that official CI is now using "hadoop-pkg-ind" target and build
then pulls in a correct container automatically. But I see you already
tried that one.
As a side note, I am looking at [1] and see the following error (which
seems to be there for some time):
Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not found in
any of the following sources:
- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Plugin Repositories (could not resolve plugin artifact
'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
Searched in the following repositories:
Gradle Central Plugin Repository
Unfortunately, it is hard to tell what kind of error you're getting in
your build. Any chance you can publish a snippet of the actual logs,
rather than a screenshot?
Thanks,Cos
[1]
https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> Hello everyone!
>
> Sorry for repeating this in dev list, but I really need help in this
> situation.
>
> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
>
> Failed to execute goal
> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> (default) on project hadoop-hdfs-httpfs: An error has occurred in
> Dependencies report generation.: zip file is empty -> [Help 1]
>
> Launched in clean bigtop-1.4.0 dir, with command docker run -v `pwd`:/ws
> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
>
> I made a --scan for this attempt - https://gradle.com/s/7iu4z3fsbrh62
>
> Maybe I'm using not a valid way to get it perform correctly?
>
> Or this can be bug and I need to fill a jira ticket for this?
>
> Also I tryed to use direct building by ./gradlew hadoop-rpm and via docker
> ./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk, this
> stage (hadoop-hdfs-httpfs) passed without error, but as I've been told,
> currently hadoop is unbuildable in master branch.
>
> Plus, the same error I get on bigtop-1.3.0.
>
> Cleaning the local maven repository does not fix it.
>
> So, likely, the problem is in building environment. How I can configure it
> correctly for successful hadoop building in bigtop?
>
> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
>
>> Currently building Hadoop across all platform on master branch is failing:
>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>
>> I'm not sure whether the master code can be built successfully since we're
>> in transition of upgrading zookeeper.
>>
>> If you just want to use bigtop, please get our latest release 1.4.0.
>>
>> If you are willing to contribute, you can report a bug on our jira first
>> and we ca work on a fix together.
>>
>> Let us know if any question.
>>
>> Evan's
>>
>>
>>
>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
>>
>>> Hi Alexander,
>>>
>>> It seems bigtop_toolchain didn't deploy correctly.
>>> Please try to build bigtop components in docker with bigtop/slaves docker
>>> images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos>.
>>>
>>> Run:
>>> cd BIGTOP_SRC_DIR
>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws ;
>>> gradle hadoop-pkg'
>>>
>>> BR,
>>> Yuqi
>>>
>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <ia...@gmail.com>
>>> wrote:
>>>
>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
>>>> error.
>>>>
>>>> Also, in webtop-trunk this particular error appeard while building
>> hadoop
>>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
>>> without
>>>> it (by ./gradlew hadoop-rpm).
>>>>
>>>> Performed steps:
>>>>
>>>> I was trying to follow the available guides, do a clean centos 7
>> install,
>>>> upgrade to latest official rpms, install openjdk 7 and unzip from repo.
>>>>
>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
>>>> bigtop_toolchain::installer".
>>>>
>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and got
>>>> this error.
>>>>
>>>>
>>>>
>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
>> wrote:
>>>>> Hi Alexander,
>>>>>
>>>>> Which version or branch of Bigtop do you use? and also can you
>> provide
>>> us
>>>>> the steps you did?
>>>>>
>>>>> Thanks,
>>>>> Youngwoo
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
>>> iamhokum@gmail.com
>>>>> wrote:
>>>>>
>>>>>> Hello everyone.
>>>>>>
>>>>>> Looks like Bigtop mailing lists is not very active, but I still
>> hope
>>>>>> someone could help me.
>>>>>>
>>>>>> We are trying to employ BigTop to build own hadoop repository
>>>> (./gradlew
>>>>>> hadoop-rpm), but after all efforts still getting error on zookeeper
>>>>> version
>>>>>> dependency:
>>>>>>
>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
>>>>>>
>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
>>>>>>
>>>>>> while all other conponents depends on
>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
>>>>>>
>>>>>> so we got:
>>>>>>
>>>>>> [ERROR] Failed to execute goal
>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
>>> (depcheck)
>>>>> on
>>>>>> project hadoop-yarn-server-tests
>>>>>>
>>>>>>
>>>>>>
>>>>>> Please advise me some solution for this show stopper, if possible.
>>>>> Thanks!
Re: Bigtop error on Centos 7
Posted by Evans Ye <ev...@apache.org>.
Oh, that's great.
Would you like to report a bug to Bigtop JIRA and attach a patch to fix it?
A patch can be applied when building. Just need to placing a .diff patch
file under bigtop-packages/src/common/hadoop. You can find two examples
there:
* patch0-HADOOP-12366.-expose-calculated-paths.diff
* patch2-HADOOP-14597.diff
Alexander Frolushkin <ia...@gmail.com> 於 2020年1月27日 週一 22:54 寫道：
> Sorry, if my text somehow made you worry. Of cause this is OSS and I just
> want to clearly understand current situation with it.
> I already manually fixed hadoop-rpm building task via pom.xml url change to
> https. Of cause it is a local workaround.
> Anyway big thanks for your help and explanation of some details of how
> Bigtop works.
>
>
> On Mon, Jan 27, 2020, 21:46 Evans Ye <ev...@apache.org> wrote:
>
> > Alex,
> >
> > Thanks for reporting the issue, but let's not drive the discussion in
> this
> > way. This is an open source project and we're just trying to make
> software
> > for public good. We don't have the resource to compete with commercial
> > distribution vendor(s). If you have an idea to improve the build quality.
> > We're happy to learn from you :)
> >
> > IMHO either way our latest release can't build. Let's figure out how to
> fix
> > it first. Then we can come up with a quick bugfix release with just the
> > apache tar.gz release. No need to publish the artifacts/repos.
> >
> > I was trying to fix the problem but I have limited time so it can be
> slow.
> > Alex if you have some finding please share to us. Technical discussions
> are
> > always welcome.
> >
> > Evans
> >
> > Konstantin Boudnik <co...@apache.org> 於 2020年1月27日 週一 上午5:55寫道：
> >
> > > Actually, it isn't about Bigtop build. Bigtop is using whatever
> > > source/build
> > > materials are particular component is supplying as a part of their
> > > release' source
> > > tree. Yes, we do an alternation or two when the "official" stuff is
> > totally
> > > broken but it is rather an exception.
> > >
> > > In this particular case, Hadoop's builds have fixed on the fly by our
> own
> > > 'do-component-build' script or to be fixed either in the upstream
> > project.
> > >
> > > --
> > > Cos
> > >
> > > On Sun, Jan 26, 2020 at 12:47PM, Alexander Frolushkin wrote:
> > > > So, someone need to remade src.rpm for Hadoop, replacing url in .pom
> > for
> > > > this resource to become available? And now Hadoop cannot be build by
> > any
> > > > version of Bigtop?
> > > >
> > > > On Fri, Jan 24, 2020, 12:04 Olaf Flebbe <of...@oflebbe.de> wrote:
> > > >
> > > > > hi
> > > > >
> > > > > the failed downloads are http:// downloads
> > > > >
> > > > > http download from nexus are now turned off for security reasons.
> > > > >
> > > > > there was another report for oozie with the same problem on this
> list
> > > iirc.
> > > > >
> > > > > imo this can only be solved by using additional tooling, smthg
> like a
> > > > > proxy changing http to https . nexus might have a property for
> that.
> > > or new
> > > > > maven might have smthg in place for working around broken
> > > dependencies, but
> > > > > i am speculating
> > > > >
> > > > > best
> > > > > olaf
> > > > >
> > > > >
> > > > >
> > > > > > Am 24.01.2020 um 05:47 schrieb Konstantin Boudnik <
> cos@apache.org
> > >:
> > > > > >
> > > > > > Looks like some sort of trouble with one of the dependencies of
> > > httpfs:
> > > > > >
> > > > > >
> > > > > > Downloading from apache.snapshots.https:
> > > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > > Downloading from repository.jboss.org:
> > > > >
> > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > > [WARNING] Checksum validation failed, expected but is
> > > > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.
> > > > > > org for
> > > > >
> > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > > [WARNING] Could not validate integrity of download from
> > > > >
> > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > :
> > > > > Checksum validation failed, expected but is
> > > > > da39a3ee5e6b4b0d3255bfef95601890afd80709
> > > > > > [WARNING] Checksum validation failed, expected but is
> > > > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org
> > for
> > > > >
> > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > > Downloaded from repository.jboss.org:
> > > > >
> > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > (0 B at 0 B/s)
> > > > > >
> > > > > > And then build fails right at that module. Doesn't look like a
> > Bigtop
> > > > > issue, methink. I am on the road right now with somewhat limited
> > > internet,
> > > > > will be back over the weekend and try to look more. Perhaps in the
> > > > > meanwhile someone on the list would have any other ideas.
> > > > > >
> > > > > > --
> > > > > > Cos
> > > > > >
> > > > > >> On 1/23/20 11:40 PM, Konstantin Boudnik wrote:
> > > > > >> Well, I am not familiar with Gradle enterprise - looks like it
> has
> > > > > everything indeed, thanks! Let me dig and see what I can figure
> out.
> > > > > >>
> > > > > >> --
> > > > > >> Cos
> > > > > >>
> > > > > >>> On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
> > > > > >>> Thank you for responding, Konstantin.
> > > > > >>>
> > > > > >>> Honestly, I was thinking --scan gradle will publish all
> necessary
> > > > > >>> information about its execution, which may be required to
> > > inverstigate
> > > > > the
> > > > > >>> problem.
> > > > > >>>
> > > > > >>> Anyway, I'm ready to provide maximum data for this dicsussion.
> > > Firstly,
> > > > > >>> here is a complete console output of entire hadoop-pkg-ind task
> > > till
> > > > > the
> > > > > >>> error -
> > > > > >>>
> > > > >
> > >
> >
> https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
> > > > > >>>
> > > > > >>> If I need to collect additional data, please direct me, what
> > > exactly I
> > > > > need
> > > > > >>> to do to get it for processing here.
> > > > > >>>
> > > > > >>> I really want to build hadoop repo by bigtop-1.4.0.
> > > > > >>>
> > > > > >>>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <
> cos@apache.org>
> > > > > wrote:
> > > > > >>>
> > > > > >>>> Thanks, Alexander, for bringing this up.
> > > > > >>>>
> > > > > >>>> I don't see anything wrong with the way you're starting the
> > build,
> > > > > >>>> except that official CI is now using "hadoop-pkg-ind" target
> and
> > > build
> > > > > >>>> then pulls in a correct container automatically. But I see you
> > > already
> > > > > >>>> tried that one.
> > > > > >>>>
> > > > > >>>> As a side note, I am looking at [1] and see the following
> error
> > > (which
> > > > > >>>> seems to be there for some time):
> > > > > >>>>
> > > > > >>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was
> not
> > > found
> > > > > in
> > > > > >>>> any of the following sources:
> > > > > >>>>
> > > > > >>>> - Gradle Core Plugins (plugin is not in 'org.gradle'
> namespace)
> > > > > >>>> - Plugin Repositories (could not resolve plugin artifact
> > > > > >>>>
> > > 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
> > > > > >>>> Searched in the following repositories:
> > > > > >>>> Gradle Central Plugin Repository
> > > > > >>>>
> > > > > >>>> Unfortunately, it is hard to tell what kind of error you're
> > > getting in
> > > > > >>>> your build. Any chance you can publish a snippet of the actual
> > > logs,
> > > > > >>>> rather than a screenshot?
> > > > > >>>>
> > > > > >>>> --
> > > > > >>>> Thanks,
> > > > > >>>> Cos
> > > > > >>>>
> > > > > >>>> [1]
> > > > > >>>>
> > > > > >>>>
> > > > >
> > >
> >
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
> > > > > >>>>
> > > > > >>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> > > > > >>>>> Hello everyone!
> > > > > >>>>>
> > > > > >>>>> Sorry for repeating this in dev list, but I really need help
> in
> > > this
> > > > > >>>>> situation.
> > > > > >>>>>
> > > > > >>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
> > > > > >>>>>
> > > > > >>>>> Failed to execute goal
> > > > > >>>>>
> > > > > >>>>
> > > > >
> > >
> >
> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> > > > > >>>>> (default) on project hadoop-hdfs-httpfs: An error has
> occurred
> > in
> > > > > >>>>> Dependencies report generation.: zip file is empty -> [Help
> 1]
> > > > > >>>>>
> > > > > >>>>> Launched in clean bigtop-1.4.0 dir, with command docker run
> -v
> > > > > `pwd`:/ws
> > > > > >>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle
> > hadoop-pkg
> > > > > >>>>>
> > > > > >>>>> I made a --scan for this attempt -
> > > > > https://gradle.com/s/7iu4z3fsbrh62
> > > > > >>>>>
> > > > > >>>>> Maybe I'm using not a valid way to get it perform correctly?
> > > > > >>>>>
> > > > > >>>>> Or this can be bug and I need to fill a jira ticket for this?
> > > > > >>>>>
> > > > > >>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm
> and
> > > via
> > > > > >>>> docker
> > > > > >>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use
> > > bigtop-trunk,
> > > > > this
> > > > > >>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've
> > been
> > > > > told,
> > > > > >>>>> currently hadoop is unbuildable in master branch.
> > > > > >>>>>
> > > > > >>>>> Plus, the same error I get on bigtop-1.3.0.
> > > > > >>>>>
> > > > > >>>>> Cleaning the local maven repository does not fix it.
> > > > > >>>>>
> > > > > >>>>> So, likely, the problem is in building environment. How I can
> > > > > configure
> > > > > >>>> it
> > > > > >>>>> correctly for successful hadoop building in bigtop?
> > > > > >>>>>
> > > > > >>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org>
> > wrote:
> > > > > >>>>>
> > > > > >>>>>> Currently building Hadoop across all platform on master
> branch
> > > is
> > > > > >>>> failing:
> > > > > >>>>>>
> > > > >
> > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > > > > >>>>>>
> > > > > >>>>>> I'm not sure whether the master code can be built
> successfully
> > > since
> > > > > >>>> we're
> > > > > >>>>>> in transition of upgrading zookeeper.
> > > > > >>>>>>
> > > > > >>>>>> If you just want to use bigtop, please get our latest
> release
> > > 1.4.0.
> > > > > >>>>>>
> > > > > >>>>>> If you are willing to contribute, you can report a bug on
> our
> > > jira
> > > > > first
> > > > > >>>>>> and we ca work on a fix together.
> > > > > >>>>>>
> > > > > >>>>>> Let us know if any question.
> > > > > >>>>>>
> > > > > >>>>>> Evan's
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
> > > > > >>>>>>
> > > > > >>>>>>> Hi Alexander,
> > > > > >>>>>>>
> > > > > >>>>>>> It seems bigtop_toolchain didn't deploy correctly.
> > > > > >>>>>>> Please try to build bigtop components in docker with
> > > bigtop/slaves
> > > > > >>>> docker
> > > > > >>>>>>> images <
> > > > > https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
> > > > > >>>>> .
> > > > > >>>>>>> Run:
> > > > > >>>>>>> cd BIGTOP_SRC_DIR
> > > > > >>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash
> -l
> > > -c 'cd
> > > > > >>>> /ws ;
> > > > > >>>>>>> gradle hadoop-pkg'
> > > > > >>>>>>>
> > > > > >>>>>>> BR,
> > > > > >>>>>>> Yuqi
> > > > > >>>>>>>
> > > > > >>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <
> > > > > iamhokum@gmail.com
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it
> fails
> > > with
> > > > > other
> > > > > >>>>>>>> error.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Also, in webtop-trunk this particular error appeard while
> > > building
> > > > > >>>>>> hadoop
> > > > > >>>>>>>> in docker (trunk-centos-7) image (by ./gradlew
> > > hadoop-pkg-ind) and
> > > > > >>>>>>> without
> > > > > >>>>>>>> it (by ./gradlew hadoop-rpm).
> > > > > >>>>>>>>
> > > > > >>>>>>>> Performed steps:
> > > > > >>>>>>>>
> > > > > >>>>>>>> I was trying to follow the available guides, do a clean
> > > centos 7
> > > > > >>>>>> install,
> > > > > >>>>>>>> upgrade to latest official rpms, install openjdk 7 and
> unzip
> > > from
> > > > > >>>> repo.
> > > > > >>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet
> apply
> > > > > >>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
> > > > > >>>>>>>> bigtop_toolchain::installer".
> > > > > >>>>>>>>
> > > > > >>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew
> > > hadoop-rpm
> > > > > and
> > > > > >>>> got
> > > > > >>>>>>>> this error.
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <
> > > ywkim@apache.org>
> > > > > >>>>>> wrote:
> > > > > >>>>>>>>> Hi Alexander,
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Which version or branch of Bigtop do you use? and also
> can
> > > you
> > > > > >>>>>> provide
> > > > > >>>>>>> us
> > > > > >>>>>>>>> the steps you did?
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Thanks,
> > > > > >>>>>>>>> Youngwoo
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> > > > > >>>>>>> iamhokum@gmail.com
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>> Hello everyone.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Looks like Bigtop mailing lists is not very active, but
> I
> > > still
> > > > > >>>>>> hope
> > > > > >>>>>>>>>> someone could help me.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> We are trying to employ BigTop to build own hadoop
> > > repository
> > > > > >>>>>>>> (./gradlew
> > > > > >>>>>>>>>> hadoop-rpm), but after all efforts still getting error
> on
> > > > > zookeeper
> > > > > >>>>>>>>> version
> > > > > >>>>>>>>>> dependency:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> while all other conponents depends on
> > > > > >>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> so we got:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> [ERROR] Failed to execute goal
> > > > > >>>>>>>>>>
> > org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> > > > > >>>>>>> (depcheck)
> > > > > >>>>>>>>> on
> > > > > >>>>>>>>>> project hadoop-yarn-server-tests
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Please advise me some solution for this show stopper, if
> > > > > possible.
> > > > > >>>>>>>>> Thanks!
> > > > >
> > > > >
> > >
> >
>
Re: Bigtop error on Centos 7
Posted by Alexander Frolushkin <ia...@gmail.com>.
Sorry, if my text somehow made you worry. Of cause this is OSS and I just
want to clearly understand current situation with it.
I already manually fixed hadoop-rpm building task via pom.xml url change to
https. Of cause it is a local workaround.
Anyway big thanks for your help and explanation of some details of how
Bigtop works.
On Mon, Jan 27, 2020, 21:46 Evans Ye <ev...@apache.org> wrote:
> Alex,
>
> Thanks for reporting the issue, but let's not drive the discussion in this
> way. This is an open source project and we're just trying to make software
> for public good. We don't have the resource to compete with commercial
> distribution vendor(s). If you have an idea to improve the build quality.
> We're happy to learn from you :)
>
> IMHO either way our latest release can't build. Let's figure out how to fix
> it first. Then we can come up with a quick bugfix release with just the
> apache tar.gz release. No need to publish the artifacts/repos.
>
> I was trying to fix the problem but I have limited time so it can be slow.
> Alex if you have some finding please share to us. Technical discussions are
> always welcome.
>
> Evans
>
> Konstantin Boudnik <co...@apache.org> 於 2020年1月27日 週一 上午5:55寫道：
>
> > Actually, it isn't about Bigtop build. Bigtop is using whatever
> > source/build
> > materials are particular component is supplying as a part of their
> > release' source
> > tree. Yes, we do an alternation or two when the "official" stuff is
> totally
> > broken but it is rather an exception.
> >
> > In this particular case, Hadoop's builds have fixed on the fly by our own
> > 'do-component-build' script or to be fixed either in the upstream
> project.
> >
> > --
> > Cos
> >
> > On Sun, Jan 26, 2020 at 12:47PM, Alexander Frolushkin wrote:
> > > So, someone need to remade src.rpm for Hadoop, replacing url in .pom
> for
> > > this resource to become available? And now Hadoop cannot be build by
> any
> > > version of Bigtop?
> > >
> > > On Fri, Jan 24, 2020, 12:04 Olaf Flebbe <of...@oflebbe.de> wrote:
> > >
> > > > hi
> > > >
> > > > the failed downloads are http:// downloads
> > > >
> > > > http download from nexus are now turned off for security reasons.
> > > >
> > > > there was another report for oozie with the same problem on this list
> > iirc.
> > > >
> > > > imo this can only be solved by using additional tooling, smthg like a
> > > > proxy changing http to https . nexus might have a property for that.
> > or new
> > > > maven might have smthg in place for working around broken
> > dependencies, but
> > > > i am speculating
> > > >
> > > > best
> > > > olaf
> > > >
> > > >
> > > >
> > > > > Am 24.01.2020 um 05:47 schrieb Konstantin Boudnik <cos@apache.org
> >:
> > > > >
> > > > > Looks like some sort of trouble with one of the dependencies of
> > httpfs:
> > > > >
> > > > >
> > > > > Downloading from apache.snapshots.https:
> > > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > Downloading from repository.jboss.org:
> > > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > [WARNING] Checksum validation failed, expected but is
> > > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.
> > > > > org for
> > > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > [WARNING] Could not validate integrity of download from
> > > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > :
> > > > Checksum validation failed, expected but is
> > > > da39a3ee5e6b4b0d3255bfef95601890afd80709
> > > > > [WARNING] Checksum validation failed, expected but is
> > > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org
> for
> > > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > > Downloaded from repository.jboss.org:
> > > >
> >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > (0 B at 0 B/s)
> > > > >
> > > > > And then build fails right at that module. Doesn't look like a
> Bigtop
> > > > issue, methink. I am on the road right now with somewhat limited
> > internet,
> > > > will be back over the weekend and try to look more. Perhaps in the
> > > > meanwhile someone on the list would have any other ideas.
> > > > >
> > > > > --
> > > > > Cos
> > > > >
> > > > >> On 1/23/20 11:40 PM, Konstantin Boudnik wrote:
> > > > >> Well, I am not familiar with Gradle enterprise - looks like it has
> > > > everything indeed, thanks! Let me dig and see what I can figure out.
> > > > >>
> > > > >> --
> > > > >> Cos
> > > > >>
> > > > >>> On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
> > > > >>> Thank you for responding, Konstantin.
> > > > >>>
> > > > >>> Honestly, I was thinking --scan gradle will publish all necessary
> > > > >>> information about its execution, which may be required to
> > inverstigate
> > > > the
> > > > >>> problem.
> > > > >>>
> > > > >>> Anyway, I'm ready to provide maximum data for this dicsussion.
> > Firstly,
> > > > >>> here is a complete console output of entire hadoop-pkg-ind task
> > till
> > > > the
> > > > >>> error -
> > > > >>>
> > > >
> >
> https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
> > > > >>>
> > > > >>> If I need to collect additional data, please direct me, what
> > exactly I
> > > > need
> > > > >>> to do to get it for processing here.
> > > > >>>
> > > > >>> I really want to build hadoop repo by bigtop-1.4.0.
> > > > >>>
> > > > >>>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org>
> > > > wrote:
> > > > >>>
> > > > >>>> Thanks, Alexander, for bringing this up.
> > > > >>>>
> > > > >>>> I don't see anything wrong with the way you're starting the
> build,
> > > > >>>> except that official CI is now using "hadoop-pkg-ind" target and
> > build
> > > > >>>> then pulls in a correct container automatically. But I see you
> > already
> > > > >>>> tried that one.
> > > > >>>>
> > > > >>>> As a side note, I am looking at [1] and see the following error
> > (which
> > > > >>>> seems to be there for some time):
> > > > >>>>
> > > > >>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not
> > found
> > > > in
> > > > >>>> any of the following sources:
> > > > >>>>
> > > > >>>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
> > > > >>>> - Plugin Repositories (could not resolve plugin artifact
> > > > >>>>
> > 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
> > > > >>>> Searched in the following repositories:
> > > > >>>> Gradle Central Plugin Repository
> > > > >>>>
> > > > >>>> Unfortunately, it is hard to tell what kind of error you're
> > getting in
> > > > >>>> your build. Any chance you can publish a snippet of the actual
> > logs,
> > > > >>>> rather than a screenshot?
> > > > >>>>
> > > > >>>> --
> > > > >>>> Thanks,
> > > > >>>> Cos
> > > > >>>>
> > > > >>>> [1]
> > > > >>>>
> > > > >>>>
> > > >
> >
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
> > > > >>>>
> > > > >>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> > > > >>>>> Hello everyone!
> > > > >>>>>
> > > > >>>>> Sorry for repeating this in dev list, but I really need help in
> > this
> > > > >>>>> situation.
> > > > >>>>>
> > > > >>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
> > > > >>>>>
> > > > >>>>> Failed to execute goal
> > > > >>>>>
> > > > >>>>
> > > >
> >
> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> > > > >>>>> (default) on project hadoop-hdfs-httpfs: An error has occurred
> in
> > > > >>>>> Dependencies report generation.: zip file is empty -> [Help 1]
> > > > >>>>>
> > > > >>>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v
> > > > `pwd`:/ws
> > > > >>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle
> hadoop-pkg
> > > > >>>>>
> > > > >>>>> I made a --scan for this attempt -
> > > > https://gradle.com/s/7iu4z3fsbrh62
> > > > >>>>>
> > > > >>>>> Maybe I'm using not a valid way to get it perform correctly?
> > > > >>>>>
> > > > >>>>> Or this can be bug and I need to fill a jira ticket for this?
> > > > >>>>>
> > > > >>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and
> > via
> > > > >>>> docker
> > > > >>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use
> > bigtop-trunk,
> > > > this
> > > > >>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've
> been
> > > > told,
> > > > >>>>> currently hadoop is unbuildable in master branch.
> > > > >>>>>
> > > > >>>>> Plus, the same error I get on bigtop-1.3.0.
> > > > >>>>>
> > > > >>>>> Cleaning the local maven repository does not fix it.
> > > > >>>>>
> > > > >>>>> So, likely, the problem is in building environment. How I can
> > > > configure
> > > > >>>> it
> > > > >>>>> correctly for successful hadoop building in bigtop?
> > > > >>>>>
> > > > >>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org>
> wrote:
> > > > >>>>>
> > > > >>>>>> Currently building Hadoop across all platform on master branch
> > is
> > > > >>>> failing:
> > > > >>>>>>
> > > >
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > > > >>>>>>
> > > > >>>>>> I'm not sure whether the master code can be built successfully
> > since
> > > > >>>> we're
> > > > >>>>>> in transition of upgrading zookeeper.
> > > > >>>>>>
> > > > >>>>>> If you just want to use bigtop, please get our latest release
> > 1.4.0.
> > > > >>>>>>
> > > > >>>>>> If you are willing to contribute, you can report a bug on our
> > jira
> > > > first
> > > > >>>>>> and we ca work on a fix together.
> > > > >>>>>>
> > > > >>>>>> Let us know if any question.
> > > > >>>>>>
> > > > >>>>>> Evan's
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
> > > > >>>>>>
> > > > >>>>>>> Hi Alexander,
> > > > >>>>>>>
> > > > >>>>>>> It seems bigtop_toolchain didn't deploy correctly.
> > > > >>>>>>> Please try to build bigtop components in docker with
> > bigtop/slaves
> > > > >>>> docker
> > > > >>>>>>> images <
> > > > https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
> > > > >>>>> .
> > > > >>>>>>> Run:
> > > > >>>>>>> cd BIGTOP_SRC_DIR
> > > > >>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l
> > -c 'cd
> > > > >>>> /ws ;
> > > > >>>>>>> gradle hadoop-pkg'
> > > > >>>>>>>
> > > > >>>>>>> BR,
> > > > >>>>>>> Yuqi
> > > > >>>>>>>
> > > > >>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <
> > > > iamhokum@gmail.com
> > > > >>>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails
> > with
> > > > other
> > > > >>>>>>>> error.
> > > > >>>>>>>>
> > > > >>>>>>>> Also, in webtop-trunk this particular error appeard while
> > building
> > > > >>>>>> hadoop
> > > > >>>>>>>> in docker (trunk-centos-7) image (by ./gradlew
> > hadoop-pkg-ind) and
> > > > >>>>>>> without
> > > > >>>>>>>> it (by ./gradlew hadoop-rpm).
> > > > >>>>>>>>
> > > > >>>>>>>> Performed steps:
> > > > >>>>>>>>
> > > > >>>>>>>> I was trying to follow the available guides, do a clean
> > centos 7
> > > > >>>>>> install,
> > > > >>>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip
> > from
> > > > >>>> repo.
> > > > >>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> > > > >>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
> > > > >>>>>>>> bigtop_toolchain::installer".
> > > > >>>>>>>>
> > > > >>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew
> > hadoop-rpm
> > > > and
> > > > >>>> got
> > > > >>>>>>>> this error.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <
> > ywkim@apache.org>
> > > > >>>>>> wrote:
> > > > >>>>>>>>> Hi Alexander,
> > > > >>>>>>>>>
> > > > >>>>>>>>> Which version or branch of Bigtop do you use? and also can
> > you
> > > > >>>>>> provide
> > > > >>>>>>> us
> > > > >>>>>>>>> the steps you did?
> > > > >>>>>>>>>
> > > > >>>>>>>>> Thanks,
> > > > >>>>>>>>> Youngwoo
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> > > > >>>>>>> iamhokum@gmail.com
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>> Hello everyone.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Looks like Bigtop mailing lists is not very active, but I
> > still
> > > > >>>>>> hope
> > > > >>>>>>>>>> someone could help me.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> We are trying to employ BigTop to build own hadoop
> > repository
> > > > >>>>>>>> (./gradlew
> > > > >>>>>>>>>> hadoop-rpm), but after all efforts still getting error on
> > > > zookeeper
> > > > >>>>>>>>> version
> > > > >>>>>>>>>> dependency:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> while all other conponents depends on
> > > > >>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> so we got:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> [ERROR] Failed to execute goal
> > > > >>>>>>>>>>
> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> > > > >>>>>>> (depcheck)
> > > > >>>>>>>>> on
> > > > >>>>>>>>>> project hadoop-yarn-server-tests
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Please advise me some solution for this show stopper, if
> > > > possible.
> > > > >>>>>>>>> Thanks!
> > > >
> > > >
> >
>
Re: Bigtop error on Centos 7
Posted by Evans Ye <ev...@apache.org>.
Alex,
Thanks for reporting the issue, but let's not drive the discussion in this
way. This is an open source project and we're just trying to make software
for public good. We don't have the resource to compete with commercial
distribution vendor(s). If you have an idea to improve the build quality.
We're happy to learn from you :)
IMHO either way our latest release can't build. Let's figure out how to fix
it first. Then we can come up with a quick bugfix release with just the
apache tar.gz release. No need to publish the artifacts/repos.
I was trying to fix the problem but I have limited time so it can be slow.
Alex if you have some finding please share to us. Technical discussions are
always welcome.
Evans
Konstantin Boudnik <co...@apache.org> 於 2020年1月27日 週一 上午5:55寫道：
> Actually, it isn't about Bigtop build. Bigtop is using whatever
> source/build
> materials are particular component is supplying as a part of their
> release' source
> tree. Yes, we do an alternation or two when the "official" stuff is totally
> broken but it is rather an exception.
>
> In this particular case, Hadoop's builds have fixed on the fly by our own
> 'do-component-build' script or to be fixed either in the upstream project.
>
> --
> Cos
>
> On Sun, Jan 26, 2020 at 12:47PM, Alexander Frolushkin wrote:
> > So, someone need to remade src.rpm for Hadoop, replacing url in .pom for
> > this resource to become available? And now Hadoop cannot be build by any
> > version of Bigtop?
> >
> > On Fri, Jan 24, 2020, 12:04 Olaf Flebbe <of...@oflebbe.de> wrote:
> >
> > > hi
> > >
> > > the failed downloads are http:// downloads
> > >
> > > http download from nexus are now turned off for security reasons.
> > >
> > > there was another report for oozie with the same problem on this list
> iirc.
> > >
> > > imo this can only be solved by using additional tooling, smthg like a
> > > proxy changing http to https . nexus might have a property for that.
> or new
> > > maven might have smthg in place for working around broken
> dependencies, but
> > > i am speculating
> > >
> > > best
> > > olaf
> > >
> > >
> > >
> > > > Am 24.01.2020 um 05:47 schrieb Konstantin Boudnik <co...@apache.org>:
> > > >
> > > > Looks like some sort of trouble with one of the dependencies of
> httpfs:
> > > >
> > > >
> > > > Downloading from apache.snapshots.https:
> > >
> https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > Downloading from repository.jboss.org:
> > >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > [WARNING] Checksum validation failed, expected but is
> > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.
> > > > org for
> > >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > [WARNING] Could not validate integrity of download from
> > >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> :
> > > Checksum validation failed, expected but is
> > > da39a3ee5e6b4b0d3255bfef95601890afd80709
> > > > [WARNING] Checksum validation failed, expected but is
> > > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org for
> > >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > > Downloaded from repository.jboss.org:
> > >
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > (0 B at 0 B/s)
> > > >
> > > > And then build fails right at that module. Doesn't look like a Bigtop
> > > issue, methink. I am on the road right now with somewhat limited
> internet,
> > > will be back over the weekend and try to look more. Perhaps in the
> > > meanwhile someone on the list would have any other ideas.
> > > >
> > > > --
> > > > Cos
> > > >
> > > >> On 1/23/20 11:40 PM, Konstantin Boudnik wrote:
> > > >> Well, I am not familiar with Gradle enterprise - looks like it has
> > > everything indeed, thanks! Let me dig and see what I can figure out.
> > > >>
> > > >> --
> > > >> Cos
> > > >>
> > > >>> On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
> > > >>> Thank you for responding, Konstantin.
> > > >>>
> > > >>> Honestly, I was thinking --scan gradle will publish all necessary
> > > >>> information about its execution, which may be required to
> inverstigate
> > > the
> > > >>> problem.
> > > >>>
> > > >>> Anyway, I'm ready to provide maximum data for this dicsussion.
> Firstly,
> > > >>> here is a complete console output of entire hadoop-pkg-ind task
> till
> > > the
> > > >>> error -
> > > >>>
> > >
> https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
> > > >>>
> > > >>> If I need to collect additional data, please direct me, what
> exactly I
> > > need
> > > >>> to do to get it for processing here.
> > > >>>
> > > >>> I really want to build hadoop repo by bigtop-1.4.0.
> > > >>>
> > > >>>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org>
> > > wrote:
> > > >>>
> > > >>>> Thanks, Alexander, for bringing this up.
> > > >>>>
> > > >>>> I don't see anything wrong with the way you're starting the build,
> > > >>>> except that official CI is now using "hadoop-pkg-ind" target and
> build
> > > >>>> then pulls in a correct container automatically. But I see you
> already
> > > >>>> tried that one.
> > > >>>>
> > > >>>> As a side note, I am looking at [1] and see the following error
> (which
> > > >>>> seems to be there for some time):
> > > >>>>
> > > >>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not
> found
> > > in
> > > >>>> any of the following sources:
> > > >>>>
> > > >>>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
> > > >>>> - Plugin Repositories (could not resolve plugin artifact
> > > >>>>
> 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
> > > >>>> Searched in the following repositories:
> > > >>>> Gradle Central Plugin Repository
> > > >>>>
> > > >>>> Unfortunately, it is hard to tell what kind of error you're
> getting in
> > > >>>> your build. Any chance you can publish a snippet of the actual
> logs,
> > > >>>> rather than a screenshot?
> > > >>>>
> > > >>>> --
> > > >>>> Thanks,
> > > >>>> Cos
> > > >>>>
> > > >>>> [1]
> > > >>>>
> > > >>>>
> > >
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
> > > >>>>
> > > >>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> > > >>>>> Hello everyone!
> > > >>>>>
> > > >>>>> Sorry for repeating this in dev list, but I really need help in
> this
> > > >>>>> situation.
> > > >>>>>
> > > >>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
> > > >>>>>
> > > >>>>> Failed to execute goal
> > > >>>>>
> > > >>>>
> > >
> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> > > >>>>> (default) on project hadoop-hdfs-httpfs: An error has occurred in
> > > >>>>> Dependencies report generation.: zip file is empty -> [Help 1]
> > > >>>>>
> > > >>>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v
> > > `pwd`:/ws
> > > >>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
> > > >>>>>
> > > >>>>> I made a --scan for this attempt -
> > > https://gradle.com/s/7iu4z3fsbrh62
> > > >>>>>
> > > >>>>> Maybe I'm using not a valid way to get it perform correctly?
> > > >>>>>
> > > >>>>> Or this can be bug and I need to fill a jira ticket for this?
> > > >>>>>
> > > >>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and
> via
> > > >>>> docker
> > > >>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use
> bigtop-trunk,
> > > this
> > > >>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've been
> > > told,
> > > >>>>> currently hadoop is unbuildable in master branch.
> > > >>>>>
> > > >>>>> Plus, the same error I get on bigtop-1.3.0.
> > > >>>>>
> > > >>>>> Cleaning the local maven repository does not fix it.
> > > >>>>>
> > > >>>>> So, likely, the problem is in building environment. How I can
> > > configure
> > > >>>> it
> > > >>>>> correctly for successful hadoop building in bigtop?
> > > >>>>>
> > > >>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
> > > >>>>>
> > > >>>>>> Currently building Hadoop across all platform on master branch
> is
> > > >>>> failing:
> > > >>>>>>
> > > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > > >>>>>>
> > > >>>>>> I'm not sure whether the master code can be built successfully
> since
> > > >>>> we're
> > > >>>>>> in transition of upgrading zookeeper.
> > > >>>>>>
> > > >>>>>> If you just want to use bigtop, please get our latest release
> 1.4.0.
> > > >>>>>>
> > > >>>>>> If you are willing to contribute, you can report a bug on our
> jira
> > > first
> > > >>>>>> and we ca work on a fix together.
> > > >>>>>>
> > > >>>>>> Let us know if any question.
> > > >>>>>>
> > > >>>>>> Evan's
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
> > > >>>>>>
> > > >>>>>>> Hi Alexander,
> > > >>>>>>>
> > > >>>>>>> It seems bigtop_toolchain didn't deploy correctly.
> > > >>>>>>> Please try to build bigtop components in docker with
> bigtop/slaves
> > > >>>> docker
> > > >>>>>>> images <
> > > https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
> > > >>>>> .
> > > >>>>>>> Run:
> > > >>>>>>> cd BIGTOP_SRC_DIR
> > > >>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l
> -c 'cd
> > > >>>> /ws ;
> > > >>>>>>> gradle hadoop-pkg'
> > > >>>>>>>
> > > >>>>>>> BR,
> > > >>>>>>> Yuqi
> > > >>>>>>>
> > > >>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <
> > > iamhokum@gmail.com
> > > >>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails
> with
> > > other
> > > >>>>>>>> error.
> > > >>>>>>>>
> > > >>>>>>>> Also, in webtop-trunk this particular error appeard while
> building
> > > >>>>>> hadoop
> > > >>>>>>>> in docker (trunk-centos-7) image (by ./gradlew
> hadoop-pkg-ind) and
> > > >>>>>>> without
> > > >>>>>>>> it (by ./gradlew hadoop-rpm).
> > > >>>>>>>>
> > > >>>>>>>> Performed steps:
> > > >>>>>>>>
> > > >>>>>>>> I was trying to follow the available guides, do a clean
> centos 7
> > > >>>>>> install,
> > > >>>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip
> from
> > > >>>> repo.
> > > >>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> > > >>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
> > > >>>>>>>> bigtop_toolchain::installer".
> > > >>>>>>>>
> > > >>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew
> hadoop-rpm
> > > and
> > > >>>> got
> > > >>>>>>>> this error.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <
> ywkim@apache.org>
> > > >>>>>> wrote:
> > > >>>>>>>>> Hi Alexander,
> > > >>>>>>>>>
> > > >>>>>>>>> Which version or branch of Bigtop do you use? and also can
> you
> > > >>>>>> provide
> > > >>>>>>> us
> > > >>>>>>>>> the steps you did?
> > > >>>>>>>>>
> > > >>>>>>>>> Thanks,
> > > >>>>>>>>> Youngwoo
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> > > >>>>>>> iamhokum@gmail.com
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hello everyone.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Looks like Bigtop mailing lists is not very active, but I
> still
> > > >>>>>> hope
> > > >>>>>>>>>> someone could help me.
> > > >>>>>>>>>>
> > > >>>>>>>>>> We are trying to employ BigTop to build own hadoop
> repository
> > > >>>>>>>> (./gradlew
> > > >>>>>>>>>> hadoop-rpm), but after all efforts still getting error on
> > > zookeeper
> > > >>>>>>>>> version
> > > >>>>>>>>>> dependency:
> > > >>>>>>>>>>
> > > >>>>>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> > > >>>>>>>>>>
> > > >>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
> > > >>>>>>>>>>
> > > >>>>>>>>>> while all other conponents depends on
> > > >>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
> > > >>>>>>>>>>
> > > >>>>>>>>>> so we got:
> > > >>>>>>>>>>
> > > >>>>>>>>>> [ERROR] Failed to execute goal
> > > >>>>>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> > > >>>>>>> (depcheck)
> > > >>>>>>>>> on
> > > >>>>>>>>>> project hadoop-yarn-server-tests
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> Please advise me some solution for this show stopper, if
> > > possible.
> > > >>>>>>>>> Thanks!
> > >
> > >
>
Re: Bigtop error on Centos 7
Posted by Konstantin Boudnik <co...@apache.org>.
Actually, it isn't about Bigtop build. Bigtop is using whatever source/build
materials are particular component is supplying as a part of their release' source
tree. Yes, we do an alternation or two when the "official" stuff is totally
broken but it is rather an exception.
In this particular case, Hadoop's builds have fixed on the fly by our own
'do-component-build' script or to be fixed either in the upstream project.
--
Cos
On Sun, Jan 26, 2020 at 12:47PM, Alexander Frolushkin wrote:
> So, someone need to remade src.rpm for Hadoop, replacing url in .pom for
> this resource to become available? And now Hadoop cannot be build by any
> version of Bigtop?
>
> On Fri, Jan 24, 2020, 12:04 Olaf Flebbe <of...@oflebbe.de> wrote:
>
> > hi
> >
> > the failed downloads are http:// downloads
> >
> > http download from nexus are now turned off for security reasons.
> >
> > there was another report for oozie with the same problem on this list iirc.
> >
> > imo this can only be solved by using additional tooling, smthg like a
> > proxy changing http to https . nexus might have a property for that. or new
> > maven might have smthg in place for working around broken dependencies, but
> > i am speculating
> >
> > best
> > olaf
> >
> >
> >
> > > Am 24.01.2020 um 05:47 schrieb Konstantin Boudnik <co...@apache.org>:
> > >
> > > Looks like some sort of trouble with one of the dependencies of httpfs:
> > >
> > >
> > > Downloading from apache.snapshots.https:
> > https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > Downloading from repository.jboss.org:
> > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > [WARNING] Checksum validation failed, expected but is
> > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.
> > > org for
> > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > [WARNING] Could not validate integrity of download from
> > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar:
> > Checksum validation failed, expected but is
> > da39a3ee5e6b4b0d3255bfef95601890afd80709
> > > [WARNING] Checksum validation failed, expected but is
> > da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org for
> > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > > Downloaded from repository.jboss.org:
> > http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > (0 B at 0 B/s)
> > >
> > > And then build fails right at that module. Doesn't look like a Bigtop
> > issue, methink. I am on the road right now with somewhat limited internet,
> > will be back over the weekend and try to look more. Perhaps in the
> > meanwhile someone on the list would have any other ideas.
> > >
> > > --
> > > Cos
> > >
> > >> On 1/23/20 11:40 PM, Konstantin Boudnik wrote:
> > >> Well, I am not familiar with Gradle enterprise - looks like it has
> > everything indeed, thanks! Let me dig and see what I can figure out.
> > >>
> > >> --
> > >> Cos
> > >>
> > >>> On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
> > >>> Thank you for responding, Konstantin.
> > >>>
> > >>> Honestly, I was thinking --scan gradle will publish all necessary
> > >>> information about its execution, which may be required to inverstigate
> > the
> > >>> problem.
> > >>>
> > >>> Anyway, I'm ready to provide maximum data for this dicsussion. Firstly,
> > >>> here is a complete console output of entire hadoop-pkg-ind task till
> > the
> > >>> error -
> > >>>
> > https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
> > >>>
> > >>> If I need to collect additional data, please direct me, what exactly I
> > need
> > >>> to do to get it for processing here.
> > >>>
> > >>> I really want to build hadoop repo by bigtop-1.4.0.
> > >>>
> > >>>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org>
> > wrote:
> > >>>
> > >>>> Thanks, Alexander, for bringing this up.
> > >>>>
> > >>>> I don't see anything wrong with the way you're starting the build,
> > >>>> except that official CI is now using "hadoop-pkg-ind" target and build
> > >>>> then pulls in a correct container automatically. But I see you already
> > >>>> tried that one.
> > >>>>
> > >>>> As a side note, I am looking at [1] and see the following error (which
> > >>>> seems to be there for some time):
> > >>>>
> > >>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not found
> > in
> > >>>> any of the following sources:
> > >>>>
> > >>>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
> > >>>> - Plugin Repositories (could not resolve plugin artifact
> > >>>> 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
> > >>>> Searched in the following repositories:
> > >>>> Gradle Central Plugin Repository
> > >>>>
> > >>>> Unfortunately, it is hard to tell what kind of error you're getting in
> > >>>> your build. Any chance you can publish a snippet of the actual logs,
> > >>>> rather than a screenshot?
> > >>>>
> > >>>> --
> > >>>> Thanks,
> > >>>> Cos
> > >>>>
> > >>>> [1]
> > >>>>
> > >>>>
> > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
> > >>>>
> > >>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> > >>>>> Hello everyone!
> > >>>>>
> > >>>>> Sorry for repeating this in dev list, but I really need help in this
> > >>>>> situation.
> > >>>>>
> > >>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
> > >>>>>
> > >>>>> Failed to execute goal
> > >>>>>
> > >>>>
> > org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> > >>>>> (default) on project hadoop-hdfs-httpfs: An error has occurred in
> > >>>>> Dependencies report generation.: zip file is empty -> [Help 1]
> > >>>>>
> > >>>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v
> > `pwd`:/ws
> > >>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
> > >>>>>
> > >>>>> I made a --scan for this attempt -
> > https://gradle.com/s/7iu4z3fsbrh62
> > >>>>>
> > >>>>> Maybe I'm using not a valid way to get it perform correctly?
> > >>>>>
> > >>>>> Or this can be bug and I need to fill a jira ticket for this?
> > >>>>>
> > >>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and via
> > >>>> docker
> > >>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk,
> > this
> > >>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've been
> > told,
> > >>>>> currently hadoop is unbuildable in master branch.
> > >>>>>
> > >>>>> Plus, the same error I get on bigtop-1.3.0.
> > >>>>>
> > >>>>> Cleaning the local maven repository does not fix it.
> > >>>>>
> > >>>>> So, likely, the problem is in building environment. How I can
> > configure
> > >>>> it
> > >>>>> correctly for successful hadoop building in bigtop?
> > >>>>>
> > >>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
> > >>>>>
> > >>>>>> Currently building Hadoop across all platform on master branch is
> > >>>> failing:
> > >>>>>>
> > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > >>>>>>
> > >>>>>> I'm not sure whether the master code can be built successfully since
> > >>>> we're
> > >>>>>> in transition of upgrading zookeeper.
> > >>>>>>
> > >>>>>> If you just want to use bigtop, please get our latest release 1.4.0.
> > >>>>>>
> > >>>>>> If you are willing to contribute, you can report a bug on our jira
> > first
> > >>>>>> and we ca work on a fix together.
> > >>>>>>
> > >>>>>> Let us know if any question.
> > >>>>>>
> > >>>>>> Evan's
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
> > >>>>>>
> > >>>>>>> Hi Alexander,
> > >>>>>>>
> > >>>>>>> It seems bigtop_toolchain didn't deploy correctly.
> > >>>>>>> Please try to build bigtop components in docker with bigtop/slaves
> > >>>> docker
> > >>>>>>> images <
> > https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
> > >>>>> .
> > >>>>>>> Run:
> > >>>>>>> cd BIGTOP_SRC_DIR
> > >>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd
> > >>>> /ws ;
> > >>>>>>> gradle hadoop-pkg'
> > >>>>>>>
> > >>>>>>> BR,
> > >>>>>>> Yuqi
> > >>>>>>>
> > >>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <
> > iamhokum@gmail.com
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with
> > other
> > >>>>>>>> error.
> > >>>>>>>>
> > >>>>>>>> Also, in webtop-trunk this particular error appeard while building
> > >>>>>> hadoop
> > >>>>>>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
> > >>>>>>> without
> > >>>>>>>> it (by ./gradlew hadoop-rpm).
> > >>>>>>>>
> > >>>>>>>> Performed steps:
> > >>>>>>>>
> > >>>>>>>> I was trying to follow the available guides, do a clean centos 7
> > >>>>>> install,
> > >>>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip from
> > >>>> repo.
> > >>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> > >>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
> > >>>>>>>> bigtop_toolchain::installer".
> > >>>>>>>>
> > >>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm
> > and
> > >>>> got
> > >>>>>>>> this error.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
> > >>>>>> wrote:
> > >>>>>>>>> Hi Alexander,
> > >>>>>>>>>
> > >>>>>>>>> Which version or branch of Bigtop do you use? and also can you
> > >>>>>> provide
> > >>>>>>> us
> > >>>>>>>>> the steps you did?
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> Youngwoo
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> > >>>>>>> iamhokum@gmail.com
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hello everyone.
> > >>>>>>>>>>
> > >>>>>>>>>> Looks like Bigtop mailing lists is not very active, but I still
> > >>>>>> hope
> > >>>>>>>>>> someone could help me.
> > >>>>>>>>>>
> > >>>>>>>>>> We are trying to employ BigTop to build own hadoop repository
> > >>>>>>>> (./gradlew
> > >>>>>>>>>> hadoop-rpm), but after all efforts still getting error on
> > zookeeper
> > >>>>>>>>> version
> > >>>>>>>>>> dependency:
> > >>>>>>>>>>
> > >>>>>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> > >>>>>>>>>>
> > >>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
> > >>>>>>>>>>
> > >>>>>>>>>> while all other conponents depends on
> > >>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
> > >>>>>>>>>>
> > >>>>>>>>>> so we got:
> > >>>>>>>>>>
> > >>>>>>>>>> [ERROR] Failed to execute goal
> > >>>>>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> > >>>>>>> (depcheck)
> > >>>>>>>>> on
> > >>>>>>>>>> project hadoop-yarn-server-tests
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Please advise me some solution for this show stopper, if
> > possible.
> > >>>>>>>>> Thanks!
> >
> >
Re: Bigtop error on Centos 7
Posted by Alexander Frolushkin <ia...@gmail.com>.
So, someone need to remade src.rpm for Hadoop, replacing url in .pom for
this resource to become available? And now Hadoop cannot be build by any
version of Bigtop?
On Fri, Jan 24, 2020, 12:04 Olaf Flebbe <of...@oflebbe.de> wrote:
> hi
>
> the failed downloads are http:// downloads
>
> http download from nexus are now turned off for security reasons.
>
> there was another report for oozie with the same problem on this list iirc.
>
> imo this can only be solved by using additional tooling, smthg like a
> proxy changing http to https . nexus might have a property for that. or new
> maven might have smthg in place for working around broken dependencies, but
> i am speculating
>
> best
> olaf
>
>
>
> > Am 24.01.2020 um 05:47 schrieb Konstantin Boudnik <co...@apache.org>:
> >
> > Looks like some sort of trouble with one of the dependencies of httpfs:
> >
> >
> > Downloading from apache.snapshots.https:
> https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > Downloading from repository.jboss.org:
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > [WARNING] Checksum validation failed, expected but is
> da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.
> > org for
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > [WARNING] Could not validate integrity of download from
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar:
> Checksum validation failed, expected but is
> da39a3ee5e6b4b0d3255bfef95601890afd80709
> > [WARNING] Checksum validation failed, expected but is
> da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org for
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> > Downloaded from repository.jboss.org:
> http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> (0 B at 0 B/s)
> >
> > And then build fails right at that module. Doesn't look like a Bigtop
> issue, methink. I am on the road right now with somewhat limited internet,
> will be back over the weekend and try to look more. Perhaps in the
> meanwhile someone on the list would have any other ideas.
> >
> > --
> > Cos
> >
> >> On 1/23/20 11:40 PM, Konstantin Boudnik wrote:
> >> Well, I am not familiar with Gradle enterprise - looks like it has
> everything indeed, thanks! Let me dig and see what I can figure out.
> >>
> >> --
> >> Cos
> >>
> >>> On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
> >>> Thank you for responding, Konstantin.
> >>>
> >>> Honestly, I was thinking --scan gradle will publish all necessary
> >>> information about its execution, which may be required to inverstigate
> the
> >>> problem.
> >>>
> >>> Anyway, I'm ready to provide maximum data for this dicsussion. Firstly,
> >>> here is a complete console output of entire hadoop-pkg-ind task till
> the
> >>> error -
> >>>
> https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
> >>>
> >>> If I need to collect additional data, please direct me, what exactly I
> need
> >>> to do to get it for processing here.
> >>>
> >>> I really want to build hadoop repo by bigtop-1.4.0.
> >>>
> >>>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org>
> wrote:
> >>>
> >>>> Thanks, Alexander, for bringing this up.
> >>>>
> >>>> I don't see anything wrong with the way you're starting the build,
> >>>> except that official CI is now using "hadoop-pkg-ind" target and build
> >>>> then pulls in a correct container automatically. But I see you already
> >>>> tried that one.
> >>>>
> >>>> As a side note, I am looking at [1] and see the following error (which
> >>>> seems to be there for some time):
> >>>>
> >>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not found
> in
> >>>> any of the following sources:
> >>>>
> >>>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
> >>>> - Plugin Repositories (could not resolve plugin artifact
> >>>> 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
> >>>> Searched in the following repositories:
> >>>> Gradle Central Plugin Repository
> >>>>
> >>>> Unfortunately, it is hard to tell what kind of error you're getting in
> >>>> your build. Any chance you can publish a snippet of the actual logs,
> >>>> rather than a screenshot?
> >>>>
> >>>> --
> >>>> Thanks,
> >>>> Cos
> >>>>
> >>>> [1]
> >>>>
> >>>>
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
> >>>>
> >>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> >>>>> Hello everyone!
> >>>>>
> >>>>> Sorry for repeating this in dev list, but I really need help in this
> >>>>> situation.
> >>>>>
> >>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
> >>>>>
> >>>>> Failed to execute goal
> >>>>>
> >>>>
> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> >>>>> (default) on project hadoop-hdfs-httpfs: An error has occurred in
> >>>>> Dependencies report generation.: zip file is empty -> [Help 1]
> >>>>>
> >>>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v
> `pwd`:/ws
> >>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
> >>>>>
> >>>>> I made a --scan for this attempt -
> https://gradle.com/s/7iu4z3fsbrh62
> >>>>>
> >>>>> Maybe I'm using not a valid way to get it perform correctly?
> >>>>>
> >>>>> Or this can be bug and I need to fill a jira ticket for this?
> >>>>>
> >>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and via
> >>>> docker
> >>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk,
> this
> >>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've been
> told,
> >>>>> currently hadoop is unbuildable in master branch.
> >>>>>
> >>>>> Plus, the same error I get on bigtop-1.3.0.
> >>>>>
> >>>>> Cleaning the local maven repository does not fix it.
> >>>>>
> >>>>> So, likely, the problem is in building environment. How I can
> configure
> >>>> it
> >>>>> correctly for successful hadoop building in bigtop?
> >>>>>
> >>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
> >>>>>
> >>>>>> Currently building Hadoop across all platform on master branch is
> >>>> failing:
> >>>>>>
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >>>>>>
> >>>>>> I'm not sure whether the master code can be built successfully since
> >>>> we're
> >>>>>> in transition of upgrading zookeeper.
> >>>>>>
> >>>>>> If you just want to use bigtop, please get our latest release 1.4.0.
> >>>>>>
> >>>>>> If you are willing to contribute, you can report a bug on our jira
> first
> >>>>>> and we ca work on a fix together.
> >>>>>>
> >>>>>> Let us know if any question.
> >>>>>>
> >>>>>> Evan's
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
> >>>>>>
> >>>>>>> Hi Alexander,
> >>>>>>>
> >>>>>>> It seems bigtop_toolchain didn't deploy correctly.
> >>>>>>> Please try to build bigtop components in docker with bigtop/slaves
> >>>> docker
> >>>>>>> images <
> https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
> >>>>> .
> >>>>>>> Run:
> >>>>>>> cd BIGTOP_SRC_DIR
> >>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd
> >>>> /ws ;
> >>>>>>> gradle hadoop-pkg'
> >>>>>>>
> >>>>>>> BR,
> >>>>>>> Yuqi
> >>>>>>>
> >>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <
> iamhokum@gmail.com
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with
> other
> >>>>>>>> error.
> >>>>>>>>
> >>>>>>>> Also, in webtop-trunk this particular error appeard while building
> >>>>>> hadoop
> >>>>>>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
> >>>>>>> without
> >>>>>>>> it (by ./gradlew hadoop-rpm).
> >>>>>>>>
> >>>>>>>> Performed steps:
> >>>>>>>>
> >>>>>>>> I was trying to follow the available guides, do a clean centos 7
> >>>>>> install,
> >>>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip from
> >>>> repo.
> >>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> >>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
> >>>>>>>> bigtop_toolchain::installer".
> >>>>>>>>
> >>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm
> and
> >>>> got
> >>>>>>>> this error.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
> >>>>>> wrote:
> >>>>>>>>> Hi Alexander,
> >>>>>>>>>
> >>>>>>>>> Which version or branch of Bigtop do you use? and also can you
> >>>>>> provide
> >>>>>>> us
> >>>>>>>>> the steps you did?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Youngwoo
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> >>>>>>> iamhokum@gmail.com
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hello everyone.
> >>>>>>>>>>
> >>>>>>>>>> Looks like Bigtop mailing lists is not very active, but I still
> >>>>>> hope
> >>>>>>>>>> someone could help me.
> >>>>>>>>>>
> >>>>>>>>>> We are trying to employ BigTop to build own hadoop repository
> >>>>>>>> (./gradlew
> >>>>>>>>>> hadoop-rpm), but after all efforts still getting error on
> zookeeper
> >>>>>>>>> version
> >>>>>>>>>> dependency:
> >>>>>>>>>>
> >>>>>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> >>>>>>>>>>
> >>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
> >>>>>>>>>>
> >>>>>>>>>> while all other conponents depends on
> >>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
> >>>>>>>>>>
> >>>>>>>>>> so we got:
> >>>>>>>>>>
> >>>>>>>>>> [ERROR] Failed to execute goal
> >>>>>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> >>>>>>> (depcheck)
> >>>>>>>>> on
> >>>>>>>>>> project hadoop-yarn-server-tests
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Please advise me some solution for this show stopper, if
> possible.
> >>>>>>>>> Thanks!
>
>
Re: Bigtop error on Centos 7
Posted by Olaf Flebbe <of...@oflebbe.de>.
hi
the failed downloads are http:// downloads
http download from nexus are now turned off for security reasons.
there was another report for oozie with the same problem on this list iirc.
imo this can only be solved by using additional tooling, smthg like a proxy changing http to https . nexus might have a property for that. or new maven might have smthg in place for working around broken dependencies, but i am speculating
best
olaf
> Am 24.01.2020 um 05:47 schrieb Konstantin Boudnik <co...@apache.org>:
>
> Looks like some sort of trouble with one of the dependencies of httpfs:
>
>
> Downloading from apache.snapshots.https: https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> Downloading from repository.jboss.org: http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> [WARNING] Checksum validation failed, expected but is da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.
> org for http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> [WARNING] Could not validate integrity of download from http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar: Checksum validation failed, expected but is da39a3ee5e6b4b0d3255bfef95601890afd80709
> [WARNING] Checksum validation failed, expected but is da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org for http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
> Downloaded from repository.jboss.org: http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar (0 B at 0 B/s)
>
> And then build fails right at that module. Doesn't look like a Bigtop issue, methink. I am on the road right now with somewhat limited internet, will be back over the weekend and try to look more. Perhaps in the meanwhile someone on the list would have any other ideas.
>
> --
> Cos
>
>> On 1/23/20 11:40 PM, Konstantin Boudnik wrote:
>> Well, I am not familiar with Gradle enterprise - looks like it has everything indeed, thanks! Let me dig and see what I can figure out.
>>
>> --
>> Cos
>>
>>> On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
>>> Thank you for responding, Konstantin.
>>>
>>> Honestly, I was thinking --scan gradle will publish all necessary
>>> information about its execution, which may be required to inverstigate the
>>> problem.
>>>
>>> Anyway, I'm ready to provide maximum data for this dicsussion. Firstly,
>>> here is a complete console output of entire hadoop-pkg-ind task till the
>>> error -
>>> https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
>>>
>>> If I need to collect additional data, please direct me, what exactly I need
>>> to do to get it for processing here.
>>>
>>> I really want to build hadoop repo by bigtop-1.4.0.
>>>
>>>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org> wrote:
>>>
>>>> Thanks, Alexander, for bringing this up.
>>>>
>>>> I don't see anything wrong with the way you're starting the build,
>>>> except that official CI is now using "hadoop-pkg-ind" target and build
>>>> then pulls in a correct container automatically. But I see you already
>>>> tried that one.
>>>>
>>>> As a side note, I am looking at [1] and see the following error (which
>>>> seems to be there for some time):
>>>>
>>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not found in
>>>> any of the following sources:
>>>>
>>>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
>>>> - Plugin Repositories (could not resolve plugin artifact
>>>> 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
>>>> Searched in the following repositories:
>>>> Gradle Central Plugin Repository
>>>>
>>>> Unfortunately, it is hard to tell what kind of error you're getting in
>>>> your build. Any chance you can publish a snippet of the actual logs,
>>>> rather than a screenshot?
>>>>
>>>> --
>>>> Thanks,
>>>> Cos
>>>>
>>>> [1]
>>>>
>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
>>>>
>>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
>>>>> Hello everyone!
>>>>>
>>>>> Sorry for repeating this in dev list, but I really need help in this
>>>>> situation.
>>>>>
>>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
>>>>>
>>>>> Failed to execute goal
>>>>>
>>>> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
>>>>> (default) on project hadoop-hdfs-httpfs: An error has occurred in
>>>>> Dependencies report generation.: zip file is empty -> [Help 1]
>>>>>
>>>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v `pwd`:/ws
>>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
>>>>>
>>>>> I made a --scan for this attempt - https://gradle.com/s/7iu4z3fsbrh62
>>>>>
>>>>> Maybe I'm using not a valid way to get it perform correctly?
>>>>>
>>>>> Or this can be bug and I need to fill a jira ticket for this?
>>>>>
>>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and via
>>>> docker
>>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk, this
>>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've been told,
>>>>> currently hadoop is unbuildable in master branch.
>>>>>
>>>>> Plus, the same error I get on bigtop-1.3.0.
>>>>>
>>>>> Cleaning the local maven repository does not fix it.
>>>>>
>>>>> So, likely, the problem is in building environment. How I can configure
>>>> it
>>>>> correctly for successful hadoop building in bigtop?
>>>>>
>>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
>>>>>
>>>>>> Currently building Hadoop across all platform on master branch is
>>>> failing:
>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>>>
>>>>>> I'm not sure whether the master code can be built successfully since
>>>> we're
>>>>>> in transition of upgrading zookeeper.
>>>>>>
>>>>>> If you just want to use bigtop, please get our latest release 1.4.0.
>>>>>>
>>>>>> If you are willing to contribute, you can report a bug on our jira first
>>>>>> and we ca work on a fix together.
>>>>>>
>>>>>> Let us know if any question.
>>>>>>
>>>>>> Evan's
>>>>>>
>>>>>>
>>>>>>
>>>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
>>>>>>
>>>>>>> Hi Alexander,
>>>>>>>
>>>>>>> It seems bigtop_toolchain didn't deploy correctly.
>>>>>>> Please try to build bigtop components in docker with bigtop/slaves
>>>> docker
>>>>>>> images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
>>>>> .
>>>>>>> Run:
>>>>>>> cd BIGTOP_SRC_DIR
>>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd
>>>> /ws ;
>>>>>>> gradle hadoop-pkg'
>>>>>>>
>>>>>>> BR,
>>>>>>> Yuqi
>>>>>>>
>>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <iamhokum@gmail.com
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
>>>>>>>> error.
>>>>>>>>
>>>>>>>> Also, in webtop-trunk this particular error appeard while building
>>>>>> hadoop
>>>>>>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
>>>>>>> without
>>>>>>>> it (by ./gradlew hadoop-rpm).
>>>>>>>>
>>>>>>>> Performed steps:
>>>>>>>>
>>>>>>>> I was trying to follow the available guides, do a clean centos 7
>>>>>> install,
>>>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip from
>>>> repo.
>>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
>>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
>>>>>>>> bigtop_toolchain::installer".
>>>>>>>>
>>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and
>>>> got
>>>>>>>> this error.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
>>>>>> wrote:
>>>>>>>>> Hi Alexander,
>>>>>>>>>
>>>>>>>>> Which version or branch of Bigtop do you use? and also can you
>>>>>> provide
>>>>>>> us
>>>>>>>>> the steps you did?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Youngwoo
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
>>>>>>> iamhokum@gmail.com
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello everyone.
>>>>>>>>>>
>>>>>>>>>> Looks like Bigtop mailing lists is not very active, but I still
>>>>>> hope
>>>>>>>>>> someone could help me.
>>>>>>>>>>
>>>>>>>>>> We are trying to employ BigTop to build own hadoop repository
>>>>>>>> (./gradlew
>>>>>>>>>> hadoop-rpm), but after all efforts still getting error on zookeeper
>>>>>>>>> version
>>>>>>>>>> dependency:
>>>>>>>>>>
>>>>>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
>>>>>>>>>>
>>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
>>>>>>>>>>
>>>>>>>>>> while all other conponents depends on
>>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
>>>>>>>>>>
>>>>>>>>>> so we got:
>>>>>>>>>>
>>>>>>>>>> [ERROR] Failed to execute goal
>>>>>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
>>>>>>> (depcheck)
>>>>>>>>> on
>>>>>>>>>> project hadoop-yarn-server-tests
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please advise me some solution for this show stopper, if possible.
>>>>>>>>> Thanks!
Re: Bigtop error on Centos 7
Posted by Konstantin Boudnik <co...@apache.org>.
Looks like some sort of trouble with one of the dependencies of httpfs:
Downloading from apache.snapshots.https:
https://repository.apache.org/content/repositories/snapshots/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
Downloading from repository.jboss.org:
http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
[WARNING] Checksum validation failed, expected but is
da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.
org for
http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
[WARNING] Could not validate integrity of download from
http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar:
Checksum validation failed, expected but is
da39a3ee5e6b4b0d3255bfef95601890afd80709
[WARNING] Checksum validation failed, expected but is
da39a3ee5e6b4b0d3255bfef95601890afd80709 from repository.jboss.org for
http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
Downloaded from repository.jboss.org:
http://repository.jboss.org/nexus/content/groups/public/org/apache/maven/skins/maven-default-skin/1.0/maven-default-skin-1.0.jar
(0 B at 0 B/s)
And then build fails right at that module. Doesn't look like a Bigtop
issue, methink. I am on the road right now with somewhat limited
internet, will be back over the weekend and try to look more. Perhaps in
the meanwhile someone on the list would have any other ideas.
--
Cos
On 1/23/20 11:40 PM, Konstantin Boudnik wrote:
> Well, I am not familiar with Gradle enterprise - looks like it has
> everything indeed, thanks! Let me dig and see what I can figure out.
>
> --
> Cos
>
> On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
>> Thank you for responding, Konstantin.
>>
>> Honestly, I was thinking --scan gradle will publish all necessary
>> information about its execution, which may be required to
>> inverstigate the
>> problem.
>>
>> Anyway, I'm ready to provide maximum data for this dicsussion. Firstly,
>> here is a complete console output of entire hadoop-pkg-ind task till the
>> error -
>> https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
>>
>>
>> If I need to collect additional data, please direct me, what exactly
>> I need
>> to do to get it for processing here.
>>
>> I really want to build hadoop repo by bigtop-1.4.0.
>>
>> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org> wrote:
>>
>>> Thanks, Alexander, for bringing this up.
>>>
>>> I don't see anything wrong with the way you're starting the build,
>>> except that official CI is now using "hadoop-pkg-ind" target and build
>>> then pulls in a correct container automatically. But I see you already
>>> tried that one.
>>>
>>> As a side note, I am looking at [1] and see the following error (which
>>> seems to be there for some time):
>>>
>>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not
>>> found in
>>> any of the following sources:
>>>
>>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
>>> - Plugin Repositories (could not resolve plugin artifact
>>> 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
>>> Searched in the following repositories:
>>> Gradle Central Plugin Repository
>>>
>>> Unfortunately, it is hard to tell what kind of error you're getting in
>>> your build. Any chance you can publish a snippet of the actual logs,
>>> rather than a screenshot?
>>>
>>> --
>>> Thanks,
>>> Cos
>>>
>>> [1]
>>>
>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
>>>
>>>
>>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
>>>> Hello everyone!
>>>>
>>>> Sorry for repeating this in dev list, but I really need help in this
>>>> situation.
>>>>
>>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
>>>>
>>>> Failed to execute goal
>>>>
>>> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
>>>
>>>> (default) on project hadoop-hdfs-httpfs: An error has occurred in
>>>> Dependencies report generation.: zip file is empty -> [Help 1]
>>>>
>>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v
>>>> `pwd`:/ws
>>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
>>>>
>>>> I made a --scan for this attempt - https://gradle.com/s/7iu4z3fsbrh62
>>>>
>>>> Maybe I'm using not a valid way to get it perform correctly?
>>>>
>>>> Or this can be bug and I need to fill a jira ticket for this?
>>>>
>>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and via
>>> docker
>>>> ./gradlew hadoop-pkg-ind, got the same error. If I use
>>>> bigtop-trunk, this
>>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've been
>>>> told,
>>>> currently hadoop is unbuildable in master branch.
>>>>
>>>> Plus, the same error I get on bigtop-1.3.0.
>>>>
>>>> Cleaning the local maven repository does not fix it.
>>>>
>>>> So, likely, the problem is in building environment. How I can
>>>> configure
>>> it
>>>> correctly for successful hadoop building in bigtop?
>>>>
>>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
>>>>
>>>>> Currently building Hadoop across all platform on master branch is
>>> failing:
>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>>
>>>>> I'm not sure whether the master code can be built successfully since
>>> we're
>>>>> in transition of upgrading zookeeper.
>>>>>
>>>>> If you just want to use bigtop, please get our latest release 1.4.0.
>>>>>
>>>>> If you are willing to contribute, you can report a bug on our jira
>>>>> first
>>>>> and we ca work on a fix together.
>>>>>
>>>>> Let us know if any question.
>>>>>
>>>>> Evan's
>>>>>
>>>>>
>>>>>
>>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
>>>>>
>>>>>> Hi Alexander,
>>>>>>
>>>>>> It seems bigtop_toolchain didn't deploy correctly.
>>>>>> Please try to build bigtop components in docker with bigtop/slaves
>>> docker
>>>>>> images
>>>>>> <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
>>>> .
>>>>>> Run:
>>>>>> cd BIGTOP_SRC_DIR
>>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd
>>> /ws ;
>>>>>> gradle hadoop-pkg'
>>>>>>
>>>>>> BR,
>>>>>> Yuqi
>>>>>>
>>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin
>>>>>> <iamhokum@gmail.com
>>>>>> wrote:
>>>>>>
>>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with
>>>>>>> other
>>>>>>> error.
>>>>>>>
>>>>>>> Also, in webtop-trunk this particular error appeard while building
>>>>> hadoop
>>>>>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
>>>>>> without
>>>>>>> it (by ./gradlew hadoop-rpm).
>>>>>>>
>>>>>>> Performed steps:
>>>>>>>
>>>>>>> I was trying to follow the available guides, do a clean centos 7
>>>>> install,
>>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip from
>>> repo.
>>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
>>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
>>>>>>> bigtop_toolchain::installer".
>>>>>>>
>>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and
>>> got
>>>>>>> this error.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우)
>>>>>>> <yw...@apache.org>
>>>>> wrote:
>>>>>>>> Hi Alexander,
>>>>>>>>
>>>>>>>> Which version or branch of Bigtop do you use? and also can you
>>>>> provide
>>>>>> us
>>>>>>>> the steps you did?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Youngwoo
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
>>>>>> iamhokum@gmail.com
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello everyone.
>>>>>>>>>
>>>>>>>>> Looks like Bigtop mailing lists is not very active, but I still
>>>>> hope
>>>>>>>>> someone could help me.
>>>>>>>>>
>>>>>>>>> We are trying to employ BigTop to build own hadoop repository
>>>>>>> (./gradlew
>>>>>>>>> hadoop-rpm), but after all efforts still getting error on
>>>>>>>>> zookeeper
>>>>>>>> version
>>>>>>>>> dependency:
>>>>>>>>>
>>>>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
>>>>>>>>>
>>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
>>>>>>>>>
>>>>>>>>> while all other conponents depends on
>>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
>>>>>>>>>
>>>>>>>>> so we got:
>>>>>>>>>
>>>>>>>>> [ERROR] Failed to execute goal
>>>>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
>>>>>> (depcheck)
>>>>>>>> on
>>>>>>>>> project hadoop-yarn-server-tests
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please advise me some solution for this show stopper, if
>>>>>>>>> possible.
>>>>>>>> Thanks!
Re: Bigtop error on Centos 7
Posted by Konstantin Boudnik <co...@apache.org>.
Well, I am not familiar with Gradle enterprise - looks like it has
everything indeed, thanks! Let me dig and see what I can figure out.
--
Cos
On 1/23/20 11:28 PM, Alexander Frolushkin wrote:
> Thank you for responding, Konstantin.
>
> Honestly, I was thinking --scan gradle will publish all necessary
> information about its execution, which may be required to inverstigate the
> problem.
>
> Anyway, I'm ready to provide maximum data for this dicsussion. Firstly,
> here is a complete console output of entire hadoop-pkg-ind task till the
> error -
> https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
>
> If I need to collect additional data, please direct me, what exactly I need
> to do to get it for processing here.
>
> I really want to build hadoop repo by bigtop-1.4.0.
>
> On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org> wrote:
>
>> Thanks, Alexander, for bringing this up.
>>
>> I don't see anything wrong with the way you're starting the build,
>> except that official CI is now using "hadoop-pkg-ind" target and build
>> then pulls in a correct container automatically. But I see you already
>> tried that one.
>>
>> As a side note, I am looking at [1] and see the following error (which
>> seems to be there for some time):
>>
>> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not found in
>> any of the following sources:
>>
>> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
>> - Plugin Repositories (could not resolve plugin artifact
>> 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
>> Searched in the following repositories:
>> Gradle Central Plugin Repository
>>
>> Unfortunately, it is hard to tell what kind of error you're getting in
>> your build. Any chance you can publish a snippet of the actual logs,
>> rather than a screenshot?
>>
>> --
>> Thanks,
>> Cos
>>
>> [1]
>>
>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
>>
>> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
>>> Hello everyone!
>>>
>>> Sorry for repeating this in dev list, but I really need help in this
>>> situation.
>>>
>>> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
>>>
>>> Failed to execute goal
>>>
>> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
>>> (default) on project hadoop-hdfs-httpfs: An error has occurred in
>>> Dependencies report generation.: zip file is empty -> [Help 1]
>>>
>>> Launched in clean bigtop-1.4.0 dir, with command docker run -v `pwd`:/ws
>>> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
>>>
>>> I made a --scan for this attempt - https://gradle.com/s/7iu4z3fsbrh62
>>>
>>> Maybe I'm using not a valid way to get it perform correctly?
>>>
>>> Or this can be bug and I need to fill a jira ticket for this?
>>>
>>> Also I tryed to use direct building by ./gradlew hadoop-rpm and via
>> docker
>>> ./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk, this
>>> stage (hadoop-hdfs-httpfs) passed without error, but as I've been told,
>>> currently hadoop is unbuildable in master branch.
>>>
>>> Plus, the same error I get on bigtop-1.3.0.
>>>
>>> Cleaning the local maven repository does not fix it.
>>>
>>> So, likely, the problem is in building environment. How I can configure
>> it
>>> correctly for successful hadoop building in bigtop?
>>>
>>> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
>>>
>>>> Currently building Hadoop across all platform on master branch is
>> failing:
>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>
>>>> I'm not sure whether the master code can be built successfully since
>> we're
>>>> in transition of upgrading zookeeper.
>>>>
>>>> If you just want to use bigtop, please get our latest release 1.4.0.
>>>>
>>>> If you are willing to contribute, you can report a bug on our jira first
>>>> and we ca work on a fix together.
>>>>
>>>> Let us know if any question.
>>>>
>>>> Evan's
>>>>
>>>>
>>>>
>>>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
>>>>
>>>>> Hi Alexander,
>>>>>
>>>>> It seems bigtop_toolchain didn't deploy correctly.
>>>>> Please try to build bigtop components in docker with bigtop/slaves
>> docker
>>>>> images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
>>> .
>>>>> Run:
>>>>> cd BIGTOP_SRC_DIR
>>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd
>> /ws ;
>>>>> gradle hadoop-pkg'
>>>>>
>>>>> BR,
>>>>> Yuqi
>>>>>
>>>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <iamhokum@gmail.com
>>>>> wrote:
>>>>>
>>>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
>>>>>> error.
>>>>>>
>>>>>> Also, in webtop-trunk this particular error appeard while building
>>>> hadoop
>>>>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
>>>>> without
>>>>>> it (by ./gradlew hadoop-rpm).
>>>>>>
>>>>>> Performed steps:
>>>>>>
>>>>>> I was trying to follow the available guides, do a clean centos 7
>>>> install,
>>>>>> upgrade to latest official rpms, install openjdk 7 and unzip from
>> repo.
>>>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
>>>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
>>>>>> bigtop_toolchain::installer".
>>>>>>
>>>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and
>> got
>>>>>> this error.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
>>>> wrote:
>>>>>>> Hi Alexander,
>>>>>>>
>>>>>>> Which version or branch of Bigtop do you use? and also can you
>>>> provide
>>>>> us
>>>>>>> the steps you did?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Youngwoo
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
>>>>> iamhokum@gmail.com
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello everyone.
>>>>>>>>
>>>>>>>> Looks like Bigtop mailing lists is not very active, but I still
>>>> hope
>>>>>>>> someone could help me.
>>>>>>>>
>>>>>>>> We are trying to employ BigTop to build own hadoop repository
>>>>>> (./gradlew
>>>>>>>> hadoop-rpm), but after all efforts still getting error on zookeeper
>>>>>>> version
>>>>>>>> dependency:
>>>>>>>>
>>>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
>>>>>>>>
>>>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
>>>>>>>>
>>>>>>>> while all other conponents depends on
>>>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
>>>>>>>>
>>>>>>>> so we got:
>>>>>>>>
>>>>>>>> [ERROR] Failed to execute goal
>>>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
>>>>> (depcheck)
>>>>>>> on
>>>>>>>> project hadoop-yarn-server-tests
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Please advise me some solution for this show stopper, if possible.
>>>>>>> Thanks!
Re: Bigtop error on Centos 7
Posted by Alexander Frolushkin <ia...@gmail.com>.
Thank you for responding, Konstantin.
Honestly, I was thinking --scan gradle will publish all necessary
information about its execution, which may be required to inverstigate the
problem.
Anyway, I'm ready to provide maximum data for this dicsussion. Firstly,
here is a complete console output of entire hadoop-pkg-ind task till the
error -
https://drive.google.com/file/d/16zmFTj485RQ44BFMh5zlhB4cyUzmyaNR/view?usp=drivesdk
If I need to collect additional data, please direct me, what exactly I need
to do to get it for processing here.
I really want to build hadoop repo by bigtop-1.4.0.
On Thu, Jan 23, 2020, 22:07 Konstantin Boudnik <co...@apache.org> wrote:
> Thanks, Alexander, for bringing this up.
>
> I don't see anything wrong with the way you're starting the build,
> except that official CI is now using "hadoop-pkg-ind" target and build
> then pulls in a correct container automatically. But I see you already
> tried that one.
>
> As a side note, I am looking at [1] and see the following error (which
> seems to be there for some time):
>
> Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not found in
> any of the following sources:
>
> - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
> - Plugin Repositories (could not resolve plugin artifact
> 'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
> Searched in the following repositories:
> Gradle Central Plugin Repository
>
> Unfortunately, it is hard to tell what kind of error you're getting in
> your build. Any chance you can publish a snippet of the actual logs,
> rather than a screenshot?
>
> --
> Thanks,
> Cos
>
> [1]
>
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
>
> On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> > Hello everyone!
> >
> > Sorry for repeating this in dev list, but I really need help in this
> > situation.
> >
> > On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
> >
> > Failed to execute goal
> >
> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> > (default) on project hadoop-hdfs-httpfs: An error has occurred in
> > Dependencies report generation.: zip file is empty -> [Help 1]
> >
> > Launched in clean bigtop-1.4.0 dir, with command docker run -v `pwd`:/ws
> > bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
> >
> > I made a --scan for this attempt - https://gradle.com/s/7iu4z3fsbrh62
> >
> > Maybe I'm using not a valid way to get it perform correctly?
> >
> > Or this can be bug and I need to fill a jira ticket for this?
> >
> > Also I tryed to use direct building by ./gradlew hadoop-rpm and via
> docker
> > ./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk, this
> > stage (hadoop-hdfs-httpfs) passed without error, but as I've been told,
> > currently hadoop is unbuildable in master branch.
> >
> > Plus, the same error I get on bigtop-1.3.0.
> >
> > Cleaning the local maven repository does not fix it.
> >
> > So, likely, the problem is in building environment. How I can configure
> it
> > correctly for successful hadoop building in bigtop?
> >
> > On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
> >
> >> Currently building Hadoop across all platform on master branch is
> failing:
> >> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >>
> >> I'm not sure whether the master code can be built successfully since
> we're
> >> in transition of upgrading zookeeper.
> >>
> >> If you just want to use bigtop, please get our latest release 1.4.0.
> >>
> >> If you are willing to contribute, you can report a bug on our jira first
> >> and we ca work on a fix together.
> >>
> >> Let us know if any question.
> >>
> >> Evan's
> >>
> >>
> >>
> >> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
> >>
> >>> Hi Alexander,
> >>>
> >>> It seems bigtop_toolchain didn't deploy correctly.
> >>> Please try to build bigtop components in docker with bigtop/slaves
> docker
> >>> images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos
> >.
> >>>
> >>> Run:
> >>> cd BIGTOP_SRC_DIR
> >>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd
> /ws ;
> >>> gradle hadoop-pkg'
> >>>
> >>> BR,
> >>> Yuqi
> >>>
> >>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <iamhokum@gmail.com
> >
> >>> wrote:
> >>>
> >>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
> >>>> error.
> >>>>
> >>>> Also, in webtop-trunk this particular error appeard while building
> >> hadoop
> >>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
> >>> without
> >>>> it (by ./gradlew hadoop-rpm).
> >>>>
> >>>> Performed steps:
> >>>>
> >>>> I was trying to follow the available guides, do a clean centos 7
> >> install,
> >>>> upgrade to latest official rpms, install openjdk 7 and unzip from
> repo.
> >>>>
> >>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> >>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
> >>>> bigtop_toolchain::installer".
> >>>>
> >>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and
> got
> >>>> this error.
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
> >> wrote:
> >>>>> Hi Alexander,
> >>>>>
> >>>>> Which version or branch of Bigtop do you use? and also can you
> >> provide
> >>> us
> >>>>> the steps you did?
> >>>>>
> >>>>> Thanks,
> >>>>> Youngwoo
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> >>> iamhokum@gmail.com
> >>>>> wrote:
> >>>>>
> >>>>>> Hello everyone.
> >>>>>>
> >>>>>> Looks like Bigtop mailing lists is not very active, but I still
> >> hope
> >>>>>> someone could help me.
> >>>>>>
> >>>>>> We are trying to employ BigTop to build own hadoop repository
> >>>> (./gradlew
> >>>>>> hadoop-rpm), but after all efforts still getting error on zookeeper
> >>>>> version
> >>>>>> dependency:
> >>>>>>
> >>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> >>>>>>
> >>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
> >>>>>>
> >>>>>> while all other conponents depends on
> >>>>>> -org.apache.zookeeper:zookeeper:3.4.13
> >>>>>>
> >>>>>> so we got:
> >>>>>>
> >>>>>> [ERROR] Failed to execute goal
> >>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> >>> (depcheck)
> >>>>> on
> >>>>>> project hadoop-yarn-server-tests
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Please advise me some solution for this show stopper, if possible.
> >>>>> Thanks!
>
Re: Bigtop error on Centos 7
Posted by Konstantin Boudnik <co...@apache.org>.
Thanks, Alexander, for bringing this up.
I don't see anything wrong with the way you're starting the build,
except that official CI is now using "hadoop-pkg-ind" target and build
then pulls in a correct container automatically. But I see you already
tried that one.
As a side note, I am looking at [1] and see the following error (which
seems to be there for some time):
Plugin [id: 'de.undercouch.download', version: '3.2.0'] was not found in
any of the following sources:
- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Plugin Repositories (could not resolve plugin artifact
'de.undercouch.download:de.undercouch.download.gradle.plugin:3.2.0')
Searched in the following repositories:
Gradle Central Plugin Repository
Unfortunately, it is hard to tell what kind of error you're getting in
your build. Any chance you can publish a snippet of the actual logs,
rather than a screenshot?
--
Thanks,
Cos
[1]
https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=centos-7/567/console
On 1/23/20 4:50 AM, Alexander Frolushkin wrote:
> Hello everyone!
>
> Sorry for repeating this in dev list, but I really need help in this
> situation.
>
> On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
>
> Failed to execute goal
> org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
> (default) on project hadoop-hdfs-httpfs: An error has occurred in
> Dependencies report generation.: zip file is empty -> [Help 1]
>
> Launched in clean bigtop-1.4.0 dir, with command docker run -v `pwd`:/ws
> bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
>
> I made a --scan for this attempt - https://gradle.com/s/7iu4z3fsbrh62
>
> Maybe I'm using not a valid way to get it perform correctly?
>
> Or this can be bug and I need to fill a jira ticket for this?
>
> Also I tryed to use direct building by ./gradlew hadoop-rpm and via docker
> ./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk, this
> stage (hadoop-hdfs-httpfs) passed without error, but as I've been told,
> currently hadoop is unbuildable in master branch.
>
> Plus, the same error I get on bigtop-1.3.0.
>
> Cleaning the local maven repository does not fix it.
>
> So, likely, the problem is in building environment. How I can configure it
> correctly for successful hadoop building in bigtop?
>
> On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
>
>> Currently building Hadoop across all platform on master branch is failing:
>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>
>> I'm not sure whether the master code can be built successfully since we're
>> in transition of upgrading zookeeper.
>>
>> If you just want to use bigtop, please get our latest release 1.4.0.
>>
>> If you are willing to contribute, you can report a bug on our jira first
>> and we ca work on a fix together.
>>
>> Let us know if any question.
>>
>> Evan's
>>
>>
>>
>> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
>>
>>> Hi Alexander,
>>>
>>> It seems bigtop_toolchain didn't deploy correctly.
>>> Please try to build bigtop components in docker with bigtop/slaves docker
>>> images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos>.
>>>
>>> Run:
>>> cd BIGTOP_SRC_DIR
>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws ;
>>> gradle hadoop-pkg'
>>>
>>> BR,
>>> Yuqi
>>>
>>> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <ia...@gmail.com>
>>> wrote:
>>>
>>>> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
>>>> error.
>>>>
>>>> Also, in webtop-trunk this particular error appeard while building
>> hadoop
>>>> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
>>> without
>>>> it (by ./gradlew hadoop-rpm).
>>>>
>>>> Performed steps:
>>>>
>>>> I was trying to follow the available guides, do a clean centos 7
>> install,
>>>> upgrade to latest official rpms, install openjdk 7 and unzip from repo.
>>>>
>>>> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
>>>> --modulepath=`pwd`:/etc/puppet/modules -e "include
>>>> bigtop_toolchain::installer".
>>>>
>>>> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and got
>>>> this error.
>>>>
>>>>
>>>>
>>>> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
>> wrote:
>>>>> Hi Alexander,
>>>>>
>>>>> Which version or branch of Bigtop do you use? and also can you
>> provide
>>> us
>>>>> the steps you did?
>>>>>
>>>>> Thanks,
>>>>> Youngwoo
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
>>> iamhokum@gmail.com
>>>>> wrote:
>>>>>
>>>>>> Hello everyone.
>>>>>>
>>>>>> Looks like Bigtop mailing lists is not very active, but I still
>> hope
>>>>>> someone could help me.
>>>>>>
>>>>>> We are trying to employ BigTop to build own hadoop repository
>>>> (./gradlew
>>>>>> hadoop-rpm), but after all efforts still getting error on zookeeper
>>>>> version
>>>>>> dependency:
>>>>>>
>>>>>> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
>>>>>>
>>>>>> it wants -org.apache.zookeeper:zookeeper:3.4.6
>>>>>>
>>>>>> while all other conponents depends on
>>>>>> -org.apache.zookeeper:zookeeper:3.4.13
>>>>>>
>>>>>> so we got:
>>>>>>
>>>>>> [ERROR] Failed to execute goal
>>>>>> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
>>> (depcheck)
>>>>> on
>>>>>> project hadoop-yarn-server-tests
>>>>>>
>>>>>>
>>>>>>
>>>>>> Please advise me some solution for this show stopper, if possible.
>>>>> Thanks!
Re: Bigtop error on Centos 7
Posted by Alexander Frolushkin <ia...@gmail.com>.
Hello everyone!
Sorry for repeating this in dev list, but I really need help in this
situation.
On bigtop-1.4.0 build fails on hadoop-hdfs-httpfs with error:
Failed to execute goal
org.apache.maven.plugins:maven-project-info-reports-plugin:2.7:dependencies
(default) on project hadoop-hdfs-httpfs: An error has occurred in
Dependencies report generation.: zip file is empty -> [Help 1]
Launched in clean bigtop-1.4.0 dir, with command docker run -v `pwd`:/ws
bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws;gradle hadoop-pkg
I made a --scan for this attempt - https://gradle.com/s/7iu4z3fsbrh62
Maybe I'm using not a valid way to get it perform correctly?
Or this can be bug and I need to fill a jira ticket for this?
Also I tryed to use direct building by ./gradlew hadoop-rpm and via docker
./gradlew hadoop-pkg-ind, got the same error. If I use bigtop-trunk, this
stage (hadoop-hdfs-httpfs) passed without error, but as I've been told,
currently hadoop is unbuildable in master branch.
Plus, the same error I get on bigtop-1.3.0.
Cleaning the local maven repository does not fix it.
So, likely, the problem is in building environment. How I can configure it
correctly for successful hadoop building in bigtop?
On Wed, Jan 22, 2020, 16:20 Evans Ye <ev...@apache.org> wrote:
> Currently building Hadoop across all platform on master branch is failing:
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>
> I'm not sure whether the master code can be built successfully since we're
> in transition of upgrading zookeeper.
>
> If you just want to use bigtop, please get our latest release 1.4.0.
>
> If you are willing to contribute, you can report a bug on our jira first
> and we ca work on a fix together.
>
> Let us know if any question.
>
> Evan's
>
>
>
> Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
>
> > Hi Alexander,
> >
> > It seems bigtop_toolchain didn't deploy correctly.
> > Please try to build bigtop components in docker with bigtop/slaves docker
> > images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos>.
> >
> > Run:
> > cd BIGTOP_SRC_DIR
> > docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws ;
> > gradle hadoop-pkg'
> >
> > BR,
> > Yuqi
> >
> > On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <ia...@gmail.com>
> > wrote:
> >
> > > I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
> > > error.
> > >
> > > Also, in webtop-trunk this particular error appeard while building
> hadoop
> > > in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
> > without
> > > it (by ./gradlew hadoop-rpm).
> > >
> > > Performed steps:
> > >
> > > I was trying to follow the available guides, do a clean centos 7
> install,
> > > upgrade to latest official rpms, install openjdk 7 and unzip from repo.
> > >
> > > Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> > > --modulepath=`pwd`:/etc/puppet/modules -e "include
> > > bigtop_toolchain::installer".
> > >
> > > After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and got
> > > this error.
> > >
> > >
> > >
> > > On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org>
> wrote:
> > >
> > > > Hi Alexander,
> > > >
> > > > Which version or branch of Bigtop do you use? and also can you
> provide
> > us
> > > > the steps you did?
> > > >
> > > > Thanks,
> > > > Youngwoo
> > > >
> > > >
> > > >
> > > > On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> > iamhokum@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Hello everyone.
> > > > >
> > > > > Looks like Bigtop mailing lists is not very active, but I still
> hope
> > > > > someone could help me.
> > > > >
> > > > > We are trying to employ BigTop to build own hadoop repository
> > > (./gradlew
> > > > > hadoop-rpm), but after all efforts still getting error on zookeeper
> > > > version
> > > > > dependency:
> > > > >
> > > > > -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> > > > >
> > > > > it wants -org.apache.zookeeper:zookeeper:3.4.6
> > > > >
> > > > > while all other conponents depends on
> > > > > -org.apache.zookeeper:zookeeper:3.4.13
> > > > >
> > > > > so we got:
> > > > >
> > > > > [ERROR] Failed to execute goal
> > > > > org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> > (depcheck)
> > > > on
> > > > > project hadoop-yarn-server-tests
> > > > >
> > > > >
> > > > >
> > > > > Please advise me some solution for this show stopper, if possible.
> > > > Thanks!
> > > > >
> > > >
> > >
> >
>
Re: Bigtop error on Centos 7
Posted by Evans Ye <ev...@apache.org>.
Currently building Hadoop across all platform on master branch is failing:
https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
I'm not sure whether the master code can be built successfully since we're
in transition of upgrading zookeeper.
If you just want to use bigtop, please get our latest release 1.4.0.
If you are willing to contribute, you can report a bug on our jira first
and we ca work on a fix together.
Let us know if any question.
Evan's
Yuqi Gu <yu...@linaro.org> 於 2020年1月22日 週三 16:31 寫道：
> Hi Alexander,
>
> It seems bigtop_toolchain didn't deploy correctly.
> Please try to build bigtop components in docker with bigtop/slaves docker
> images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos>.
>
> Run:
> cd BIGTOP_SRC_DIR
> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws ;
> gradle hadoop-pkg'
>
> BR,
> Yuqi
>
> On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <ia...@gmail.com>
> wrote:
>
> > I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
> > error.
> >
> > Also, in webtop-trunk this particular error appeard while building hadoop
> > in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and
> without
> > it (by ./gradlew hadoop-rpm).
> >
> > Performed steps:
> >
> > I was trying to follow the available guides, do a clean centos 7 install,
> > upgrade to latest official rpms, install openjdk 7 and unzip from repo.
> >
> > Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> > --modulepath=`pwd`:/etc/puppet/modules -e "include
> > bigtop_toolchain::installer".
> >
> > After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and got
> > this error.
> >
> >
> >
> > On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org> wrote:
> >
> > > Hi Alexander,
> > >
> > > Which version or branch of Bigtop do you use? and also can you provide
> us
> > > the steps you did?
> > >
> > > Thanks,
> > > Youngwoo
> > >
> > >
> > >
> > > On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <
> iamhokum@gmail.com
> > >
> > > wrote:
> > >
> > > > Hello everyone.
> > > >
> > > > Looks like Bigtop mailing lists is not very active, but I still hope
> > > > someone could help me.
> > > >
> > > > We are trying to employ BigTop to build own hadoop repository
> > (./gradlew
> > > > hadoop-rpm), but after all efforts still getting error on zookeeper
> > > version
> > > > dependency:
> > > >
> > > > -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> > > >
> > > > it wants -org.apache.zookeeper:zookeeper:3.4.6
> > > >
> > > > while all other conponents depends on
> > > > -org.apache.zookeeper:zookeeper:3.4.13
> > > >
> > > > so we got:
> > > >
> > > > [ERROR] Failed to execute goal
> > > > org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
> (depcheck)
> > > on
> > > > project hadoop-yarn-server-tests
> > > >
> > > >
> > > >
> > > > Please advise me some solution for this show stopper, if possible.
> > > Thanks!
> > > >
> > >
> >
>
Re: Bigtop error on Centos 7
Posted by Yuqi Gu <yu...@linaro.org>.
Hi Alexander,
It seems bigtop_toolchain didn't deploy correctly.
Please try to build bigtop components in docker with bigtop/slaves docker
images <https://hub.docker.com/r/bigtop/slaves/tags?page=1&name=centos>.
Run:
cd BIGTOP_SRC_DIR
docker run -v `pwd`:/ws bigtop/slaves:1.4.0-centos-7 bash -l -c 'cd /ws ;
gradle hadoop-pkg'
BR,
Yuqi
On Wed, 22 Jan 2020 at 15:05, Alexander Frolushkin <ia...@gmail.com>
wrote:
> I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
> error.
>
> Also, in webtop-trunk this particular error appeard while building hadoop
> in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and without
> it (by ./gradlew hadoop-rpm).
>
> Performed steps:
>
> I was trying to follow the available guides, do a clean centos 7 install,
> upgrade to latest official rpms, install openjdk 7 and unzip from repo.
>
> Then puppetize.sh from bigtop_toolchain/bin and puppet apply
> --modulepath=`pwd`:/etc/puppet/modules -e "include
> bigtop_toolchain::installer".
>
> After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and got
> this error.
>
>
>
> On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org> wrote:
>
> > Hi Alexander,
> >
> > Which version or branch of Bigtop do you use? and also can you provide us
> > the steps you did?
> >
> > Thanks,
> > Youngwoo
> >
> >
> >
> > On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <iamhokum@gmail.com
> >
> > wrote:
> >
> > > Hello everyone.
> > >
> > > Looks like Bigtop mailing lists is not very active, but I still hope
> > > someone could help me.
> > >
> > > We are trying to employ BigTop to build own hadoop repository
> (./gradlew
> > > hadoop-rpm), but after all efforts still getting error on zookeeper
> > version
> > > dependency:
> > >
> > > -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> > >
> > > it wants -org.apache.zookeeper:zookeeper:3.4.6
> > >
> > > while all other conponents depends on
> > > -org.apache.zookeeper:zookeeper:3.4.13
> > >
> > > so we got:
> > >
> > > [ERROR] Failed to execute goal
> > > org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (depcheck)
> > on
> > > project hadoop-yarn-server-tests
> > >
> > >
> > >
> > > Please advise me some solution for this show stopper, if possible.
> > Thanks!
> > >
> >
>
Re: Bigtop error on Centos 7
Posted by Alexander Frolushkin <ia...@gmail.com>.
I have this error on bigtop-trunk. In webtop-1.4.0 it fails with other
error.
Also, in webtop-trunk this particular error appeard while building hadoop
in docker (trunk-centos-7) image (by ./gradlew hadoop-pkg-ind) and without
it (by ./gradlew hadoop-rpm).
Performed steps:
I was trying to follow the available guides, do a clean centos 7 install,
upgrade to latest official rpms, install openjdk 7 and unzip from repo.
Then puppetize.sh from bigtop_toolchain/bin and puppet apply
--modulepath=`pwd`:/etc/puppet/modules -e "include
bigtop_toolchain::installer".
After this run ./gradlew hadoop-pkg-ind or ./gradlew hadoop-rpm and got
this error.
On Wed, Jan 22, 2020, 12:56 Youngwoo Kim (김영우) <yw...@apache.org> wrote:
> Hi Alexander,
>
> Which version or branch of Bigtop do you use? and also can you provide us
> the steps you did?
>
> Thanks,
> Youngwoo
>
>
>
> On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <ia...@gmail.com>
> wrote:
>
> > Hello everyone.
> >
> > Looks like Bigtop mailing lists is not very active, but I still hope
> > someone could help me.
> >
> > We are trying to employ BigTop to build own hadoop repository (./gradlew
> > hadoop-rpm), but after all efforts still getting error on zookeeper
> version
> > dependency:
> >
> > -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
> >
> > it wants -org.apache.zookeeper:zookeeper:3.4.6
> >
> > while all other conponents depends on
> > -org.apache.zookeeper:zookeeper:3.4.13
> >
> > so we got:
> >
> > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (depcheck)
> on
> > project hadoop-yarn-server-tests
> >
> >
> >
> > Please advise me some solution for this show stopper, if possible.
> Thanks!
> >
>
Re: Bigtop error on Centos 7
Posted by "Youngwoo Kim (김영우)" <yw...@apache.org>.
Hi Alexander,
Which version or branch of Bigtop do you use? and also can you provide us
the steps you did?
Thanks,
Youngwoo
On Wed, Jan 22, 2020 at 2:00 PM Alexander Frolushkin <ia...@gmail.com>
wrote:
> Hello everyone.
>
> Looks like Bigtop mailing lists is not very active, but I still hope
> someone could help me.
>
> We are trying to employ BigTop to build own hadoop repository (./gradlew
> hadoop-rpm), but after all efforts still getting error on zookeeper version
> dependency:
>
> -org.apache.hadoop:hadoop-yarn-server-resourcemanager:2.8.5
>
> it wants -org.apache.zookeeper:zookeeper:3.4.6
>
> while all other conponents depends on
> -org.apache.zookeeper:zookeeper:3.4.13
>
> so we got:
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (depcheck) on
> project hadoop-yarn-server-tests
>
>
>
> Please advise me some solution for this show stopper, if possible. Thanks!
>