You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Steve Loughran <st...@hortonworks.com> on 2018/07/05 11:45:18 UTC
Why are some Jenkins builds failing in Yarn-UI
Hi
The HADOOP-15407 "abfs" branch has started failing
https://builds.apache.org/job/PreCommit-HADOOP-Build/14861/artifact/out/patch-compile-root.txt
[INFO] --- frontend-maven-plugin:1.5:bower (bower install) @ hadoop-yarn-ui ---
[INFO] Running 'bower install' in /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/target/webapp
[ERROR] bower ember-cli-test-loader#0.2.1 EINVRES Request to https://bower.herokuapp.com/packages/ember-cli-test-loader failed with 502
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
And the namedc web page returns : This Bower version is deprecated. Please update it: npm install -g bower. The new registry address is https://registry.bower.io
We haven't gone near the YARN code, and yet a branch which is only a few weeks old is now failing to build on Jenkins systems
I have a few questions for the Yarn team
1. What is Bower?
2. Why is it breaking Jenkins builds
3. Can you nominate someone to provide the patch to fix this?
4. Will every active branch need a similar patch?
5. Have any releases stopped building (3.1.0?)
6. Do we have any stability guarantees about Bower's build process in future?
Given a fork I'm worried about the long-term reproducibility of builds. Is this just a one off move of some build artifact repository, or is this a moving target which will stop source code releases from working. I can deal with cherry picking patches to WiP branches, tuning Jenkins, etc, but if we lose the ability to rebuild releases, the whole notion of "stable release" is dead. I know Maven exposes us to similar risks, but the maven central repo is stabile and available, so hasn't forced us into building an SCM-msnaged artifact tree the way I've done elsewhere (Ivy makes that straightforward; maven less so as you need to be online to boot). And we are now relying on docker for release builds. So we are already vulnerable...I'm just worried that Bower is making things worse.
-Steve
(ps: Sides from Olaf Febbe on the bigtop team on attacking builds through mavan https://oflebbe.de/presentations/2018/attackingiotdev.pdf)
Re: Why are some Jenkins builds failing in Yarn-UI
Posted by Steve Loughran <st...@hortonworks.com>.
thanks.
This is "unfortunate". I know nobody wanted this to happen, but it is still pretty serious.
For the specific HADOOP-15407 branch I'd just rebase onto trunk and force-push it out, but apache git seems to be blocking me from doing that...
I will keep an eye on YARN-8387; unrepeatable builds are the enemy of us all.
-steve
On 5 Jul 2018, at 19:11, Sunil G <su...@apache.org>> wrote:
Hi Steve
I was recently looking into this failure and this is due to a repo hosting
problem. It seems Bower team has deprecated old registry url
https://bower.herokuapp.com<https://bower.herokuapp.com/> and need to change registry url to
https://registry.bower.io<https://registry.bower.io/>. YARN-8457 addressed this problem is backported
till branch-2.9.
I think the branch also needs a rebase to run away from this problem.
1. What is Bower?
YARN ui2 is a single page web application and using ember js framework.
ember-2 is using bower as package manager. Bibin is doing some work in
YARN-8387 to move away from bower to resolve such issue in future.
2. Why is it breaking Jenkins builds
Bower team has deprecated old registry url https://bower.herokuapp.com<https://bower.herokuapp.com/> and
need to change registry url to https://registry.bower.io<https://registry.bower.io/>.
3. Can you nominate someone to provide the patch to fix this?
YARN-8457 addressed this problem. Cud u pls backport this patch alone to
the branch and help to verify.
4. Will every active branch need a similar patch?
Yes. This is a bit unfortunate. If these branches can be rebased to top of
respective main branches or backport this patch alone, we could avoid this
build error.
5. Have any releases stopped building (3.1.0?)
Since this patch is already in 3.1.0, this is unblocked.
6. Do we have any stability guarantees about Bower's build process in
future?
YARN-8387 will be helping on this. Bibin Chundatt has some offline patches
which he still working on with the help of other communities to resolve
bower dependencies and upgrade to ember 3 which will help in offline
compilation.
- Sunil
On Thu, Jul 5, 2018 at 4:45 AM Steve Loughran <st...@hortonworks.com>>
wrote:
Hi
The HADOOP-15407 "abfs" branch has started failing
https://builds.apache.org/job/PreCommit-HADOOP-Build/14861/artifact/out/patch-compile-root.txt
[INFO] --- frontend-maven-plugin:1.5:bower (bower install) @
hadoop-yarn-ui ---
[INFO] Running 'bower install' in
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/target/webapp
[ERROR] bower ember-cli-test-loader#0.2.1 EINVRES Request to
https://bower.herokuapp.com/packages/ember-cli-test-loader failed with 502
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
And the namedc web page returns : This Bower version is deprecated. Please
update it: npm install -g bower. The new registry address is
https://registry.bower.io
We haven't gone near the YARN code, and yet a branch which is only a few
weeks old is now failing to build on Jenkins systems
I have a few questions for the Yarn team
1. What is Bower?
2. Why is it breaking Jenkins builds
3. Can you nominate someone to provide the patch to fix this?
4. Will every active branch need a similar patch?
5. Have any releases stopped building (3.1.0?)
6. Do we have any stability guarantees about Bower's build process in
future?
Given a fork I'm worried about the long-term reproducibility of builds. Is
this just a one off move of some build artifact repository, or is this a
moving target which will stop source code releases from working. I can deal
with cherry picking patches to WiP branches, tuning Jenkins, etc, but if we
lose the ability to rebuild releases, the whole notion of "stable release"
is dead. I know Maven exposes us to similar risks, but the maven central
repo is stabile and available, so hasn't forced us into building an
SCM-msnaged artifact tree the way I've done elsewhere (Ivy makes that
straightforward; maven less so as you need to be online to boot). And we
are now relying on docker for release builds. So we are already
vulnerable...I'm just worried that Bower is making things worse.
-Steve
(ps: Sides from Olaf Febbe on the bigtop team on attacking builds through
mavan https://oflebbe.de/presentations/2018/attackingiotdev.pdf)
Re: Why are some Jenkins builds failing in Yarn-UI
Posted by Steve Loughran <st...@hortonworks.com>.
thanks.
This is "unfortunate". I know nobody wanted this to happen, but it is still pretty serious.
For the specific HADOOP-15407 branch I'd just rebase onto trunk and force-push it out, but apache git seems to be blocking me from doing that...
I will keep an eye on YARN-8387; unrepeatable builds are the enemy of us all.
-steve
On 5 Jul 2018, at 19:11, Sunil G <su...@apache.org>> wrote:
Hi Steve
I was recently looking into this failure and this is due to a repo hosting
problem. It seems Bower team has deprecated old registry url
https://bower.herokuapp.com<https://bower.herokuapp.com/> and need to change registry url to
https://registry.bower.io<https://registry.bower.io/>. YARN-8457 addressed this problem is backported
till branch-2.9.
I think the branch also needs a rebase to run away from this problem.
1. What is Bower?
YARN ui2 is a single page web application and using ember js framework.
ember-2 is using bower as package manager. Bibin is doing some work in
YARN-8387 to move away from bower to resolve such issue in future.
2. Why is it breaking Jenkins builds
Bower team has deprecated old registry url https://bower.herokuapp.com<https://bower.herokuapp.com/> and
need to change registry url to https://registry.bower.io<https://registry.bower.io/>.
3. Can you nominate someone to provide the patch to fix this?
YARN-8457 addressed this problem. Cud u pls backport this patch alone to
the branch and help to verify.
4. Will every active branch need a similar patch?
Yes. This is a bit unfortunate. If these branches can be rebased to top of
respective main branches or backport this patch alone, we could avoid this
build error.
5. Have any releases stopped building (3.1.0?)
Since this patch is already in 3.1.0, this is unblocked.
6. Do we have any stability guarantees about Bower's build process in
future?
YARN-8387 will be helping on this. Bibin Chundatt has some offline patches
which he still working on with the help of other communities to resolve
bower dependencies and upgrade to ember 3 which will help in offline
compilation.
- Sunil
On Thu, Jul 5, 2018 at 4:45 AM Steve Loughran <st...@hortonworks.com>>
wrote:
Hi
The HADOOP-15407 "abfs" branch has started failing
https://builds.apache.org/job/PreCommit-HADOOP-Build/14861/artifact/out/patch-compile-root.txt
[INFO] --- frontend-maven-plugin:1.5:bower (bower install) @
hadoop-yarn-ui ---
[INFO] Running 'bower install' in
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/target/webapp
[ERROR] bower ember-cli-test-loader#0.2.1 EINVRES Request to
https://bower.herokuapp.com/packages/ember-cli-test-loader failed with 502
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
And the namedc web page returns : This Bower version is deprecated. Please
update it: npm install -g bower. The new registry address is
https://registry.bower.io
We haven't gone near the YARN code, and yet a branch which is only a few
weeks old is now failing to build on Jenkins systems
I have a few questions for the Yarn team
1. What is Bower?
2. Why is it breaking Jenkins builds
3. Can you nominate someone to provide the patch to fix this?
4. Will every active branch need a similar patch?
5. Have any releases stopped building (3.1.0?)
6. Do we have any stability guarantees about Bower's build process in
future?
Given a fork I'm worried about the long-term reproducibility of builds. Is
this just a one off move of some build artifact repository, or is this a
moving target which will stop source code releases from working. I can deal
with cherry picking patches to WiP branches, tuning Jenkins, etc, but if we
lose the ability to rebuild releases, the whole notion of "stable release"
is dead. I know Maven exposes us to similar risks, but the maven central
repo is stabile and available, so hasn't forced us into building an
SCM-msnaged artifact tree the way I've done elsewhere (Ivy makes that
straightforward; maven less so as you need to be online to boot). And we
are now relying on docker for release builds. So we are already
vulnerable...I'm just worried that Bower is making things worse.
-Steve
(ps: Sides from Olaf Febbe on the bigtop team on attacking builds through
mavan https://oflebbe.de/presentations/2018/attackingiotdev.pdf)
Re: Why are some Jenkins builds failing in Yarn-UI
Posted by Sunil G <su...@apache.org>.
Hi Steve
I was recently looking into this failure and this is due to a repo hosting
problem. It seems Bower team has deprecated old registry url
https://bower.herokuapp.com and need to change registry url to
https://registry.bower.io. YARN-8457 addressed this problem is backported
till branch-2.9.
I think the branch also needs a rebase to run away from this problem.
1. What is Bower?
YARN ui2 is a single page web application and using ember js framework.
ember-2 is using bower as package manager. Bibin is doing some work in
YARN-8387 to move away from bower to resolve such issue in future.
2. Why is it breaking Jenkins builds
Bower team has deprecated old registry url https://bower.herokuapp.com and
need to change registry url to https://registry.bower.io.
3. Can you nominate someone to provide the patch to fix this?
YARN-8457 addressed this problem. Cud u pls backport this patch alone to
the branch and help to verify.
4. Will every active branch need a similar patch?
Yes. This is a bit unfortunate. If these branches can be rebased to top of
respective main branches or backport this patch alone, we could avoid this
build error.
5. Have any releases stopped building (3.1.0?)
Since this patch is already in 3.1.0, this is unblocked.
6. Do we have any stability guarantees about Bower's build process in
future?
YARN-8387 will be helping on this. Bibin Chundatt has some offline patches
which he still working on with the help of other communities to resolve
bower dependencies and upgrade to ember 3 which will help in offline
compilation.
- Sunil
On Thu, Jul 5, 2018 at 4:45 AM Steve Loughran <st...@hortonworks.com>
wrote:
>
> Hi
>
> The HADOOP-15407 "abfs" branch has started failing
>
>
> https://builds.apache.org/job/PreCommit-HADOOP-Build/14861/artifact/out/patch-compile-root.txt
>
> [INFO] --- frontend-maven-plugin:1.5:bower (bower install) @
> hadoop-yarn-ui ---
> [INFO] Running 'bower install' in
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/target/webapp
> [ERROR] bower ember-cli-test-loader#0.2.1 EINVRES Request to
> https://bower.herokuapp.com/packages/ember-cli-test-loader failed with 502
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
>
> And the namedc web page returns : This Bower version is deprecated. Please
> update it: npm install -g bower. The new registry address is
> https://registry.bower.io
>
>
> We haven't gone near the YARN code, and yet a branch which is only a few
> weeks old is now failing to build on Jenkins systems
>
> I have a few questions for the Yarn team
>
>
> 1. What is Bower?
> 2. Why is it breaking Jenkins builds
> 3. Can you nominate someone to provide the patch to fix this?
> 4. Will every active branch need a similar patch?
> 5. Have any releases stopped building (3.1.0?)
> 6. Do we have any stability guarantees about Bower's build process in
> future?
>
> Given a fork I'm worried about the long-term reproducibility of builds. Is
> this just a one off move of some build artifact repository, or is this a
> moving target which will stop source code releases from working. I can deal
> with cherry picking patches to WiP branches, tuning Jenkins, etc, but if we
> lose the ability to rebuild releases, the whole notion of "stable release"
> is dead. I know Maven exposes us to similar risks, but the maven central
> repo is stabile and available, so hasn't forced us into building an
> SCM-msnaged artifact tree the way I've done elsewhere (Ivy makes that
> straightforward; maven less so as you need to be online to boot). And we
> are now relying on docker for release builds. So we are already
> vulnerable...I'm just worried that Bower is making things worse.
>
> -Steve
>
> (ps: Sides from Olaf Febbe on the bigtop team on attacking builds through
> mavan https://oflebbe.de/presentations/2018/attackingiotdev.pdf)
>
>
>
>
>
Re: Why are some Jenkins builds failing in Yarn-UI
Posted by Sunil G <su...@apache.org>.
Hi Steve
I was recently looking into this failure and this is due to a repo hosting
problem. It seems Bower team has deprecated old registry url
https://bower.herokuapp.com and need to change registry url to
https://registry.bower.io. YARN-8457 addressed this problem is backported
till branch-2.9.
I think the branch also needs a rebase to run away from this problem.
1. What is Bower?
YARN ui2 is a single page web application and using ember js framework.
ember-2 is using bower as package manager. Bibin is doing some work in
YARN-8387 to move away from bower to resolve such issue in future.
2. Why is it breaking Jenkins builds
Bower team has deprecated old registry url https://bower.herokuapp.com and
need to change registry url to https://registry.bower.io.
3. Can you nominate someone to provide the patch to fix this?
YARN-8457 addressed this problem. Cud u pls backport this patch alone to
the branch and help to verify.
4. Will every active branch need a similar patch?
Yes. This is a bit unfortunate. If these branches can be rebased to top of
respective main branches or backport this patch alone, we could avoid this
build error.
5. Have any releases stopped building (3.1.0?)
Since this patch is already in 3.1.0, this is unblocked.
6. Do we have any stability guarantees about Bower's build process in
future?
YARN-8387 will be helping on this. Bibin Chundatt has some offline patches
which he still working on with the help of other communities to resolve
bower dependencies and upgrade to ember 3 which will help in offline
compilation.
- Sunil
On Thu, Jul 5, 2018 at 4:45 AM Steve Loughran <st...@hortonworks.com>
wrote:
>
> Hi
>
> The HADOOP-15407 "abfs" branch has started failing
>
>
> https://builds.apache.org/job/PreCommit-HADOOP-Build/14861/artifact/out/patch-compile-root.txt
>
> [INFO] --- frontend-maven-plugin:1.5:bower (bower install) @
> hadoop-yarn-ui ---
> [INFO] Running 'bower install' in
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/target/webapp
> [ERROR] bower ember-cli-test-loader#0.2.1 EINVRES Request to
> https://bower.herokuapp.com/packages/ember-cli-test-loader failed with 502
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
>
> And the namedc web page returns : This Bower version is deprecated. Please
> update it: npm install -g bower. The new registry address is
> https://registry.bower.io
>
>
> We haven't gone near the YARN code, and yet a branch which is only a few
> weeks old is now failing to build on Jenkins systems
>
> I have a few questions for the Yarn team
>
>
> 1. What is Bower?
> 2. Why is it breaking Jenkins builds
> 3. Can you nominate someone to provide the patch to fix this?
> 4. Will every active branch need a similar patch?
> 5. Have any releases stopped building (3.1.0?)
> 6. Do we have any stability guarantees about Bower's build process in
> future?
>
> Given a fork I'm worried about the long-term reproducibility of builds. Is
> this just a one off move of some build artifact repository, or is this a
> moving target which will stop source code releases from working. I can deal
> with cherry picking patches to WiP branches, tuning Jenkins, etc, but if we
> lose the ability to rebuild releases, the whole notion of "stable release"
> is dead. I know Maven exposes us to similar risks, but the maven central
> repo is stabile and available, so hasn't forced us into building an
> SCM-msnaged artifact tree the way I've done elsewhere (Ivy makes that
> straightforward; maven less so as you need to be online to boot). And we
> are now relying on docker for release builds. So we are already
> vulnerable...I'm just worried that Bower is making things worse.
>
> -Steve
>
> (ps: Sides from Olaf Febbe on the bigtop team on attacking builds through
> mavan https://oflebbe.de/presentations/2018/attackingiotdev.pdf)
>
>
>
>
>