You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "hector.acosta@cloudera.com (Code Review)" <ge...@cloudera.org> on 2019/02/09 01:06:57 UTC
[native-toolchain-CR] Enable publishing on the docker-based build
hector.acosta@cloudera.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12423
Change subject: Enable publishing on the docker-based build
......................................................................
Enable publishing on the docker-based build
In order to enable publishing from each container this commit:
* Adds maven and aws commands
* Modifies the upload function to conditionally upload to
artifactory/s3
* Generate TOOLCHAIN_ID (if not set) and share it with all the
containers. This is done in the Makefile so that all containers
get the same TOOLCHAIN_ID.
* Generate the BUILD_TARGET_LABEL based on the docker image name.
Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Signed-off-by: Hector Acosta <he...@cloudera.com>
---
M Makefile
M docker/all/assert-dependencies-present.py
M docker/debian7.df
M docker/debian8.df
M docker/redhat6.df
M docker/redhat7.df
M docker/sles12.df
M docker/ubuntu1204.df
M docker/ubuntu1404.df
M docker/ubuntu1604.df
M docker/ubuntu1804.df
M functions.sh
M in-docker.py
M init.sh
14 files changed, 168 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/23/12423/1
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 1
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "hector.acosta@cloudera.com (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Tim Armstrong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12423
to look at the new patch set (#4).
Change subject: Enable publishing on the docker-based build
......................................................................
Enable publishing on the docker-based build
In order to enable publishing from each container this commit:
* Adds maven and aws commands
* Modifies the upload function to conditionally upload to
artifactory/s3
* Generate TOOLCHAIN_ID (if not set) and share it with all the
containers. This is done in the Makefile so that all containers
get the same TOOLCHAIN_ID.
* Generate the BUILD_TARGET_LABEL based on the docker image name.
Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
---
M Makefile
M docker/all/assert-dependencies-present.py
M docker/debian7.df
M docker/debian8.df
M docker/redhat6.df
M docker/redhat7.df
M docker/sles12.df
M docker/ubuntu1204.df
M docker/ubuntu1404.df
M docker/ubuntu1604.df
M docker/ubuntu1804.df
M functions.sh
M in-docker.py
M init.sh
14 files changed, 185 insertions(+), 23 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/23/12423/4
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 4
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: hector.acosta@cloudera.com <he...@cloudera.com>
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Patch Set 4: Verified+1 Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 4
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Feb 2019 18:36:03 +0000
Gerrit-HasComments: No
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "hector.acosta@cloudera.com (Code Review)" <ge...@cloudera.org>.
hector.acosta@cloudera.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12423/2/in-docker.py
File in-docker.py:
http://gerrit.cloudera.org:8080/#/c/12423/2/in-docker.py@57
PS2, Line 57: Maps docker images to BUILD_TARGET_LABELs which is ultimately included
: # in the path for each built package
> It may be worth mentioning here that Impala/bin/bootstrap_toolchain.py cont
Done
http://gerrit.cloudera.org:8080/#/c/12423/2/in-docker.py@130
PS2, Line 130: PUBLISH_D
> Maybe add the AWS credential variables to the passthrough list, in case the
Done
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 3
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Feb 2019 18:27:20 +0000
Gerrit-HasComments: Yes
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "hector.acosta@cloudera.com (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12423
to look at the new patch set (#3).
Change subject: Enable publishing on the docker-based build
......................................................................
Enable publishing on the docker-based build
In order to enable publishing from each container this commit:
* Adds maven and aws commands
* Modifies the upload function to conditionally upload to
artifactory/s3
* Generate TOOLCHAIN_ID (if not set) and share it with all the
containers. This is done in the Makefile so that all containers
get the same TOOLCHAIN_ID.
* Generate the BUILD_TARGET_LABEL based on the docker image name.
Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
---
M Makefile
M docker/all/assert-dependencies-present.py
M docker/debian7.df
M docker/debian8.df
M docker/redhat6.df
M docker/redhat7.df
M docker/sles12.df
M docker/ubuntu1204.df
M docker/ubuntu1404.df
M docker/ubuntu1604.df
M docker/ubuntu1804.df
M functions.sh
M in-docker.py
M init.sh
14 files changed, 185 insertions(+), 23 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/23/12423/3
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 3
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Patch Set 5: Verified+1 Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 5
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Feb 2019 18:41:12 +0000
Gerrit-HasComments: No
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Enable publishing on the docker-based build
In order to enable publishing from each container this commit:
* Adds maven and aws commands
* Modifies the upload function to conditionally upload to
artifactory/s3
* Generate TOOLCHAIN_ID (if not set) and share it with all the
containers. This is done in the Makefile so that all containers
get the same TOOLCHAIN_ID.
* Generate the BUILD_TARGET_LABEL based on the docker image name.
Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Reviewed-on: http://gerrit.cloudera.org:8080/12423
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Tim Armstrong <ta...@cloudera.com>
---
M Makefile
M docker/all/assert-dependencies-present.py
M docker/debian7.df
M docker/debian8.df
M docker/redhat6.df
M docker/redhat7.df
M docker/sles12.df
M docker/ubuntu1204.df
M docker/ubuntu1404.df
M docker/ubuntu1604.df
M docker/ubuntu1804.df
M functions.sh
M in-docker.py
M init.sh
14 files changed, 185 insertions(+), 23 deletions(-)
Approvals:
Tim Armstrong: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 6
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: hector.acosta@cloudera.com <he...@cloudera.com>
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12423/2/in-docker.py
File in-docker.py:
http://gerrit.cloudera.org:8080/#/c/12423/2/in-docker.py@57
PS2, Line 57: Maps docker images to BUILD_TARGET_LABELs which is ultimately included
: # in the path for each built package
It may be worth mentioning here that Impala/bin/bootstrap_toolchain.py contains the logic that depends on these strings.
http://gerrit.cloudera.org:8080/#/c/12423/2/in-docker.py@130
PS2, Line 130: S3_BUCKET
Maybe add the AWS credential variables to the passthrough list, in case they are supplied for the bucket?
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_SESSION_TOKEN
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 2
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Feb 2019 16:58:44 +0000
Gerrit-HasComments: Yes
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Patch Set 3:
I'm going to do a trial (non-docker) build on your stack of 4 patches and verify + submit them if it succeeds.
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 3
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Comment-Date: Thu, 21 Feb 2019 21:23:57 +0000
Gerrit-HasComments: No
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Patch Set 3: Code-Review+1
(1 comment)
LGTM except for the typo :)
http://gerrit.cloudera.org:8080/#/c/12423/3/in-docker.py
File in-docker.py:
http://gerrit.cloudera.org:8080/#/c/12423/3/in-docker.py@120
PS3, Line 120: V
typo: extra 'V' character
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 3
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: hector.acosta@cloudera.com <he...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 Feb 2019 22:05:48 +0000
Gerrit-HasComments: Yes
[native-toolchain-CR] Enable publishing on the docker-based build
Posted by "hector.acosta@cloudera.com (Code Review)" <ge...@cloudera.org>.
hector.acosta@cloudera.com has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12423 )
Change subject: Enable publishing on the docker-based build
......................................................................
Enable publishing on the docker-based build
In order to enable publishing from each container this commit:
* Adds maven and aws commands
* Modifies the upload function to conditionally upload to
artifactory/s3
* Generate TOOLCHAIN_ID (if not set) and share it with all the
containers. This is done in the Makefile so that all containers
get the same TOOLCHAIN_ID.
* Generate the BUILD_TARGET_LABEL based on the docker image name.
Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
---
M Makefile
M docker/all/assert-dependencies-present.py
M docker/debian7.df
M docker/debian8.df
M docker/redhat6.df
M docker/redhat7.df
M docker/sles12.df
M docker/ubuntu1204.df
M docker/ubuntu1404.df
M docker/ubuntu1604.df
M docker/ubuntu1804.df
M functions.sh
M in-docker.py
M init.sh
14 files changed, 181 insertions(+), 23 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/23/12423/2
--
To view, visit http://gerrit.cloudera.org:8080/12423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6
Gerrit-Change-Number: 12423
Gerrit-PatchSet: 2
Gerrit-Owner: hector.acosta@cloudera.com <he...@cloudera.com>