You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2021/01/21 18:33:21 UTC
[kudu-CR] [build] Fix occasional Gradle build failures
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16968
Change subject: [build] Fix occasional Gradle build failures
......................................................................
[build] Fix occasional Gradle build failures
Occasionally, on a clean build environment often as a part of the
CI pipline, the C++ build will fail with cryptic often Protobuf
related failures. It turns out this is a result of multiple
./gradlew calls which depend on the kudu-client and therefore
kudu-proto module. This change fixes the issue by defining a
single CMake target to build the kudu-client.
Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
---
M CMakeLists.txt
M src/kudu/hms/CMakeLists.txt
M src/kudu/subprocess/CMakeLists.txt
3 files changed, 13 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/68/16968/1
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
[kudu-CR] [build] Fix occasional Gradle build failures
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16968 )
Change subject: [build] Fix occasional Gradle build failures
......................................................................
[build] Fix occasional Gradle build failures
Occasionally, on a clean build environment often as a part of the
CI pipline, the C++ build will fail with cryptic often Protobuf
related failures. It turns out this is a result of multiple
./gradlew calls which depend on the kudu-proto module.
This change fixes the issue by defining a single CMake target
to build the kudu-proto jar.
Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Reviewed-on: http://gerrit.cloudera.org:8080/16968
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
M CMakeLists.txt
M src/kudu/hms/CMakeLists.txt
M src/kudu/subprocess/CMakeLists.txt
3 files changed, 13 insertions(+), 2 deletions(-)
Approvals:
Kudu Jenkins: Verified
Alexey Serbin: Looks good to me, approved
Andrew Wong: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [build] Fix occasional Gradle build failures
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16968 )
Change subject: [build] Fix occasional Gradle build failures
......................................................................
Patch Set 1:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/16968/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16968/1//COMMIT_MSG@10
PS1, Line 10: cryptic often Protobuf
> nit: maybe paste snippet of an exception?
I have seen a few different exceptions occur depending on the race/timing. Given this failure is the result of a recent change causing the potential for the race, I don't think it will be all that helpful of a discovery tool to note the various exceptions/failures we have seen so far.
https://gerrit.cloudera.org/#/c/16953/
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt@123
PS1, Line 123: kudu-subprocess
> I'm a bit surprised that kudu-subprocess uses kudu-client mode. Is that so
Yeah, I suppose kudu-proto is more accurate. I will fix it.
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt@123
PS1, Line 123: kudu-hms
> nit: kudu-hive ?
Done
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt@127
PS1, Line 127: add_custom_command(OUTPUT ${CLIENT_JAR}
: COMMAND ./gradlew :kudu-client:jar ${GRADLE_FLAGS}
: WORKING_DIRECTORY "${JAVA_DIR}"
: DEPENDS init_gradle)
> Just making sure I understand this correctly. The race is that kudu-subproc
Yes your description is correct. For some reason I thought both modules used the client when initially building the patch. I adjusted it to use kudu-proto instead.
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 25 Jan 2021 20:51:17 +0000
Gerrit-HasComments: Yes
[kudu-CR] [build] Fix occasional Gradle build failures
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16968 )
Change subject: [build] Fix occasional Gradle build failures
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt@123
PS1, Line 123: kudu-subprocess
I'm a bit surprised that kudu-subprocess uses kudu-client mode. Is that some sort of dependency not tracked in $KUDU_ROOT/java/kudu-subprocess/build.gradle, but somewhere else?
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt@123
PS1, Line 123: kudu-hms
nit: kudu-hive ?
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 25 Jan 2021 20:19:01 +0000
Gerrit-HasComments: Yes
[kudu-CR] [build] Fix occasional Gradle build failures
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16968 )
Change subject: [build] Fix occasional Gradle build failures
......................................................................
Patch Set 2: Code-Review+2
Thank you for fixing the issue!
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 25 Jan 2021 21:45:41 +0000
Gerrit-HasComments: No
[kudu-CR] [build] Fix occasional Gradle build failures
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16968 )
Change subject: [build] Fix occasional Gradle build failures
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16968/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16968/1//COMMIT_MSG@10
PS1, Line 10: cryptic often Protobuf
nit: maybe paste snippet of an exception?
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt@127
PS1, Line 127: add_custom_command(OUTPUT ${CLIENT_JAR}
: COMMAND ./gradlew :kudu-client:jar ${GRADLE_FLAGS}
: WORKING_DIRECTORY "${JAVA_DIR}"
: DEPENDS init_gradle)
Just making sure I understand this correctly. The race is that kudu-subprocess and kudu-hms try to build kudu-proto, and in the off-chance these happen simultaneously, we hit an error. So the solution is to build kudu-client (and all its innards, including kudu-proto) before running either, ensuring the subprocess and hms builds don't collide. Is that right? Would it also have worked to have built just the protobuf module, rather than the entire client?
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 25 Jan 2021 19:50:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] [build] Fix occasional Gradle build failures
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16968 )
Change subject: [build] Fix occasional Gradle build failures
......................................................................
Patch Set 2: Code-Review+2
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16968/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16968/1//COMMIT_MSG@10
PS1, Line 10: cryptic often Protobuf
> I have seen a few different exceptions occur depending on the race/timing.
Ack
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/16968/1/CMakeLists.txt@127
PS1, Line 127: add_custom_command(OUTPUT ${PROTO_JAR}
: COMMAND ./gradlew :kudu-proto:jar ${GRADLE_FLAGS}
: WORKING_DIRECTORY "${JAVA_DIR}"
: DEPENDS init_gradle)
> Yes your description is correct. For some reason I thought both modules use
Thanks for confirming!
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 25 Jan 2021 21:53:30 +0000
Gerrit-HasComments: Yes
[kudu-CR] [build] Fix occasional Gradle build failures
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16968
to look at the new patch set (#2).
Change subject: [build] Fix occasional Gradle build failures
......................................................................
[build] Fix occasional Gradle build failures
Occasionally, on a clean build environment often as a part of the
CI pipline, the C++ build will fail with cryptic often Protobuf
related failures. It turns out this is a result of multiple
./gradlew calls which depend on the kudu-proto module.
This change fixes the issue by defining a single CMake target
to build the kudu-proto jar.
Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
---
M CMakeLists.txt
M src/kudu/hms/CMakeLists.txt
M src/kudu/subprocess/CMakeLists.txt
3 files changed, 13 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/68/16968/2
--
To view, visit http://gerrit.cloudera.org:8080/16968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefa02e52a74f8a60c3029188995574ffdb0d3d5c
Gerrit-Change-Number: 16968
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)