You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org> on 2019/10/04 13:03:00 UTC

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec 

Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14372


Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................

IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Building impala-minimal-hive-exec was flaky (sometimes it failed
to download Guava 14.0.1), and my guess is that it is related to
downloading the same dependency in parallel with ext-data-source.

Similarly to the solution in IMPALA-7051, I try to fix it by
adding a fictional dependency to serialize the builds.

Testing:
- I never reproduced the original flakiness, so I just hope
  that it disappears with this change.

Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
---
M ext-data-source/CMakeLists.txt
1 file changed, 3 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/14372/1
-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 09 Oct 2019 15:58:03 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14372

to look at the new patch set (#3).

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................

IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Building impala-minimal-hive-exec was flaky (sometimes it failed
to download Guava 14.0.1), and my guess is that it is related to
downloading the same dependency in parallel with ext-data-source.

Similarly to the solution in IMPALA-7051, I try to fix it by
adding a fictional dependency (ext-data-source -> shaded-deps)
to serialize the build.

Full Maven dependency graph:
fe -> yarn-extras -> ext-data-source -> shaded-deps -> impala-parent
The ordering of yarn-extras, ext-data-source, and shaded-deps are
arbitrary.

Testing:
- I never reproduced the original flakiness, so I just hope
  that it disappears with this change.

Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
---
M ext-data-source/CMakeLists.txt
1 file changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/14372/3
-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 1: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5052/


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Oct 2019 17:29:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 3: Code-Review+2

Thanks!


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2019 21:39:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 07 Oct 2019 15:07:32 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5059/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 07 Oct 2019 10:51:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 2: Code-Review+1

(1 comment)

I think this makes sense (and I think you are right that this is a similar issue to IMPALA-7051). Thanks for taking this on. One small nit, then I'll bump to +2.

http://gerrit.cloudera.org:8080/#/c/14372/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14372/2//COMMIT_MSG@13
PS2, Line 13: Similarly to the solution in IMPALA-7051, I try to fix it by
            : adding a fictional dependency to serialize the builds.
Nit: One small thing. Can you add one line noting the order of dependencies like we had in the IMPALA-7051 comment. i.e.
fe -> yarn-extras -> ext-data-source -> shaded-deps -> impala-parent
And note that the yarn-extras, ext-data-source, shaded-deps are arbitrary.



-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2019 20:52:44 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5078/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 09 Oct 2019 11:41:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5052/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Oct 2019 13:08:14 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4728/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 04 Oct 2019 13:45:48 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................

IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Building impala-minimal-hive-exec was flaky (sometimes it failed
to download Guava 14.0.1), and my guess is that it is related to
downloading the same dependency in parallel with ext-data-source.

Similarly to the solution in IMPALA-7051, I try to fix it by
adding a fictional dependency (ext-data-source -> shaded-deps)
to serialize the build.

Full Maven dependency graph:
fe -> yarn-extras -> ext-data-source -> shaded-deps -> impala-parent
The ordering of yarn-extras, ext-data-source, and shaded-deps are
arbitrary.

Testing:
- I never reproduced the original flakiness, so I just hope
  that it disappears with this change.

Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Reviewed-on: http://gerrit.cloudera.org:8080/14372
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M ext-data-source/CMakeLists.txt
1 file changed, 3 insertions(+), 1 deletion(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14372 )

Change subject: IMPALA-9008: Serialize Maven invocations to deflake impala-minimal-hive-exec
......................................................................


Patch Set 4: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/14372
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic951ad7313e5ad9add2a00e6a442c6e97f235967
Gerrit-Change-Number: 14372
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 09 Oct 2019 11:41:09 +0000
Gerrit-HasComments: No