You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2021/02/04 00:13:33 UTC

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics 

Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17020


Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................

IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

When using a Maven mirror that uses a mirrorOf pattern, the order
of repositories in the pom.xml has a strong influence on whether the
build tries the mirror for a particular artifact. If an early
repository matches the mirrorOf condition, Maven may try the mirror
for all artifacts, even those that only exist in the s3 bucket.
This extra check can slow down the build, especially if the mirror
is slow to respond for unknown artifacts.

For Impala, the common case is for a mirror to cover everything
except the artifacts that come from the Kudu local repository or
the s3 bucket. To optimize for that case, this reorders the Maven
repositories to be in this order:
1. Local/S3 repositories
2. Regular repositories
3. Banned repositories
The repositories are otherwise unchanged.

Testing:
 - Ran an ordinary build
 - Ran a build with a mirrorOf "external:*,!impala.cdp.repo" and verified
   that the build went directly to the s3 bucket first.

Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
---
M java/pom.xml
1 file changed, 54 insertions(+), 37 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/17020/1
-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Apr 2021 21:38:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................


Patch Set 3:

Rebased, carrying +2


-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Apr 2021 15:50:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/8075/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 Feb 2021 00:35:32 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................

IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

When using a Maven mirror that uses a mirrorOf pattern, the order
of repositories in the pom.xml has a strong influence on whether the
build tries the mirror for a particular artifact. If an early
repository matches the mirrorOf condition, Maven may try the mirror
for all artifacts, even those that only exist in the s3 bucket.
This extra check can slow down the build, especially if the mirror
is slow to respond for unknown artifacts.

For Impala, the common case is for a mirror to cover everything
except the artifacts that come from the Kudu local repository or
the s3 bucket. To optimize for that case, this reorders the Maven
repositories to be in this order:
1. Local/S3 repositories
2. Regular repositories
3. Banned repositories
The repositories are otherwise unchanged.

Testing:
 - Ran an ordinary build
 - Ran a build with a mirrorOf "external:*,!impala.cdp.repo" and verified
   that the build went directly to the s3 bucket first.

Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Reviewed-on: http://gerrit.cloudera.org:8080/17020
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M java/pom.xml
1 file changed, 54 insertions(+), 37 deletions(-)

Approvals:
  Joe McDonnell: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7055/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Apr 2021 15:51:22 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................


Patch Set 2: Code-Review+1

Hi Joe, nice improvement. LGTM!


-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Mar 2021 14:21:14 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................


Patch Set 2: Code-Review+2

LGTM


-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Mar 2021 20:25:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17020 )

Change subject: IMPALA-10455: Reorder Maven repositories for cleaner mirror semantics
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7046c7ec5391833e98ee6a463fb8c08b6a04cb26
Gerrit-Change-Number: 17020
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Apr 2021 15:50:52 +0000
Gerrit-HasComments: No