You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2026/09/17 04:08:22 UTC
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24869
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
This changelist introduces an option to avoid re-creation of symbolic
links from /opt/kudu/thirdparty to the actual location of pre-built
3rd-party artifacts when building Kudu 3rd-party components.
The behavior is controlled by the newly introduced env/shell variable
PREBUILT_THIRDPARTY_RECREATE_SYMLINKS, which is set to 1 by default.
In multi-tenant environments with multiple concurrent actors each using
its own Kudu workspace but exactly same versions of 3rd-party
components, setting PREBUILT_THIRDPARTY_RECREATE_SYMLINKS=0 helps to
avoid races that often lead to build failures of earlier started builds.
Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
---
M thirdparty/build-thirdparty.sh
M thirdparty/prebuilt-utils.sh
2 files changed, 31 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/69/24869/1
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Ashwani Raina (Code Review)" <ge...@cloudera.org>.
Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG@17
PS1, Line 17: setting PREBUILT_THIRDPARTY_RECREATE_SYMLINKS=0 helps to
: avoid races that often lead to build failures of earlier started builds.
Does this take care of the race between builds that resulted in reference to absent libraries like libprotoc?
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Sep 2026 06:02:15 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG@14
PS1, Line 14: which is set to 1 by default
> Ah, thanks for the explanation. I think this approach is low overhead. Let
Alternatives aside, what's about this particular changelist?
I see many pre-commit builds were failing recently, and this update should help to address the issue.
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 13:01:54 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG@14
PS1, Line 14: which is set to 1 by default
> Should this be 0 by default? As setting it to 1 still always runs ln -nsf "
The idea is to always reset the links by default -- it helps to avoid stale links for the use case the pre-built 3rd-party approach is designed for.
Case of multiple concurrent actors with different workspaces is something special, and it cannot work flawlessly by design when it's required to have the same pre-compiled path in 3rd-party artifacts.
I'm open to ideas of how to make pre-built 3rd-party artifacts work in multi-tenant environments, while keeping all the pkgconfig and compiled-in paths consistent for all the tenants.
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Sep 2026 19:54:39 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/2985/
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 17 Sep 2026 04:08:32 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 2: -Verified
Build Started http://jenkins.kudu.apache.org/job/pre_commit/3008/
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 18:04:24 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/2985/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Sep 2026 04:51:19 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Ashwani Raina (Code Review)" <ge...@cloudera.org>.
Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh
File thirdparty/prebuilt-utils.sh:
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh@267
PS1, Line 267: ln -nsf "$target_path" "$subdir"
ln -nsf is not atomic. It does 'unlink' and 'symlink' under the hood, introducing a window where link doesn't exist.
Consider using 'ln to temp path' and 'mv temp to final path'.
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Sep 2026 06:22:48 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 2:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/3006/
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 14:24:59 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 2: Code-Review+2
> I see many pre-commit builds were failing recently, and this update should help to address the issue.
Let's get this merged, and probably if there are any alternatives I'll post a follow up patch. Thanks
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 21:32:57 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/build-thirdparty.sh
File thirdparty/build-thirdparty.sh:
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/build-thirdparty.sh@55
PS1, Line 55: enviroments
> typo: environments
Done
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh
File thirdparty/prebuilt-utils.sh:
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh@258
PS1, Line 258: if [ -L "$subdir" ] && stat -L "$subdir" > /dev/null 2>&1; then
> nit: should we have also `[ -d "$subdir" ] ` in the if statement?
Why? We are verifying that the link exists ('test -L "$subdir"') and points to an existing directory (stat -L "$subdir"). That's all what we need.
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh@265
PS1, Line 265: echo "Using existing symlink to $target_path"
> nit: What do you think about adding more detail to this log statement like:
Done
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 14:24:32 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 2: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/3006/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 14:46:15 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG@14
PS1, Line 14: which is set to 1 by default
> The idea is to always reset the links by default -- it helps to avoid stale
Ah, thanks for the explanation. I think this approach is low overhead. Let me ponder over a bit and write back if there are any other alternatives to this issue.
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Sat, 19 Sep 2026 19:15:20 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 2: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/3008/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 19:00:42 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1: Code-Review+1
(3 comments)
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/build-thirdparty.sh
File thirdparty/build-thirdparty.sh:
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/build-thirdparty.sh@55
PS1, Line 55: enviroments
typo: environments
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh
File thirdparty/prebuilt-utils.sh:
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh@258
PS1, Line 258: if [ -L "$subdir" ] && stat -L "$subdir" > /dev/null 2>&1; then
nit: should we have also `[ -d "$subdir" ] ` in the if statement?
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh@265
PS1, Line 265: echo "Using existing symlink to $target_path"
nit: What do you think about adding more detail to this log statement like:
echo "Using existing symlink $subdir -> $(readlink "$subdir")"
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 13:46:16 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1: Verified+1
unrelated test failures:
* RELEASE: webserver-crawl-itest
* ASAN: client_symbol-test
* TSAN: known flaky tests org.apache.kudu.spark.kudu.DefaultSourceTest and DisableWriteWhenExceedingQuotaTest.TestDisableWritePrivilegeWhenExceedingSizeQuota
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Sep 2026 18:36:48 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Martonka, Ashwani Raina, Kudu Jenkins, Abhishek Chennaka,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24869
to look at the new patch set (#2).
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
This changelist introduces an option to avoid re-creation of symbolic
links from /opt/kudu/thirdparty to the actual location of pre-built
3rd-party artifacts when building Kudu 3rd-party components.
The behavior is controlled by the newly introduced env/shell variable
PREBUILT_THIRDPARTY_RECREATE_SYMLINKS, which is set to 1 by default.
In multi-tenant environments with multiple concurrent actors each using
its own Kudu workspace but exactly same versions of 3rd-party
components, setting PREBUILT_THIRDPARTY_RECREATE_SYMLINKS=0 helps to
avoid races that often lead to build failures of earlier started builds.
Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
---
M thirdparty/build-thirdparty.sh
M thirdparty/prebuilt-utils.sh
2 files changed, 31 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/69/24869/2
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 2: Verified+1
unrelated test failures:
* ASAN: relatevely recently added Kudu3762Test
* TSAN: org.apache.kudu.spark.kudu.DefaultSourceTest (known flaky test)
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 19:59:19 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
This changelist introduces an option to avoid re-creation of symbolic
links from /opt/kudu/thirdparty to the actual location of pre-built
3rd-party artifacts when building Kudu 3rd-party components.
The behavior is controlled by the newly introduced env/shell variable
PREBUILT_THIRDPARTY_RECREATE_SYMLINKS, which is set to 1 by default.
In multi-tenant environments with multiple concurrent actors each using
its own Kudu workspace but exactly same versions of 3rd-party
components, setting PREBUILT_THIRDPARTY_RECREATE_SYMLINKS=0 helps to
avoid races that often lead to build failures of earlier started builds.
Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Reviewed-on: http://gerrit.cloudera.org:8080/24869
Tested-by: Alexey Serbin <al...@apache.org>
Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
---
M thirdparty/build-thirdparty.sh
M thirdparty/prebuilt-utils.sh
2 files changed, 31 insertions(+), 10 deletions(-)
Approvals:
Alexey Serbin: Verified
Abhishek Chennaka: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG@17
PS1, Line 17: setting PREBUILT_THIRDPARTY_RECREATE_SYMLINKS=0 helps to
: avoid races that often lead to build failures of earlier started builds.
> Does this take care of the race between builds that resulted in reference t
Yes: the situations we saw recently, like the absence of the protoc and protobuf libraries, were caused by the race described below.
This update takes care of situations when a later started build in workspaceB (that contains almost nothing of 3rd-party installed yet) re-establishes symbolic links in /opt/kudu/thirdparty, pointing away from workspaceA (that contains all or almost all 3rd-party components installed already) to workspaceB. Once that's done, the workspaceA build would fail at a later stage (e.g., when building Kudu bits themselves) because it couldn't find libraries or header files that are already present in workspaceA but aren't yet present in workspaceB.
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Sep 2026 18:25:26 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh
File thirdparty/prebuilt-utils.sh:
http://gerrit.cloudera.org:8080/#/c/24869/1/thirdparty/prebuilt-utils.sh@267
PS1, Line 267: ln -nsf "$target_path" "$subdir"
> ln -nsf is not atomic. It does 'unlink' and 'symlink' under the hood, intro
That wouldn't help taking into account the semantics of what's going on here.
Also, atomicity isn't a goal here.
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Sep 2026 14:14:42 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24869/1//COMMIT_MSG@14
PS1, Line 14: which is set to 1 by default
Should this be 0 by default? As setting it to 1 still always runs ln -nsf "$target_path" "$subdir" IIUC
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Sep 2026 18:56:53 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24869 )
Change subject: KUDU-3788 an option to keep existing symlinks in /opt/kudu/thirdparty
......................................................................
Patch Set 2:
> > I see many pre-commit builds were failing recently, and this
> update should help to address the issue.
> Let's get this merged, and probably if there are any alternatives
> I'll post a follow up patch. Thanks
Thanks a lot for the review!
Indeed, there are some recent failures there: this changelist is supposed to address them.
--
To view, visit http://gerrit.cloudera.org:8080/24869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f612f343a3f5ea3cca48836ebc76a167af59e7
Gerrit-Change-Number: 24869
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 21 Sep 2026 21:37:44 +0000
Gerrit-HasComments: No