You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2022/09/05 06:04:40 UTC
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18939
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 16.04 and CentOS 7.5 using
./buildall.sh -noclean -skiptests -ninja -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
A bin/deploy/common.sh
A bin/deploy/start-catalogd.sh
A bin/deploy/start-impalad.sh
A bin/deploy/start-statestore.sh
M buildall.sh
A fe/src/test/resources/deploy/catalogd_flags
A fe/src/test/resources/deploy/core-site.xml
A fe/src/test/resources/deploy/fair-scheduler.xml
A fe/src/test/resources/deploy/hdfs-site.xml
A fe/src/test/resources/deploy/hive-site.xml
A fe/src/test/resources/deploy/impalad_flags
A fe/src/test/resources/deploy/llama-site.xml
A fe/src/test/resources/deploy/statestore_flags
17 files changed, 278 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/1
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 4: Code-Review+1
LGTM,thanks quanlong!
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Fri, 16 Sep 2022 02:37:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Thu, 15 Sep 2022 17:59:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10:
I ran the command './buildall.sh -noclean -notests -release -package' and found the ctests were built even with '--notests' option.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Thu, 13 Jul 2023 09:26:34 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
> (1 comment)
>
> Found another issue when building RPM on Redhat 8:
>
> + PYTHON3=/usr/libexec/platform-python
> + /usr/lib/rpm/redhat/brp-mangle-shebangs
> *** ERROR: ambiguous python shebang in /opt/impala/shell/lib/option_parser.py:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/lib/impala_client.py:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/lib/TSSLSocketWithWildcardSAN.py:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/StatestoreService/StatestoreService-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/StatestoreService/StatestoreSubscriber-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/beeswaxd/BeeswaxService-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/ImpalaService/ImpalaService-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/ImpalaService/ImpalaHiveServer2Service-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/TCLIService/TCLIService-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/fb303/FacebookService-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/hive_metastore/ThriftHiveMetastore-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/CatalogService/CatalogService-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/NetworkTest/NetworkTestService-remote:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
> *** ERROR: ambiguous python shebang in /opt/impala/shell/impala_shell.py:
> #!/usr/bin/env python. Change it to python3 (or python2)
> explicitly.
Hmm, some of these files don't need shebangs (option_parser.py, impala_client.py, TSSLSocketWithWildcardSAN.py), because they don't have main functions. I have been meaning to crack down on this (and files marked as executable). I'll file a JIRA.
Maybe we can exclude the *-remote files? I don't think we use those.
That leaves impala_shell.py. I also think we might be able to rework this so it doesn't need a shebang. We have a shell script that invokes it (impala-shell). We can change the locations that call this to not execute impala_shell.py directly. I can file a JIRA to track that down.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 28 Jun 2023 20:40:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 3:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/11351/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 14 Sep 2022 03:44:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt@614
PS8, Line 614: if ($ENV{STRIP_DEPLOYMENT_IMPALAD})
: set(CPACK_STRIP_FILES ${IMPALA_INSTALLDIR}/bin/impalad)
: message("Binaries in the package will be stripped")
: endif()
One thing I tried out is using the CPACK_DEBIAN_DEBUGINFO_PACKAGE option. That would produce a package with stripped binaries along with a separate package for debuginfo. That might be nice. CPACK_STRIP_FILES is good for now.
That requires us to build things with Build IDs. Linux distributions always do that for packages. That is doable, but we would need to build a new toolchain because our libgcc_s.so and libstdc++.so don't have Build IDs right now.
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt@622
PS8, Line 622: execute_process(
: COMMAND $ENV{IMPALA_HOME}/docker/install_os_packages.sh --dry-run
: COMMAND tail -1
: OUTPUT_VARIABLE PKG_LIST
: OUTPUT_STRIP_TRAILING_WHITESPACE
: )
When I run this on Ubuntu 20, somehow this doesn't work. It works on the commandline, but CMake has PKG_LIST empty.
I messed around with this for a while, and this is a formula that seems to work:
execute_process(
COMMAND bash -c "${CMAKE_SOURCE_DIR}/docker/install_os_packages.sh --dry-run | tail -n1"
OUTPUT_VARIABLE PKG_LIST
OUTPUT_STRIP_TRAILING_WHITESPACE
)
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt@628
PS8, Line 628: message(STATUS "Get required package list: ${PKG_LIST}")
As a sanity check, can we raise an error if PKG_LIST is empty:
if ("${PKG_LIST}" STREQUAL "")
message(FATAL_ERROR "Package list is empty: '${PKG_LIST}'")
else()
message(STATUS "Get required package list: '${PKG_LIST}'")
endif()
http://gerrit.cloudera.org:8080/#/c/18939/8/package/bin/impala-env.sh
File package/bin/impala-env.sh:
http://gerrit.cloudera.org:8080/#/c/18939/8/package/bin/impala-env.sh@32
PS8, Line 32: export LIBHDFS_OPTS="${LIBHDFS_OPTS:-} -Djava.library.path=${HADOOP_LIB_DIR}/native/"
: echo "Using hadoop native libs in ${HADOOP_LIB_DIR}/native/"
: else
: echo "WARNING: HDFS short-circuit reads are not enabled due to HADOOP_HOME not set."
> nit: Could we also pack hadoop native libs into the final package?
Good point, it looks like we don't include that right now, and it is needed for short-circuit reads.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 30 Jun 2023 03:05:52 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 9:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13484/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 07 Jul 2023 09:28:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#2).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 16.04 and CentOS 7.5 using
./buildall.sh -noclean -skiptests -ninja -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
A bin/deploy/common.sh
A bin/deploy/start-catalogd.sh
A bin/deploy/start-impalad.sh
A bin/deploy/start-statestore.sh
M bin/rat_exclude_files.txt
M buildall.sh
A fe/src/test/resources/deploy/catalogd_flags
A fe/src/test/resources/deploy/core-site.xml
A fe/src/test/resources/deploy/fair-scheduler.xml
A fe/src/test/resources/deploy/hdfs-site.xml
A fe/src/test/resources/deploy/hive-site.xml
A fe/src/test/resources/deploy/impalad_flags
A fe/src/test/resources/deploy/llama-site.xml
A fe/src/test/resources/deploy/statestore_flags
18 files changed, 347 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/2
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 3:
(7 comments)
Thanks for your feedback!
http://gerrit.cloudera.org:8080/#/c/18939/2/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/2/CMakeLists.txt@27
PS2, Line 27: option(BUILD_PACKAGES "Build deployment packages")
> typo in 'installater'?
changed to "deployment".
http://gerrit.cloudera.org:8080/#/c/18939/2/CMakeLists.txt@608
PS2, Line 608:
> Is this will strip the final 'bin/impalad'? can we make it optional?
Yeah, added an env STRIP_DEPLOYMENT_IMPALAD to control this.
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/common.sh
File bin/deploy/common.sh:
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/common.sh@1
PS2, Line 1:
> Is it better to rename it to impala-env.sh? make it's function more clear,
Yeah, make sense.
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/start-catalogd.sh
File bin/deploy/start-catalogd.sh:
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/start-catalogd.sh@29
PS2, Line 29:
> maybe better to assign IMPALA_HOME to current directory using '$(cd $(dirna
I tend to make IMPALA_HOME configurable so if you want to deploy different versions of Impala, you can just switch to a new location.
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/start-catalogd.sh@31
PS2, Line 31:
> I have some suggestions here:
These are pretty good points! But they are feature requests. I think we can add them in follow-up JIRAs so other guys can also join this work.
http://gerrit.cloudera.org:8080/#/c/18939/2/fe/src/test/resources/deploy/catalogd_flags
File fe/src/test/resources/deploy/catalogd_flags:
PS2:
> files below 'fe/src/test/resources/deploy' have little relation to fe, mayb
Done
http://gerrit.cloudera.org:8080/#/c/18939/2/fe/src/test/resources/deploy/core-site.xml
File fe/src/test/resources/deploy/core-site.xml:
http://gerrit.cloudera.org:8080/#/c/18939/2/fe/src/test/resources/deploy/core-site.xml@1
PS2, Line 1:
> can we read xxx-site.xml by envirement variables first?
I'm not sure if users need different sets of configuration in practise. Let's support this in a follow-up JIRA as well.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 14 Sep 2022 03:24:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
(3 comments)
Thank Joe's comments! I found an issue on Redhat8 when installing rpm-build. It might impact downstream builds so we'd better fix it.
http://gerrit.cloudera.org:8080/#/c/18939/6/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/6/CMakeLists.txt@574
PS6, Line 574:
: SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -stdlib=libstdc++")
:
: # packaging
: if (BUILD_PACKAGES)
:
> I noticed recently that /etc/os-release is available on all the distributio
That would be much cleaner! I just inherrit these from Shant's original patch: https://gerrit.cloudera.org/c/16612/
After some search, I found that we can use cmake_host_system_information(). DISTRIB_<name> is added since cmake-3.22. So I bump the minimal required cmake version to 3.22 accordingly.
https://cmake.org/cmake/help/latest/command/cmake_host_system_information.html
http://gerrit.cloudera.org:8080/#/c/18939/6/CMakeLists.txt@613
PS6, Line 613:
: if ($ENV{STRIP_DEPLOYMENT_IMPALAD})
: set(CPACK_STRIP_FILES ${IMPALA_INSTALLDIR}/bin/impalad)
: message("Binaries in the package will be stripped")
> Ideally, this would match the docker/install_os_packages.sh logic almost ex
Good point! We have the package list in several places so it'd be nice to consolidate them. For this patch, I would just avoid adding one more place for hard-coding the package list.
Added a --dry-run option for docker/install_os_packages.sh to print the package list only. Java version is also included in the result. This CMake code then calls the script to get the list.
http://gerrit.cloudera.org:8080/#/c/18939/7/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:
http://gerrit.cloudera.org:8080/#/c/18939/7/bin/bootstrap_system.sh@246
PS7, Line 246: postgresql postgresql-server rpm-build \
This fails in Redhat8:
# yum install -y rpm-build
Last metadata expiration check: 0:05:58 ago on Mon 05 Jun 2023 04:04:45 AM PDT.
Error:
Problem: package rpm-build-4.14.2-11.el8_0.x86_64 requires (ansible-packaging if rpm-build), but none of the providers can be installed
- package rpm-build-4.14.2-26.el8_1.x86_64 requires (ansible-packaging if rpm-build), but none of the providers can be installed
- package rpm-build-4.14.2-25.el8.x86_64 requires (ansible-packaging if rpm-build), but none of the providers can be installed
- package rpm-build-4.14.2-9.el8.x86_64 requires (ansible-packaging if rpm-build), but none of the providers can be installed
- package rpm-build-4.14.2-10.el8_0.x86_64 requires (ansible-packaging if rpm-build), but none of the providers can be installed
- package rpm-build-4.14.2-37.el8.x86_64 requires (ansible-packaging if rpm-build), but none of the providers can be installed
- problem with installed package ansible-2.9.27-3.el8.noarch
- cannot install the best candidate for the job
- nothing provides python(abi) = 3.11 needed by ansible-7.2.0-1.el8.noarch
- nothing provides python3.11dist(ansible-core) >= 2.14.2 needed by ansible-7.2.0-1.el8.noarch
- nothing provides /usr/bin/python3.11 needed by ansible-7.2.0-1.el8.noarch
- nothing provides ansible-core needed by ansible-packaging-1-8.2.el8.noarch
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 05 Jun 2023 11:26:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10:
(5 comments)
Thanks for the feedbacks! Addressed the comments and improved the scripts
- Add scripts to stop daemons
- Add waiting logic in start scripts to wait until service available
Also skip packaging impala-shell on redhat8 to avoid the shebang issue.
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt@614
PS8, Line 614: # Set a meaningful package name, e.g. Impala-4.3.0-SNAPSHOT_hive-3.1.3000.7.2.18.0-41-x86_64.el7
: set(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_NAME}-${CPACK_PACKAGE_VERSION}")
: set(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_FILE_NAME}_hive-$ENV{IMPALA_HIVE_VERSION}")
: set(CPA
> One thing I tried out is using the CPACK_DEBIAN_DEBUGINFO_PACKAGE option. T
Good to know that. We can do it in future changes.
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt@622
PS8, Line 622: endif()
: message(STATUS "Package name: ${CPACK_PACKAGE_FILE_NAME}")
:
: if ($ENV{STRIP_DEPLOYMENT_IMPALAD})
: set(CPACK_STRIP_FILES ${IMPALA_INSTALLDIR}/bin/impalad)
:
> When I run this on Ubuntu 20, somehow this doesn't work. It works on the co
It's strange that I tried Ubuntu 20 on EC2 and it works. Changed it anyway since this might be more compatible.
http://gerrit.cloudera.org:8080/#/c/18939/8/CMakeLists.txt@628
PS8, Line 628: endif()
> As a sanity check, can we raise an error if PKG_LIST is empty:
Sure. Done.
http://gerrit.cloudera.org:8080/#/c/18939/8/docker/install_os_packages.sh
File docker/install_os_packages.sh:
http://gerrit.cloudera.org:8080/#/c/18939/8/docker/install_os_packages.sh@178
PS8, Line 178: woul
> Nit: maybe change to 'would' ?
Done
http://gerrit.cloudera.org:8080/#/c/18939/8/package/bin/impala-env.sh
File package/bin/impala-env.sh:
http://gerrit.cloudera.org:8080/#/c/18939/8/package/bin/impala-env.sh@32
PS8, Line 32:
: #TODO: Add graceful shutdown for impalads
: function stop_process {
: name=$1
> Good point, it looks like we don't include that right now, and it is needed
I thought the native libs should exist if DataNode is deployed on the node. But it's harmless to include them in the package. Done.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 07 Jul 2023 09:09:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11286/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 05 Sep 2022 07:10:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
(1 comment)
Found another issue when building RPM on Redhat 8:
+ PYTHON3=/usr/libexec/platform-python
+ /usr/lib/rpm/redhat/brp-mangle-shebangs
*** ERROR: ambiguous python shebang in /opt/impala/shell/lib/option_parser.py: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/lib/impala_client.py: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/lib/TSSLSocketWithWildcardSAN.py: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/StatestoreService/StatestoreService-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/StatestoreService/StatestoreSubscriber-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/beeswaxd/BeeswaxService-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/ImpalaService/ImpalaService-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/ImpalaService/ImpalaHiveServer2Service-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/TCLIService/TCLIService-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/fb303/FacebookService-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/hive_metastore/ThriftHiveMetastore-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/CatalogService/CatalogService-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/gen-py/NetworkTest/NetworkTestService-remote: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
*** ERROR: ambiguous python shebang in /opt/impala/shell/impala_shell.py: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
http://gerrit.cloudera.org:8080/#/c/18939/7/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:
http://gerrit.cloudera.org:8080/#/c/18939/7/bin/bootstrap_system.sh@246
PS7, Line 246: postgresql postgresql-server rpm-build \
> It would be interesting to see which exact version, release form factor (Do
Finally found it's an issue of the image I used. When I tried an EC2 image, it works..
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 07 Jun 2023 02:34:52 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 18.04/20.04 and CentOS 7 using
./buildall.sh -noclean -skiptests -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
- Deployed the DEB package on a docker container. Verified the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Reviewed-on: http://gerrit.cloudera.org:8080/18939
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
M docker/install_os_packages.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestored.sh
A package/bin/stop-catalogd.sh
A package/bin/stop-impalad.sh
A package/bin/stop-statestored.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
23 files changed, 579 insertions(+), 7 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 12
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13196/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 05 Jun 2023 11:46:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Yifan Zhang, Wenzhe Zhou, Joe McDonnell, Impala Public Jenkins, Xiang Yang,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#9).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 18.04/20.04 and CentOS 7 using
./buildall.sh -noclean -skiptests -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
- Deployed the DEB package on a docker container. Verified the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
M docker/install_os_packages.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestored.sh
A package/bin/stop-catalogd.sh
A package/bin/stop-impalad.sh
A package/bin/stop-statestored.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
23 files changed, 579 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/9
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Thu, 13 Jul 2023 03:33:03 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10:
> I ran the command './buildall.sh -noclean -notests -release
> -package' and found the ctests were built even with '--notests'
> option.
I ran into that on my system as well. -notests doesn't change the actual dependency graph generated by CMake (i.e. all the targets including the tests still exist). It instead works by changing the make target that buildall.sh invokes, and that make target doesn't build the tests. We did that so that developers could afterward build a single test after doing a -notests build. e.g. "make promise-test"
It looks like cpack is invoking "make" without a target and that builds everything including all the tests.
I will file a JIRA to see if we can have a mode that doesn't build any tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 14 Jul 2023 23:29:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10: Code-Review+2
I filed IMPALA-12288 to track having a mode that avoids building the tests when packaging.
I'm bumping up to +2, because I think we can address any issues we find in subsequent changes.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 14 Jul 2023 23:36:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/11285/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 05 Sep 2022 06:25:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/18939/1/bin/deploy/start-catalogd.sh
File bin/deploy/start-catalogd.sh:
http://gerrit.cloudera.org:8080/#/c/18939/1/bin/deploy/start-catalogd.sh@5
PS1, Line 5: # Edit core-site.xml, hdfs-site.xml, hive-site.xml, etc. in conf based on the cluster info.
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/18939/1/bin/deploy/start-impalad.sh
File bin/deploy/start-impalad.sh:
http://gerrit.cloudera.org:8080/#/c/18939/1/bin/deploy/start-impalad.sh@5
PS1, Line 5: # Edit core-site.xml, hdfs-site.xml, hive-site.xml, etc. in conf based on the cluster info.
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 05 Sep 2022 06:05:38 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/18939/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18939/4//COMMIT_MSG@20
PS4, Line 20: on a CDP cluster
> Is there restriction for CDP version?
Yes, the CDP version corresponding to CDP_BUILD_NUMBER in bin/impala-config.sh is the best. For impala-4.1, it's CDP 7.2.15. For impala-4.2, it would be CDP 7.2.16. Although the RPM can be installed on any nodes with other CDP versions, it's not guarenteed that the installed impala works with other CDP components.
> Could we upgrade or downgrade Impala package if there is old version of Impala package installed on the CDP cluster? If yes, old package will be overwrote or removed?
No, there are installed in independent folders. You can shutdown the CDP impala and lauch the Apache Impala using scripts provided by this patch. But the installation won't touch anyting inside the CDP parcel.
http://gerrit.cloudera.org:8080/#/c/18939/4/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/4/CMakeLists.txt@596
PS4, Line 596: endif()
> Should we print error message if os is not matching with redhat or ubuntu?
I don't have such a system for test. I think in this case, no packages will be generated unless we can specify some cmake options in the command line:
https://cmake.org/cmake/help/v3.22/module/CPack.html#variable:CPACK_GENERATOR
https://cmake.org/cmake/help/v3.22/module/CPack.html#variable:CPACK_BINARY_%3CGENNAME%3E
I'll add a warning.
http://gerrit.cloudera.org:8080/#/c/18939/4/be/CMakeLists.txt
File be/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/4/be/CMakeLists.txt@662
PS4, Line 662: install(DIRECTORY ${LLVM_IR_OUTPUT_DIRECTORY} DESTINATION ${IMPALA_INSTALLDIR}/)
> Should we run 'install' only when BUILD_PACKAGES is on?
This won't install anything unless we explicitly run "cmake --install ." in the IMPALA_HOME dir. I think it's ok to keep them if we wan't to compile and install Impala on the same node.
http://gerrit.cloudera.org:8080/#/c/18939/4/be/src/service/CMakeLists.txt
File be/src/service/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/4/be/src/service/CMakeLists.txt@117
PS4, Line 117: install(FILES ${STATESTORED_SYMLINK} DESTINATION ${IMPALA_INSTALLDIR}/bin)
> Should we run 'install' only when BUILD_PACKAGES is on?
Same reply as above
http://gerrit.cloudera.org:8080/#/c/18939/4/bin/impala-config.sh
File bin/impala-config.sh:
http://gerrit.cloudera.org:8080/#/c/18939/4/bin/impala-config.sh@850
PS4, Line 850: export STRIP_DEPLOYMENT_IMPALAD=true
> should we strip binaries for state-stored and catalogd as well?
statestored and catalogd are just symlinks to the impalad binary. So strip on impalad is enough.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 26 Oct 2022 07:22:42 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11708/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 26 Oct 2022 07:47:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 6:
(5 comments)
Addressed comments. Modified the script to set up HDFS short circuit read.
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@513
PS5, Line 513: install(DIRECTORY fe/target/dependency/ DESTINATION ${IMPALA_INSTALLDIR}/jar
: FILES_MATCHING PATTERN "*.jar")
> Nit: the regular build statically links this, so it may not be necessary.
Done
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@523
PS5, Line 523: add_custom_target(gen-deps ALL DEPENDS thrift-deps proto-deps fb-deps
: kudu-util-proto-deps kudu-rpc-proto-deps kudu-security-proto-de
> Nit: the regular build statically links this, so it may not be necessary.
Done
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@618
PS5, Line 618: # libsasl2-modules-gssapi-mit will include dependencies o
> For the docker build, we also strip the kudu client share library and the l
I tried adding the so files but they are not stripped. Later I realized this variable is changed to boolean type and all files except external libs will be stripeed:
https://cmake.cmake.narkive.com/jj2FbwKG/cpack-strip-files-not-working-for-files-not-built-by-project
We need additional works to strip the external so files. Is it ok to do it in a separate JIRA?
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@628
PS5, Line 628:
> How do we decide what belongs on this list?
Good point! Updated this list.
http://gerrit.cloudera.org:8080/#/c/18939/5/be/CMakeLists.txt
File be/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/5/be/CMakeLists.txt@712
PS5, Line 712:
> The llvm-ir bytecode is baked into the impalad executable, so it isn't nece
Thanks for pointing this out!
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Tue, 25 Apr 2023 02:56:11 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Thu, 13 Jul 2023 05:51:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10:
> Patch Set 10: Code-Review+2
>
> I filed IMPALA-12288 to track having a mode that avoids building the tests when packaging.
>
> I'm bumping up to +2, because I think we can address any issues we find in subsequent changes.
That makes sense to me. Thanks for filing the JIRA.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 15 Jul 2023 01:19:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 6:
(2 comments)
I think this is looking pretty good. I haven't tried it locally with the latest revision.
I would like to think about unifying the calculation of the dependency packages.
For ongoing development / functionality, how do we keep this tested / working? That doesn't necessarily have to happen in this change. One option would be to add code that creates a separate set of docker images by installing this package (rather than our usual process), and then we modify our dockerised tests to run with the modified images. If all of that got checked in, then it could be run periodically.
http://gerrit.cloudera.org:8080/#/c/18939/6/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/6/CMakeLists.txt@574
PS6, Line 574: if (EXISTS "/etc/redhat-release")
: set(IMPALA_LINUX_ID_FILE "/etc/redhat-release")
: elseif (EXISTS "/etc/os-release")
: set(IMPALA_LINUX_ID_FILE "/etc/os-release")
: elseif (EXISTS "/etc/issue")
: set(IMPALA_LINUX_ID_FILE "/etc/issue")
I noticed recently that /etc/os-release is available on all the distributions we support and I was going to replace lsb_release with /etc/os-release. Some of that work may end up being useful for this location as well. I was thinking there could be a script in bin/cmake_aux that essentially does this:
OS_NAME=$(source /etc/os-release && echo "$ID")
OS_VERSION=$(source /etc/os-release && echo "$VERSION_ID")
Either way, I think your logic would work fine for now.
http://gerrit.cloudera.org:8080/#/c/18939/6/CMakeLists.txt@613
PS6, Line 613: set(CPACK_RPM_PACKAGE_AUTOREQPROV " no")
: set(CPACK_RPM_PACKAGE_RELOCATABLE TRUE)
: set(CPACK_RPM_PACKAGE_REQUIRES
: "cyrus-sasl-gssapi, cyrus-sasl-plain, krb5-workstation, openldap-devel, tzdata")
Ideally, this would match the docker/install_os_packages.sh logic almost exactly (without the debug tools). If we changed that script to calculate the package list as a first step (before installing / verifying), then we could have an option that dumps the list of packages after that step and then exits. This CMake code could call that and feed its output into the list of requires here?
To be specific:
1. This list is already a bit out of date. We have some logic for Redhat 8 that adds glibc-langpack-en/langpacks-en to fix a couple Unicode things. We added a requirement on hostname. The docker images may have slightly different requirements, but if we could stick to a single list I think it will be more likely to stay up to date.
2. Do we need a dependency on Java? If so, which version?
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Fri, 02 Jun 2023 03:02:27 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10:
Yeah, not just the tests, but the benchmarks will also be built. Thanks for filing the follow-up JIRA, Joe! Thanks everyone's review!
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 15 Jul 2023 11:11:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
(2 comments)
Thanks a lot for picking this up, Quanlong; it addresses a long-standing gap.
http://gerrit.cloudera.org:8080/#/c/18939/7/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:
http://gerrit.cloudera.org:8080/#/c/18939/7/bin/bootstrap_system.sh@246
PS7, Line 246: postgresql postgresql-server rpm-build \
> This fails in Redhat8:
It would be interesting to see which exact version, release form factor (Docker image vs VM image) caused this failure.
I ran some quick tests on a pretty basic (company-internal) RedHat 8.6 image (it didn't even have Git), and rpm-build installed happily.
Maybe we should have a separate discussion on the mailing list to figure out the minimum Redhat8 version we want to support.
http://gerrit.cloudera.org:8080/#/c/18939/8/docker/install_os_packages.sh
File docker/install_os_packages.sh:
http://gerrit.cloudera.org:8080/#/c/18939/8/docker/install_os_packages.sh@178
PS8, Line 178: will
Nit: maybe change to 'would' ?
Since this is a DRY_RUN, nothing will actually be installed, this is just a reporting exercise. Using 'will' could mislead the user.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Tue, 06 Jun 2023 23:06:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 15 Jul 2023 01:16:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, Xiang Yang,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#4).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 16.04 and CentOS 7.5 using
./buildall.sh -noclean -skiptests -ninja -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestore.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
20 files changed, 354 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/4
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13485/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Fri, 07 Jul 2023 09:30:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11355/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Thu, 15 Sep 2022 01:45:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Xiang Yang (Code Review)" <ge...@cloudera.org>.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 2:
(7 comments)
Hi quanlong, thanks for your work! I have some questions and suggestions on this patch, waiting for your reply, thanks.
http://gerrit.cloudera.org:8080/#/c/18939/2/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/2/CMakeLists.txt@27
PS2, Line 27: option(BUILD_PACKAGES "Build installater packages")
typo in 'installater'?
http://gerrit.cloudera.org:8080/#/c/18939/2/CMakeLists.txt@608
PS2, Line 608: set(CPACK_STRIP_FILES ${IMPALA_INSTALLDIR}/bin/impalad)
Is this will strip the final 'bin/impalad'? can we make it optional?
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/common.sh
File bin/deploy/common.sh:
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/common.sh@1
PS2, Line 1: #!/bin/bash
Is it better to rename it to impala-env.sh? make it's function more clear, and follow common name format in other bigdata project.
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/start-catalogd.sh
File bin/deploy/start-catalogd.sh:
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/start-catalogd.sh@29
PS2, Line 29: echo "Using IMPALA_HOME: ${IMPALA_HOME:=/opt/impala}"
maybe better to assign IMPALA_HOME to current directory using '$(cd $(dirname ${BASH_SOURCE[0]});pwd)'.
http://gerrit.cloudera.org:8080/#/c/18939/2/bin/deploy/start-catalogd.sh@31
PS2, Line 31: $IMPALA_HOME/bin/catalogd --flagfile=$IMPALA_HOME/conf/catalogd_flags
I have some suggestions here:
1. Is it better to append script parameters to the tail of command so that can override the same flag in flagfile?
2. Is is possible to start all services by one script? further more, do start, stop, status operation by single script.
3. maybe lost the support of admissiond service.
http://gerrit.cloudera.org:8080/#/c/18939/2/fe/src/test/resources/deploy/catalogd_flags
File fe/src/test/resources/deploy/catalogd_flags:
PS2:
files below 'fe/src/test/resources/deploy' have little relation to fe, maybe better to move to a unified place(such as '${IMPALA_SOURCE}/package') combined with currently 'bin/deploy', then the directory will looks like:
/package/bin/start_xxx.sh
/package/conf/xxx-site.xml
http://gerrit.cloudera.org:8080/#/c/18939/2/fe/src/test/resources/deploy/core-site.xml
File fe/src/test/resources/deploy/core-site.xml:
http://gerrit.cloudera.org:8080/#/c/18939/2/fe/src/test/resources/deploy/core-site.xml@1
PS2, Line 1: <?xml version="1.0" encoding="UTF-8"?>
can we read xxx-site.xml by envirement variables first?
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 07 Sep 2022 12:53:09 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, Xiang Yang,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#3).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 16.04 and CentOS 7.5 using
./buildall.sh -noclean -skiptests -ninja -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestore.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
20 files changed, 352 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/3
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8575/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Thu, 15 Sep 2022 12:47:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 5:
(5 comments)
Thanks for putting this together
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@513
PS5, Line 513: FILE(GLOB hdfs_lib ${HDFS_LIB_PATHS}/libhdfs.so*)
: install(FILES ${hdfs_lib} DESTINATION ${IMPALA_INSTALLDIR}/lib)
Nit: the regular build statically links this, so it may not be necessary.
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@523
PS5, Line 523: FILE(GLOB hadoop_lib ${HDFS_LIB_PATHS}/libhadoop.so*)
: install(FILES ${hadoop_lib} DESTINATION ${IMPALA_INSTALLDIR}/lib)
Nit: the regular build statically links this, so it may not be necessary.
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@618
PS5, Line 618: set(CPACK_STRIP_FILES ${IMPALA_INSTALLDIR}/bin/impalad)
For the docker build, we also strip the kudu client share library and the libstdc++ shared library.
http://gerrit.cloudera.org:8080/#/c/18939/5/CMakeLists.txt@628
PS5, Line 628: set(CPACK_DEBIAN_PACKAGE_DEPENDS "libbz2-1.0,zlib1g,libstdc++6,libc6,libevent-1.4-2,libunwind7,libgcc1")
How do we decide what belongs on this list?
In our docker image, we install things like Cyrus SASL and Kerberos.
https://github.com/apache/impala/blob/master/docker/install_os_packages.sh#L93-L100
We statically link a lot of stuff from our toolchain (zlib, bzip2, libunwind, etc), so I'm not sure which of these we need.
http://gerrit.cloudera.org:8080/#/c/18939/5/be/CMakeLists.txt
File be/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/5/be/CMakeLists.txt@712
PS5, Line 712: install(DIRECTORY ${LLVM_IR_OUTPUT_DIRECTORY} DESTINATION ${IMPALA_INSTALLDIR}/)
The llvm-ir bytecode is baked into the impalad executable, so it isn't necessary to ship it.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Thu, 16 Feb 2023 02:15:52 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 11: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/9508/
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 15 Jul 2023 15:22:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 11: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sun, 16 Jul 2023 11:13:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 11: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 15 Jul 2023 11:12:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Wenzhe Zhou, Joe McDonnell, Impala Public Jenkins, Xiang Yang,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#8).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 18.04 and CentOS 7.5 using
./buildall.sh -noclean -skiptests -ninja -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
- Deployed the DEB package on a docker container. Verified the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
M docker/install_os_packages.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestore.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
20 files changed, 421 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/8
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
> For ongoing development / functionality, how do we keep this tested / working? That doesn't necessarily have to happen in this change. One option would be to add code that creates a separate set of docker images by installing this package (rather than our usual process), and then we modify our dockerised tests to run with the modified images. If all of that got checked in, then it could be run periodically.
Yeah, agree these. Currently we just add test for building packages. I can work on adding functional tests in future patches.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 05 Jun 2023 13:57:22 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10: Code-Review+1
This change looks good to me. We can expand on it in subsequent JIRAs as needed. I'm +1'ing it now in case other reviewers have comments, but I'm willing to +2 it.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 12 Jul 2023 22:05:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Wenzhe Zhou, Joe McDonnell, Impala Public Jenkins, Xiang Yang,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#6).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 18.04 and CentOS 7.5 using
./buildall.sh -noclean -skiptests -ninja -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
- Deployed the DEB package on a docker container. Verified the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestore.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
19 files changed, 373 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/6
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12858/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Tue, 25 Apr 2023 01:11:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Yifan Zhang, Wenzhe Zhou, Joe McDonnell, Impala Public Jenkins, Xiang Yang,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#10).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 18.04/20.04 and CentOS 7 using
./buildall.sh -noclean -skiptests -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
- Deployed the DEB package on a docker container. Verified the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
M docker/install_os_packages.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestored.sh
A package/bin/stop-catalogd.sh
A package/bin/stop-impalad.sh
A package/bin/stop-statestored.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
23 files changed, 579 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/10
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 10:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9497/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Thu, 13 Jul 2023 01:39:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 11:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9508/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sat, 15 Jul 2023 11:12:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 11:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9509/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Sun, 16 Jul 2023 06:58:34 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18939/8/package/bin/impala-env.sh
File package/bin/impala-env.sh:
http://gerrit.cloudera.org:8080/#/c/18939/8/package/bin/impala-env.sh@32
PS8, Line 32: export LIBHDFS_OPTS="${LIBHDFS_OPTS:-} -Djava.library.path=${HADOOP_LIB_DIR}/native/"
: echo "Using hadoop native libs in ${HADOOP_LIB_DIR}/native/"
: else
: echo "WARNING: HDFS short-circuit reads are not enabled due to HADOOP_HOME not set."
nit: Could we also pack hadoop native libs into the final package?
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Mon, 26 Jun 2023 07:04:21 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Wenzhe Zhou, Impala Public Jenkins, Xiang Yang,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18939
to look at the new patch set (#5).
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
IMPALA-10262: RPM/DEB Packaging Support
This patch bases on a previous patch contributed by Shant Hovsepian:
https://gerrit.cloudera.org/c/16612/
It adds a new option, -package, to buildall.sh for building a package
for the current OS type (e.g. CentOS/Ubuntu). You can also use
"make/ninja package" to build the package. Scripts for launching the
services and the required configuration files are also added.
Tests:
- Built on Ubuntu 16.04 and CentOS 7.5 using
./buildall.sh -noclean -skiptests -ninja -release -package
- Deployed the RPM package on a CDP cluster. Verifed the scripts.
Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/service/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M bin/rat_exclude_files.txt
M buildall.sh
A package/bin/impala-env.sh
A package/bin/start-catalogd.sh
A package/bin/start-impalad.sh
A package/bin/start-statestore.sh
A package/conf/catalogd_flags
A package/conf/core-site.xml
A package/conf/fair-scheduler.xml
A package/conf/hdfs-site.xml
A package/conf/hive-site.xml
A package/conf/impalad_flags
A package/conf/llama-site.xml
A package/conf/statestore_flags
20 files changed, 356 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/18939/5
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 5: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Wed, 26 Oct 2022 17:18:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10262: RPM/DEB Packaging Support
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/18939 )
Change subject: IMPALA-10262: RPM/DEB Packaging Support
......................................................................
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/18939/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18939/4//COMMIT_MSG@20
PS4, Line 20: on a CDP cluster
Is there restriction for CDP version? Could we upgrade or downgrade Impala package if there is old version of Impala package installed on the CDP cluster? If yes, old package will be overwrote or removed?
http://gerrit.cloudera.org:8080/#/c/18939/4/CMakeLists.txt
File CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/4/CMakeLists.txt@596
PS4, Line 596: endif()
Should we print error message if os is not matching with redhat or ubuntu?
http://gerrit.cloudera.org:8080/#/c/18939/4/be/CMakeLists.txt
File be/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/4/be/CMakeLists.txt@662
PS4, Line 662: install(DIRECTORY ${LLVM_IR_OUTPUT_DIRECTORY} DESTINATION ${IMPALA_INSTALLDIR}/)
Should we run 'install' only when BUILD_PACKAGES is on?
http://gerrit.cloudera.org:8080/#/c/18939/4/be/src/service/CMakeLists.txt
File be/src/service/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18939/4/be/src/service/CMakeLists.txt@117
PS4, Line 117: install(FILES ${STATESTORED_SYMLINK} DESTINATION ${IMPALA_INSTALLDIR}/bin)
Should we run 'install' only when BUILD_PACKAGES is on?
http://gerrit.cloudera.org:8080/#/c/18939/4/bin/impala-config.sh
File bin/impala-config.sh:
http://gerrit.cloudera.org:8080/#/c/18939/4/bin/impala-config.sh@850
PS4, Line 850: export STRIP_DEPLOYMENT_IMPALAD=true
should we strip binaries for state-stored and catalogd as well?
--
To view, visit http://gerrit.cloudera.org:8080/18939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64419fd400fe8d233dac016b6306157fe9461d82
Gerrit-Change-Number: 18939
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Xiang Yang <yx...@126.com>
Gerrit-Comment-Date: Mon, 19 Sep 2022 08:00:47 +0000
Gerrit-HasComments: Yes