You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2023/01/10 17:05:13 UTC
[doris] branch branch-1.2-lts updated (8779a010fd -> de520a68c9)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
from 8779a010fd [fix](tvf) cancel strict restrictions on tvf parameters (#15764)
new e3aad63f74 [chore](macOS) Don't build useless third-party stuff (#15763)
new de520a68c9 [deps](hdfs) update libhdfs3 to v2.3.5 to support KMS (#15770)
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/en/docs/ecosystem/external-table/multi-catalog.md | 5 +++++
docs/zh-CN/docs/ecosystem/external-table/multi-catalog.md | 5 +++++
thirdparty/build-thirdparty.sh | 10 ++++++----
thirdparty/vars.sh | 8 ++++----
4 files changed, 20 insertions(+), 8 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[doris] 01/02: [chore](macOS) Don't build useless third-party stuff (#15763)
Posted by mo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
commit e3aad63f74fec7a323091ecf7580362fcac52f88
Author: Adonis Ling <ad...@gmail.com>
AuthorDate: Wed Jan 11 00:20:37 2023 +0800
[chore](macOS) Don't build useless third-party stuff (#15763)
On macOS, we need some extra libraries to build the codebase,
therefore two packages were introduced to the project. They are `binutils` and `gettext`.
It takes a lot of time to build these packages completely. This PR introduces a way to build the needed libraries
and other stuff are skipped to build. It can save the time to build the third-party libraries on macOS.
---
thirdparty/build-thirdparty.sh | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 6e64b89f5c..641a870350 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -1059,7 +1059,7 @@ build_bitshuffle() {
if [[ ! -f "${nm}" ]]; then nm="$(command -v nm)"; fi
if [[ ! -f "${objcopy}" ]]; then
if ! objcopy="$(command -v objcopy)"; then
- objcopy="${TP_INSTALL_DIR}/bin/objcopy"
+ objcopy="${TP_INSTALL_DIR}/binutils/bin/objcopy"
fi
fi
@@ -1507,9 +1507,10 @@ build_binutils() {
mkdir -p "${BUILD_DIR}"
cd "${BUILD_DIR}"
- ../configure --prefix="${TP_INSTALL_DIR}" --enable-install-libiberty
+ ../configure --prefix="${TP_INSTALL_DIR}/binutils" --includedir="${TP_INCLUDE_DIR}" --libdir="${TP_LIB_DIR}" \
+ --enable-install-libiberty --without-msgpack
make -j "${PARALLEL}"
- make install
+ make install-bfd install-libiberty install-binutils
}
build_gettext() {
@@ -1520,7 +1521,8 @@ build_gettext() {
mkdir -p "${BUILD_DIR}"
cd "${BUILD_DIR}"
- ../configure --prefix="${TP_INSTALL_DIR}" --disable-java
+ ../gettext-runtime/configure --prefix="${TP_INSTALL_DIR}" --disable-java
+ cd intl
make -j "${PARALLEL}"
make install
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[doris] 02/02: [deps](hdfs) update libhdfs3 to v2.3.5 to support KMS (#15770)
Posted by mo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
commit de520a68c9370a5bf71ab2e1f5301c1de171a069
Author: Ashin Gau <As...@users.noreply.github.com>
AuthorDate: Tue Jan 10 23:21:53 2023 +0800
[deps](hdfs) update libhdfs3 to v2.3.5 to support KMS (#15770)
Support KMS in libhdfs3: apache/doris-thirdparty#22
---
docs/en/docs/ecosystem/external-table/multi-catalog.md | 5 +++++
docs/zh-CN/docs/ecosystem/external-table/multi-catalog.md | 5 +++++
thirdparty/vars.sh | 8 ++++----
3 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/docs/en/docs/ecosystem/external-table/multi-catalog.md b/docs/en/docs/ecosystem/external-table/multi-catalog.md
index b4d50f028d..41f94ad4b0 100644
--- a/docs/en/docs/ecosystem/external-table/multi-catalog.md
+++ b/docs/en/docs/ecosystem/external-table/multi-catalog.md
@@ -150,6 +150,11 @@ CREATE CATALOG hive PROPERTIES (
);
```
+If you want to connect to Hadoop with KMS authentication, you should add the follow configuration into properties:
+```
+'dfs.encryption.key.provider.uri' = 'kms://http@kms_host:kms_port/kms'
+```
+
Once created, you can view the catalog with the `SHOW CATALOGS` command:
```
diff --git a/docs/zh-CN/docs/ecosystem/external-table/multi-catalog.md b/docs/zh-CN/docs/ecosystem/external-table/multi-catalog.md
index 6af4d9b848..58f7bfe578 100644
--- a/docs/zh-CN/docs/ecosystem/external-table/multi-catalog.md
+++ b/docs/zh-CN/docs/ecosystem/external-table/multi-catalog.md
@@ -150,6 +150,11 @@ CREATE CATALOG hive PROPERTIES (
);
```
+如果需要 hadoop KMS 认证，可以在properties中添加:
+```
+'dfs.encryption.key.provider.uri' = 'kms://http@kms_host:kms_port/kms'
+```
+
创建后，可以通过 `SHOW CATALOGS` 命令查看 catalog：
```
diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh
index 87b6cc49b0..7a7e1b11af 100644
--- a/thirdparty/vars.sh
+++ b/thirdparty/vars.sh
@@ -357,10 +357,10 @@ KRB5_SOURCE="krb5-1.19"
KRB5_MD5SUM="aaf18447a5a014aa3b7e81814923f4c9"
# hdfs3
-HDFS3_DOWNLOAD="https://github.com/apache/doris-thirdparty/archive/refs/tags/libhdfs3-v2.3.4.tar.gz"
-HDFS3_NAME="doris-thirdparty-libhdfs3-v2.3.4.tar.gz"
-HDFS3_SOURCE="doris-thirdparty-libhdfs3-v2.3.4"
-HDFS3_MD5SUM="5da1d32ec83b3732ccbe26346a7f5f9a"
+HDFS3_DOWNLOAD="https://github.com/apache/doris-thirdparty/archive/refs/tags/libhdfs3-v2.3.5.tar.gz"
+HDFS3_NAME="doris-thirdparty-libhdfs3-v2.3.5.tar.gz"
+HDFS3_SOURCE="doris-thirdparty-libhdfs3-v2.3.5"
+HDFS3_MD5SUM="8da623120add76f5595e5978b123e157"
#libdivide
LIBDIVIDE_DOWNLOAD="https://github.com/ridiculousfish/libdivide/archive/5.0.tar.gz"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org