You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/06/16 00:35:00 UTC
[jira] [Commented] (IMPALA-11359) bootstrap_toolchain.py failed to download Apache Hive3
[ https://issues.apache.org/jira/browse/IMPALA-11359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554837#comment-17554837 ]
Quanlong Huang commented on IMPALA-11359:
-----------------------------------------
I guess there are some issue with my local certificates. Simply resolved this by adding the --no-check-certificate option:
{code:python}
diff --git a/bin/bootstrap_toolchain.py b/bin/bootstrap_toolchain.py
index 60bb570..6e09536 100755
--- a/bin/bootstrap_toolchain.py
+++ b/bin/bootstrap_toolchain.py
@@ -130,7 +130,7 @@ def wget_and_unpack_package(download_path, file_name, destination, wget_no_clobb
download_path, destination, file_name, attempt))
# --no-clobber avoids downloading the file if a file with the name already exists
try:
- cmd = ["wget", download_path,
+ cmd = ["wget", "--no-check-certificate", download_path,
"--output-document={0}/{1}".format(destination, file_name)]
if wget_no_clobber:
cmd.append("--no-clobber") {code}
> bootstrap_toolchain.py failed to download Apache Hive3
> ------------------------------------------------------
>
> Key: IMPALA-11359
> URL: https://issues.apache.org/jira/browse/IMPALA-11359
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> I try building with Apache Hive 3 locally and hit this error:
> {code:bash}
> $ export USE_APACHE_HIVE=true
> $ ./buildall.sh -noclean -notests -ninja
> IMPALA_VERSION = 4.2.0-SNAPSHOT
> IMPALA_HOME = /home/quanlong/workspace/Impala
> HADOOP_HOME = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33/
> HADOOP_CONF_DIR = /home/quanlong/workspace/Impala/fe/src/test/resources
> HADOOP_INCLUDE_DIR = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//include
> HADOOP_LIB_DIR = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib
> IMPALA_CLUSTER_NODES_DIR= /home/quanlong/workspace/Impala/testdata/cluster/cdh7
> HIVE_HOME = /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin
> HIVE_CONF_DIR = /home/quanlong/workspace/Impala/fe/./src/test/resources
> HIVE_SRC_DIR = /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src
> HBASE_HOME = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hbase-2.4.6.7.2.16.0-33/
> HBASE_CONF_DIR = /home/quanlong/workspace/Impala/fe/src/test/resources
> RANGER_HOME = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/ranger-2.1.0.7.2.16.0-33-admin
> RANGER_CONF_DIR = /home/quanlong/workspace/Impala/fe/src/test/resources
> THRIFT_HOME = /home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc7.5.0/thrift-0.11.0-p5
> CLASSPATH = /home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency::/home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency::/home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency:
> LIBHDFS_OPTS = -Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/ -XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m -Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/ -XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m -Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/ -XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m
> JAVA_HOME = /usr/lib/jvm/java-8-openjdk-amd64
> POSTGRES_JDBC_DRIVER = /home/quanlong/workspace/Impala/fe/target/dependency/postgresql-42.3.3.jar
> IMPALA_TOOLCHAIN = /home/quanlong/workspace/Impala/toolchain
> IMPALA_TOOLCHAIN_PACKAGES_HOME = /home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc7.5.0
> METASTORE_DB = HMS_home_quanlong_workspace_Impala_cdp
> DOWNLOAD_CDH_COMPONENTS = true
> IMPALA_MAVEN_OPTIONS =
> IMPALA_TOOLCHAIN_HOST = native-toolchain.s3.amazonaws.com
> CDP_BUILD_NUMBER = 26065553
> CDP_COMPONENTS_HOME = /home/quanlong/workspace/Impala/toolchain/cdp_components-26065553
> APACHE_MIRROR = https://www.apache.org/dyn/closer.cgi
> APACHE_COMPONENTS_HOME = /home/quanlong/workspace/Impala/toolchain/apache_components
> IMPALA_HADOOP_VERSION = 3.1.1.7.2.16.0-33
> IMPALA_AVRO_JAVA_VERSION= 1.8.2.7.2.16.0-33
> IMPALA_PARQUET_VERSION = 1.10.99.7.2.16.0-33
> IMPALA_HIVE_VERSION = 3.1.2
> IMPALA_HBASE_VERSION = 2.4.6.7.2.16.0-33
> IMPALA_HUDI_VERSION = 0.5.0-incubating
> IMPALA_KUDU_VERSION = 09a6c8833e
> IMPALA_RANGER_VERSION = 2.1.0.7.2.16.0-33
> IMPALA_ICEBERG_VERSION = 0.13.1.7.2.16.0-33
> IMPALA_GCS_VERSION = 2.1.2.7.2.16.0-33
> IMPALA_COS_VERSION = 3.1.0-5.9.3
> SKIP_PYTHON_DOWNLOAD is true, skipping python dependencies download.
> SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping toolchain bootstrap.
> >>> Downloading and extracting cdh components.
> 2022-06-16 08:12:30,701 Thread-2 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz (attempt 1)
> 2022-06-16 08:12:30,702 Thread-1 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz (attempt 1)
> 2022-06-16 08:12:31,534 Thread-2 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz']' failed with exit code 5:
> --2022-06-16 08:12:30-- https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
> Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
> Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
> ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
> Issued certificate has expired.
> To connect to www.apache.org insecurely, use `--no-check-certificate'.
> 2022-06-16 08:12:31,535 Thread-1 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']' failed with exit code 5:
> --2022-06-16 08:12:30-- https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
> Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
> Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
> ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
> Issued certificate has expired.
> To connect to www.apache.org insecurely, use `--no-check-certificate'.
> 2022-06-16 08:12:43,924 Thread-1 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz (attempt 2)
> 2022-06-16 08:12:44,749 Thread-1 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']' failed with exit code 5:
> --2022-06-16 08:12:43-- https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
> Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
> Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
> ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
> Issued certificate has expired.
> To connect to www.apache.org insecurely, use `--no-check-certificate'.
> 2022-06-16 08:12:45,670 Thread-2 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz (attempt 2)
> 2022-06-16 08:12:46,503 Thread-2 ERROR: Download failed; retrying after sleep: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz']' failed with exit code 5:
> --2022-06-16 08:12:45-- https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
> Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
> Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
> ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
> Issued certificate has expired.
> To connect to www.apache.org insecurely, use `--no-check-certificate'.
> 2022-06-16 08:12:55,981 Thread-1 INFO: Downloading https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download to /home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz (attempt 3)
> Traceback (most recent call last):
> File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 568, in <module>
> if __name__ == "__main__": main()
> File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 565, in main
> execute_many(download, components_needing_download)
> File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 441, in execute_many
> return pool.map(f, args, 1)
> File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map
> return self.map_async(func, iterable, chunksize).get()
> File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
> raise self._value
> Exception: Command with args '['wget', 'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download', '--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']' failed with exit code 5:
> --2022-06-16 08:12:55-- https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
> Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
> Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
> ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
> Issued certificate has expired.
> To connect to www.apache.org insecurely, use `--no-check-certificate'.{code}
> CC [~chufucun]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org