You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by jo...@apache.org on 2020/03/03 00:23:51 UTC
[impala] branch master updated (48e08e4 -> 8357f61)
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git.
from 48e08e4 IMPALA-9445: skip test_local_tz_conversion for hbase
new d3a2d73 IMPALA-9439: Make --scale a mandatory option in single_node_perf_run.py
new 8357f61 IMPALA-9444: Fix URL for postgresql jar download
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
bin/create-test-configuration.sh | 2 +-
bin/single_node_perf_run.py | 15 +++++++++++++--
2 files changed, 14 insertions(+), 3 deletions(-)
[impala] 02/02: IMPALA-9444: Fix URL for postgresql jar download
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 8357f61886fe53fe9800df281099ae487b32f3a5
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Mon Mar 2 10:54:46 2020 -0800
IMPALA-9444: Fix URL for postgresql jar download
The current URL uses http://central.maven.org, which has been
decommissioned as part of the transition to HTTPS. See:
https://central.sonatype.org/articles/2019/Jul/15/central-http-deprecation-update/
https://central.sonatype.org/articles/2020/Jan/15/501-https-required-error/
This switches the URL to use https://repo.maven.apache.org/.
Testing:
- Removed postgresql jar, ran bin/create-test-configuration.sh,
verified that it downloaded the jar.
Change-Id: I7ee9a1ce77bc3f8c6b3f728633cafe4eb37e669d
Reviewed-on: http://gerrit.cloudera.org:8080/15337
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
bin/create-test-configuration.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bin/create-test-configuration.sh b/bin/create-test-configuration.sh
index 6b012f7..1574e59 100755
--- a/bin/create-test-configuration.sh
+++ b/bin/create-test-configuration.sh
@@ -243,7 +243,7 @@ if [[ -f "${POSTGRES_JDBC_DRIVER}" ]]; then
cp -f "${POSTGRES_JDBC_DRIVER}" "${RANGER_SERVER_LIB_DIR}"
else
# IMPALA-8261: Running this script should not fail when FE has not been built.
- MAVEN_URL="http://central.maven.org/maven2/org/postgresql/postgresql"
+ MAVEN_URL="https://repo.maven.apache.org/maven2/org/postgresql/postgresql"
JDBC_JAR="postgresql-${IMPALA_POSTGRES_JDBC_DRIVER_VERSION}.jar"
wget -P "${RANGER_SERVER_LIB_DIR}" \
"${MAVEN_URL}/${IMPALA_POSTGRES_JDBC_DRIVER_VERSION}/${JDBC_JAR}"
[impala] 01/02: IMPALA-9439: Make --scale a mandatory option in
single_node_perf_run.py
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit d3a2d73fda48fde5e35c2c2d92730d4c2c9a3eb9
Author: Sahil Takiar <ta...@gmail.com>
AuthorDate: Mon Mar 2 08:30:50 2020 -0800
IMPALA-9439: Make --scale a mandatory option in single_node_perf_run.py
This makes the --scale option mandatory when running
./bin/single_node_perf_run.py. If the option is not set, the script
attempts to run the workloads against the database
'[workload-name]None_[file-format]', which is typically not what the
user wants.
Makes some minor documentation improvements to the script.
Testing:
* Confirmed that running without the --scale option set causes the
script to error out with a help message
Change-Id: I9ad13580f8f74388981a37d6960087d95cde574b
Reviewed-on: http://gerrit.cloudera.org:8080/15335
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
bin/single_node_perf_run.py | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/bin/single_node_perf_run.py b/bin/single_node_perf_run.py
index 6114223..d215019 100755
--- a/bin/single_node_perf_run.py
+++ b/bin/single_node_perf_run.py
@@ -34,6 +34,13 @@
# to git_hash_B, the second two refer to git_hash_A. The column "Delta(Avg)"
# is negative if git_hash_B is faster and is positive if git_hash_A is faster.
#
+# To run this script against data stored in Kudu, set '--table_formats=kudu/none/none'.
+#
+# For a given workload, the target database used will be:
+# '[workload-name][scale-factor]_[table_format]'. Typically, on the first run of this
+# script the target database will not exist. The --load option will be needed to load
+# the database.
+#
# WARNING: This script will run git checkout. You should not touch the tree
# while the script is running. You should start the script from a clean git
# tree.
@@ -46,7 +53,7 @@
# --workloads=WORKLOADS
# comma-separated list of workloads. Choices: tpch,
# targeted-perf, tpcds. Default: targeted-perf
-# --scale=SCALE scale factor for the workloads
+# --scale=SCALE scale factor for the workloads [required]
# --iterations=ITERATIONS
# number of times to run each query
# --table_formats=TABLE_FORMATS
@@ -266,7 +273,7 @@ def parse_options():
parser.add_option("--workloads", default="targeted-perf",
help="comma-separated list of workloads. Choices: tpch, "
"targeted-perf, tpcds. Default: targeted-perf")
- parser.add_option("--scale", help="scale factor for the workloads")
+ parser.add_option("--scale", help="scale factor for the workloads [required]")
parser.add_option("--iterations", default=30, help="number of times to run each query")
parser.add_option("--table_formats", default="parquet/none", help="comma-separated "
"list of table formats. Default: parquet/none")
@@ -315,6 +322,10 @@ def parse_options():
parser.print_usage(sys.stderr)
raise Exception("Invalid arguments: either 1 or 2 Git hashes allowed")
+ if not options.scale:
+ parser.print_help(sys.stderr)
+ raise Exception("--scale is required")
+
return options, args