You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by jo...@apache.org on 2023/07/14 19:04:04 UTC
[impala] branch master updated (14a5e9a20 -> 0df29b410)
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
from 14a5e9a20 IMPALA-12239: BitWidthZeroRepeated seems to be flaky
new b41339c22 IMPALA-12285: Use targetType in string-to-numeric literal conversion
new 0df29b410 IMPALA-12283: Remove Hive libs from PYTHONPATH
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
README-build.md | 2 +-
bin/set-pythonpath.sh | 3 ---
fe/src/main/java/org/apache/impala/analysis/StringLiteral.java | 10 +++++-----
.../functional-query/queries/QueryTest/insert-unsafe.test | 3 +++
4 files changed, 9 insertions(+), 9 deletions(-)
[impala] 01/02: IMPALA-12285: Use targetType in string-to-numeric literal conversion
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit b41339c22d94a26b8cd707ddc8a97698785389de
Author: Peter Rozsa <pr...@cloudera.com>
AuthorDate: Fri Jul 14 11:14:07 2023 +0200
IMPALA-12285: Use targetType in string-to-numeric literal conversion
This change fixes mismatched type problems when an implicitly casted
string literal gets converted to a numeric type. Example:
'INSERT INTO example(float_col) VALUES ("0"), (15629);'
After this change, StringLiteral's 'convertToNumber' method will
consider the targetType parameter when creates a new NumericLiteral.
Test:
- test case added to insert-unsafe.test
Change-Id: I2141e7ab164af55a7fa66dda05fe6dcbd7379b69
Reviewed-on: http://gerrit.cloudera.org:8080/20197
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
fe/src/main/java/org/apache/impala/analysis/StringLiteral.java | 10 +++++-----
.../functional-query/queries/QueryTest/insert-unsafe.test | 3 +++
2 files changed, 8 insertions(+), 5 deletions(-)
diff --git a/fe/src/main/java/org/apache/impala/analysis/StringLiteral.java b/fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
index d0b93ef8e..eb1f331fe 100644
--- a/fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
+++ b/fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
@@ -156,7 +156,7 @@ public class StringLiteral extends LiteralExpr {
} else if (targetType.isStringType()) {
type_ = targetType;
} else if (targetType.isNumericType()) {
- return convertToNumber();
+ return convertToNumber(targetType);
} else if (targetType.isDateOrTimeType()) {
// Let the BE do the cast
// - it is in Boost format in case target type is TIMESTAMP
@@ -169,14 +169,14 @@ public class StringLiteral extends LiteralExpr {
/**
* Convert this string literal to numeric literal.
*
+ * @param targetType sets the target type of the newly created literal
* @return new converted literal (not null)
* the type of the literal is determined by the lexical scanner
* @throws AnalysisException
* if NumberFormatException occurs,
* or if floating point value is NaN or infinite
*/
- public LiteralExpr convertToNumber()
- throws AnalysisException {
+ public LiteralExpr convertToNumber(Type targetType) throws AnalysisException {
StringReader reader = new StringReader(value_);
SqlScanner scanner = new SqlScanner(reader);
// For distinguishing positive and negative numbers.
@@ -201,12 +201,12 @@ public class StringLiteral extends LiteralExpr {
if (sym.sym == SqlParserSymbols.INTEGER_LITERAL) {
BigDecimal val = (BigDecimal) sym.value;
if (negative) val = val.negate();
- return new NumericLiteral(val);
+ return new NumericLiteral(val, targetType);
}
if (sym.sym == SqlParserSymbols.DECIMAL_LITERAL) {
BigDecimal val = (BigDecimal) sym.value;
if (negative) val = val.negate();
- return new NumericLiteral(val);
+ return new NumericLiteral(val, targetType);
}
// Symbol is not an integer or floating point literal.
throw new AnalysisException("Failed to convert string literal '"
diff --git a/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test b/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
index 05e588632..d21eeacd5 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
@@ -152,6 +152,9 @@ INSERT INTO unsafe_insert(char_col) values (cast("100" as STRING));
INSERT INTO unsafe_insert(varchar_col) values (cast("100" as STRING));
====
---- QUERY
+INSERT INTO unsafe_insert(float_col) values ("100"), (15629);
+====
+---- QUERY
INSERT INTO unsafe_insert(bigint_col) select string_col from unsafe_insert;
---- CATCH
AnalysisException: Unsafe implicit cast is prohibited for non-const expression: string_col
[impala] 02/02: IMPALA-12283: Remove Hive libs from PYTHONPATH
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 0df29b4105c0e884cabde3289874b9850e3df21d
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Mon Apr 3 21:20:22 2023 -0700
IMPALA-12283: Remove Hive libs from PYTHONPATH
bin/set-pythonpath.sh include $HIVE_HOME/lib/py. This is a
historical thing that is no longer needed today. Impala
should not be getting Python code directly from Hive. As a
cleanup, this removes $HIVE_HOME/lib/py from the
PYTHONPATH.
Testing:
- Ran a core job
Change-Id: I56d1ae3b1433d6240159f20da4680888b5f37357
Reviewed-on: http://gerrit.cloudera.org:8080/19689
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
---
README-build.md | 2 +-
bin/set-pythonpath.sh | 3 ---
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/README-build.md b/README-build.md
index 51f750552..d96827006 100644
--- a/README-build.md
+++ b/README-build.md
@@ -35,7 +35,7 @@ can do so through the environment variables and scripts listed below.
| IMPALA_JDK_VERSION | "system" | Set to 8 or 11 to select a system Java version. Default will set JAVA_HOME based on the javac symlink in PATH. |
| JAVA | "${JAVA_HOME}/bin/java" | Java binary location. |
| CLASSPATH | | See bin/set-classpath.sh for details. |
-| PYTHONPATH | Will be changed to include: "${IMPALA_HOME}/shell/gen-py" "${IMPALA_HOME}/testdata" "${THRIFT_PY_HOME}/python/lib/python2.7/site-packages" "${HIVE_HOME}/lib/py" |
+| PYTHONPATH | Will be changed to include: "${IMPALA_HOME}/shell/gen-py" "${IMPALA_HOME}/testdata" "${THRIFT_PY_HOME}/python/lib/python2.7/site-packages" |
## Source Directories for Impala
diff --git a/bin/set-pythonpath.sh b/bin/set-pythonpath.sh
index f2823ab55..9e1c933d1 100755
--- a/bin/set-pythonpath.sh
+++ b/bin/set-pythonpath.sh
@@ -42,6 +42,3 @@ for PYTHON_DIR in ${THRIFT_PY_HOME}/python/lib{64,}; do
PYTHONPATH=${PYTHONPATH}:${PKG_DIR}/
done
done
-
-# Add Hive after Thrift because Hive supplies its own Thrift modules
-PYTHONPATH=${PYTHONPATH}:${HIVE_HOME}/lib/py