You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by st...@apache.org on 2019/11/26 00:04:16 UTC
[impala] branch master updated (4c09975 -> 1f792be)
This is an automated email from the ASF dual-hosted git repository.
stakiar pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git.
from 4c09975 IMPALA-9165: Add back hard kill to kill-hbase.sh
new db69fe8 IMPALA-9157: Make helper function exec_local_command python 2.6 compatible
new 1f792be IMPALA-9188: Composite primary keys should use same constraint name
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
fe/src/main/java/org/apache/impala/analysis/TableDef.java | 7 ++++++-
lib/python/impala_py_lib/helpers.py | 15 ++++++++++++++-
2 files changed, 20 insertions(+), 2 deletions(-)
[impala] 01/02: IMPALA-9157: Make helper function
exec_local_command python 2.6 compatible
Posted by st...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
stakiar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit db69fe87a8c152709f069fe048fe4b0f595f4864
Author: David Knupp <dk...@cloudera.com>
AuthorDate: Thu Nov 21 15:56:48 2019 -0800
IMPALA-9157: Make helper function exec_local_command python 2.6 compatible
Change-Id: I928a4df9dc883e8e801a42ead14ec7875169d4ae
Reviewed-on: http://gerrit.cloudera.org:8080/14788
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
lib/python/impala_py_lib/helpers.py | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/lib/python/impala_py_lib/helpers.py b/lib/python/impala_py_lib/helpers.py
index 93a868f..a6c68d8 100644
--- a/lib/python/impala_py_lib/helpers.py
+++ b/lib/python/impala_py_lib/helpers.py
@@ -16,22 +16,35 @@
# under the License.
import fnmatch
+import logging
import os
import re
import subprocess
+logging.basicConfig()
+LOG = logging.getLogger('impala_lib_python_helpers')
+
def exec_local_command(cmd):
"""
Executes a command for the local bash shell and return stdout as a string.
+ Raise CalledProcessError in case of non-zero return code.
+
Args:
cmd: command as a string
Return:
STDOUT
"""
- return subprocess.check_output(cmd.split())
+ proc = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ output, error = proc.communicate()
+ retcode = proc.poll()
+ if retcode:
+ LOG.error("{0} returned status {1}: {2}".format(cmd, retcode, error))
+ raise subprocess.CalledProcessError()
+ else:
+ return output
def find_all_files(fname_pattern, base_dir=os.getenv('IMPALA_HOME', '.')):
[impala] 02/02: IMPALA-9188: Composite primary keys should use same
constraint name
Posted by st...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
stakiar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 1f792bea4779229acbaaa40d4904df4546658bfb
Author: Sahil Takiar <ta...@gmail.com>
AuthorDate: Sun Nov 24 11:46:16 2019 -0800
IMPALA-9188: Composite primary keys should use same constraint name
After IMPALA-9104, dataload started failing when USE_CDP_HIVE=true
because of HIVE-16603 (Enforce foreign keys to refer to primary keys or
unique keys), which is in Hive 3 but not Hive 2.
This patch fixes a bug from IMPALA-2112 where Impala was generating a
unique constraint name for each column in a composite primary key,
rather than using the same constraint name for each column.
Testing:
* Ran dataload + core tests with USE_CDP_HIVE=[true/false]
Change-Id: I7a91658945b0355753c3cf05be195757b37edaf4
Reviewed-on: http://gerrit.cloudera.org:8080/14792
Reviewed-by: Sahil Takiar <st...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
fe/src/main/java/org/apache/impala/analysis/TableDef.java | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/fe/src/main/java/org/apache/impala/analysis/TableDef.java b/fe/src/main/java/org/apache/impala/analysis/TableDef.java
index 14dfe05..e196af1 100644
--- a/fe/src/main/java/org/apache/impala/analysis/TableDef.java
+++ b/fe/src/main/java/org/apache/impala/analysis/TableDef.java
@@ -470,6 +470,7 @@ class TableDef {
}
Map<String, ColumnDef> colDefsByColName = ColumnDef.mapByColumnNames(columnDefs_);
int keySeq = 1;
+ String constraintName = null;
for (String colName: primaryKeyColNames_) {
colName = colName.toLowerCase();
ColumnDef colDef = colDefsByColName.remove(colName);
@@ -494,7 +495,11 @@ class TableDef {
if (primaryKey_.isValidateCstr()) {
throw new AnalysisException("VALIDATE feature is not supported yet.");
}
- String constraintName = generateConstraintName();
+ // All primary keys in a composite key should have the same constraint name. This
+ // is necessary because of HIVE-16603. See IMPALA-9188 for details.
+ if (constraintName == null) {
+ constraintName = generateConstraintName();
+ }
// Each column of a primary key definition will be an SQLPrimaryKey.
sqlPrimaryKeys_.add(new SQLPrimaryKey(getTblName().getDb(), getTbl(),
colDef.getColName(), keySeq++, constraintName, primaryKey_.enableCstr,