incubator-impala git commit: IMPALA-4100, 4112: Qgen: Replace EXTRACT UDF + IS [NOT] DISTINCT FROM in HiveSqlWriter

[ta...@apache.org]

Repository: incubator-impala
Updated Branches:
  refs/heads/master f9ac593ff -> 13fc14d8a


IMPALA-4100,4112: Qgen: Replace EXTRACT UDF + IS [NOT] DISTINCT FROM in HiveSqlWriter

IMPALA-4100:

* Postgres and Impala support EXTRACT([field] from [date-type]), but Hive doesn't
* Hive has other UDFs that perform the same function, but they have different names
* This commit modifies the HiveSqlWriter to use the corresponding Hive functions

IMPALA-4112:

* Postgres and Impala support IS [NOT] DISTINCT FROM clauses as a null safe equals
* Hive doesn't support this clause, but has a null safe equals operator: <=>
* This commit modifies the HiveSqlWriter to use <=> instead of IS [NOT] DISTINCT FROM

Testing:

* This commit only modifies the HiveSqlWriter, so no testing against Impala was done
* Tested locally against Hive

Change-Id: I3922ca61af59ecd2899c911b1a03e11ab5c26e11
Reviewed-on: http://gerrit.cloudera.org:8080/4357
Reviewed-by: Michael Brown <mi...@cloudera.com>
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Tim Armstrong <ta...@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/13fc14d8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/13fc14d8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/13fc14d8

Branch: refs/heads/master
Commit: 13fc14d8a9c2d80392b706d3c3396b4abe3e12ad
Parents: f9ac593
Author: Sahil Takiar <st...@cloudera.com>
Authored: Thu Sep 8 17:07:16 2016 -0700
Committer: Tim Armstrong <ta...@cloudera.com>
Committed: Mon Sep 12 17:07:29 2016 +0000

----------------------------------------------------------------------
 tests/comparison/model_translator.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/13fc14d8/tests/comparison/model_translator.py
----------------------------------------------------------------------
diff --git a/tests/comparison/model_translator.py b/tests/comparison/model_translator.py
index 9c693a3..202b5ea 100644
--- a/tests/comparison/model_translator.py
+++ b/tests/comparison/model_translator.py
@@ -456,6 +456,15 @@ class HiveSqlWriter(SqlWriter):
 
   DIALECT = 'HIVE'
 
+  def __init__(self, *args, **kwargs):
+    super(HiveSqlWriter, self).__init__(*args, **kwargs)
+
+    self.operator_funcs.update({
+      'IsNotDistinctFrom': '({0}) <=> ({1})',
+      'IsNotDistinctFromOp': '({0}) <=> ({1})',
+      'IsDistinctFrom': 'NOT(({0}) <=> ({1}))'
+    })
+
   # Hive greatest UDF is strict on type equality
   # Hive Profile already restricts to signatures with the same types,
   # but sometimes expression with UDF's like 'count'
@@ -530,6 +539,24 @@ class HiveSqlWriter(SqlWriter):
           options.append('rows unbounded preceding')
     return sql + ' '.join(options) + ')'
 
+  def _write_extract_year(self, func):
+    return 'YEAR(%s)' % self._write(func.args[0])
+
+  def _write_extract_month(self, func):
+    return 'MONTH(%s)' % self._write(func.args[0])
+
+  def _write_extract_day(self, func):
+    return 'DAY(%s)' % self._write(func.args[0])
+
+  def _write_extract_hour(self, func):
+    return 'HOUR(%s)' % self._write(func.args[0])
+
+  def _write_extract_minute(self, func):
+    return 'MINUTE(%s)' % self._write(func.args[0])
+
+  def _write_extract_second(self, func):
+    return 'SECOND(%s)' % self._write(func.args[0])
+
 
 class PostgresqlSqlWriter(SqlWriter):