You are viewing a plain text version of this content. The canonical link for it is here.
Posted to codereview@trafodion.apache.org by zellerh <gi...@git.apache.org> on 2016/12/20 22:53:32 UTC
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
GitHub user zellerh opened a pull request:
https://github.com/apache/incubator-trafodion/pull/882
TMUDF fixes and related changes
This PR has fixes for 5 JIRAs in a set of 4 commits (sorry, two of the fixes had overlaps, so I bundled them into a single commit). It may be easier to review each commit individually, since they are logically independent.
**Commit 1:**
[TRAFODION-2382] No plan produced when joining two TMUDFs
A simple join query resulted in no plan. One of the reasons is
the way we handle operator types for TMUDFs. Changed this so that
the operator type indicates the arity of the operator.
**Commit 2:**
[TRAFODION-2399] Syntax error when loading from salted table
This was caused by methods to generate an SQL string literal
from a constant value. In some cases it repeated the character
set name introducer.
**Commit 3:**
[TRAFODION-2382] Support tinyint and boolean in tmudfs
Adding support for these new data types in TMUDFs.
- core/sql/generator/LmExpr.cpp
- core/sql/optimizer/UdfDllInteraction.cpp
- core/sql/sqludr/sqludr.cpp
- core/sql/sqludr/sqludr.h
- core/sql/src/main/java/org/trafodion/sql/udr/TupleInfo.java
- core/sql/src/main/java/org/trafodion/sql/udr/TypeInfo.java
- core/sql/src/main/java/org/trafodion/sql/udr/UDRInvocationInfo.java
- core/sql/regress/udr/TEST001
- core/sql/regress/udr/EXPECTED001
TRAFODION-2392 avoid costly sort in reducer TMUDFs
Add a REDUCER_NC function type. The only difference is in the
createContextForAChild function, where we don't generate a required
arrangement for UDFs of this type.
- core/sql/optimizer/OptPhysRelExpr.cpp
- core/sql/sqludr/sqludr.cpp
- core/sql/sqludr/sqludr.h
- core/sql/src/main/java/org/trafodion/sql/udr/UDR.java
- core/sql/src/main/java/org/trafodion/sql/udr/UDRInvocationInfo.java
Misc changes, not related to a JIRA:
- core/sql/regress/compGeneral/EXPECTED071
- core/sql/regress/compGeneral/TEST071
**Commit 4:**
[TRAFODION-2400] wrong results for passthru cols
Wrong results were returned for passthru columns in TMUDFs
when we had equal predicates on the source table. This was
related to VEG rewrite, where the rewritten expression had
the wrong type.
Add a cast node when the type of an expression changes after VEG rewrite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zellerh/incubator-trafodion cses_1026
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/882.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #882
----
commit 91a809b1b7494476e1fb34696ff6496c3b492015
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:23:52Z
[TRAFODION-2382] No plan produced when joining two TMUDFs
A simple join query resulted in no plan. One of the reasons is
the way we handle operator types for TMUDFs. Changed this so that
the operator type indicates the arity of the operator.
commit 1417baf4d15b45dd390c6ff1bfd6078cdf137f53
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:26:13Z
[TRAFODION-2399] Syntax error when loading from salted table
This was caused by methods to generate an SQL string literal
from a constant value. In some cases it repeated the character
set name introducer.
commit 2faae457be9928909a9e84aa5bf9979944324600
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:39:58Z
[TRAFODION-2382] Support tinyint and boolean in tmudfs
Adding support for these new data types in TMUDFs.
[TRAFODION-2392] Avoid costly sort in reducer TMUDFs
Adding a new function type for TMUDFs that can avoid
a costly sort for highly reducing UDFs that implement
an internal hash table (or equivalent).
commit 5097291082615ac4f05b302e3cc5478fb7f3dfb7
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:40:55Z
[TRAFODION-2400] wrong results for passthru cols
Wrong results were returned for passthru columns in TMUDFs
when we had equal predicates on the source table. This was
related to VEG rewrite, where the rewritten expression had
the wrong type.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by sureshsubbiah <gi...@git.apache.org>.
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93643561
--- Diff: core/sql/src/main/java/org/trafodion/sql/udr/TupleInfo.java ---
@@ -1079,19 +1230,48 @@ else if (t.getByteLength() == 8)
row_.putLong(t.getDataOffset(), val);
break;
+ case TINYINT_UNSIGNED:
+ if (val < 0 || val > Byte.MAX_VALUE)
+ if (val > (2 * Byte.MAX_VALUE + 1))
+ throw new UDRException(
+ 38900,
+ "Overflow/underflow when assigning to TINYINT UNSIGNED type");
+ else
+ // use the signed value that has the same bit
+ // pattern as the desired unsigned value, since
+ // Java doesn't support "unsigned" basic types
+ val = val + Byte.MIN_VALUE + Byte.MIN_VALUE;
--- End diff --
I suppose this is basic arithmetic. If the comment could please include an example that will help. Is Byte.MIN_VALUE = -127. If yes, how will adding it twice to val, give the same bit pattern as the unsigned type? I am quite sure there is no defect here, but that I am not understanding something obvious. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by zellerh <gi...@git.apache.org>.
Github user zellerh commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93665587
--- Diff: core/sql/src/main/java/org/trafodion/sql/udr/TupleInfo.java ---
@@ -1079,19 +1230,48 @@ else if (t.getByteLength() == 8)
row_.putLong(t.getDataOffset(), val);
break;
+ case TINYINT_UNSIGNED:
+ if (val < 0 || val > Byte.MAX_VALUE)
+ if (val > (2 * Byte.MAX_VALUE + 1))
+ throw new UDRException(
+ 38900,
+ "Overflow/underflow when assigning to TINYINT UNSIGNED type");
+ else
+ // use the signed value that has the same bit
+ // pattern as the desired unsigned value, since
+ // Java doesn't support "unsigned" basic types
+ val = val + Byte.MIN_VALUE + Byte.MIN_VALUE;
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by zellerh <gi...@git.apache.org>.
Github user zellerh commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93665561
--- Diff: core/sql/src/main/java/org/trafodion/sql/udr/TupleInfo.java ---
@@ -1079,19 +1230,48 @@ else if (t.getByteLength() == 8)
row_.putLong(t.getDataOffset(), val);
break;
+ case TINYINT_UNSIGNED:
+ if (val < 0 || val > Byte.MAX_VALUE)
+ if (val > (2 * Byte.MAX_VALUE + 1))
+ throw new UDRException(
+ 38900,
+ "Overflow/underflow when assigning to TINYINT UNSIGNED type");
+ else
--- End diff --
Thanks for finding this! Fixed in all three cases.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by zellerh <gi...@git.apache.org>.
Github user zellerh commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93661838
--- Diff: core/sql/optimizer/RelRoutine.cpp ---
@@ -1027,7 +1052,7 @@ OperatorTypeEnum PredefinedTableMappingFunction::nameIsAPredefinedTMF(const Corr
const NAString &funcName = qualName.getObjectName();
--- End diff --
Maybe small, but still a valid point! I stumbled over this while re-reading the code. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-trafodion/pull/882
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by sureshsubbiah <gi...@git.apache.org>.
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93642938
--- Diff: core/sql/src/main/java/org/trafodion/sql/udr/TupleInfo.java ---
@@ -1079,19 +1230,48 @@ else if (t.getByteLength() == 8)
row_.putLong(t.getDataOffset(), val);
break;
+ case TINYINT_UNSIGNED:
+ if (val < 0 || val > Byte.MAX_VALUE)
+ if (val > (2 * Byte.MAX_VALUE + 1))
+ throw new UDRException(
+ 38900,
+ "Overflow/underflow when assigning to TINYINT UNSIGNED type");
+ else
--- End diff --
Will this handle underflow correctly? If val < 0 is an exception supposed to be thrown?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by zellerh <gi...@git.apache.org>.
Github user zellerh commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93661429
--- Diff: core/sql/generator/LmExpr.cpp ---
@@ -362,7 +362,7 @@ LmExprResult CreateLmOutputExpr(const NAType &formalType,
// specified type t to be converted to/from C strings. The only
// SQL types that do not need to be converted to C strings are:
//
-// INT, SMALLINT, LARGEINT, FLOAT, REAL, DOUBLE PRECISION
+// INT, SMALLINT, LARGEINT, FLOAT, REAL, DOUBLE PRECISION, BOOLEAN
--- End diff --
Yes, good point, I'll add tinyint.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by sureshsubbiah <gi...@git.apache.org>.
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93637494
--- Diff: core/sql/generator/LmExpr.cpp ---
@@ -362,7 +362,7 @@ LmExprResult CreateLmOutputExpr(const NAType &formalType,
// specified type t to be converted to/from C strings. The only
// SQL types that do not need to be converted to C strings are:
//
-// INT, SMALLINT, LARGEINT, FLOAT, REAL, DOUBLE PRECISION
+// INT, SMALLINT, LARGEINT, FLOAT, REAL, DOUBLE PRECISION, BOOLEAN
--- End diff --
Does Tinyint belong to this set of types? I suppose the answer is No, but thought it safer to check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #882: TMUDF fixes and related changes
Posted by sureshsubbiah <gi...@git.apache.org>.
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/882#discussion_r93589838
--- Diff: core/sql/optimizer/RelRoutine.cpp ---
@@ -1027,7 +1052,7 @@ OperatorTypeEnum PredefinedTableMappingFunction::nameIsAPredefinedTMF(const Corr
const NAString &funcName = qualName.getObjectName();
--- End diff --
This comment is for line 1047. It must be the smallest nit recorded. The comment could be changed to REL_ANY_TABLE_MAPPING_UDF from REL_TABLE_MAPPING_UDF.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---