You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/12/20 22:53:58 UTC
[jira] [Commented] (TRAFODION-2382) No plan produced when joining
two TMUDFs
[ https://issues.apache.org/jira/browse/TRAFODION-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765485#comment-15765485 ]
ASF GitHub Bot commented on TRAFODION-2382:
-------------------------------------------
GitHub user zellerh opened a pull request:
https://github.com/apache/incubator-trafodion/pull/882
TMUDF fixes and related changes
This PR has fixes for 5 JIRAs in a set of 4 commits (sorry, two of the fixes had overlaps, so I bundled them into a single commit). It may be easier to review each commit individually, since they are logically independent.
**Commit 1:**
[TRAFODION-2382] No plan produced when joining two TMUDFs
A simple join query resulted in no plan. One of the reasons is
the way we handle operator types for TMUDFs. Changed this so that
the operator type indicates the arity of the operator.
**Commit 2:**
[TRAFODION-2399] Syntax error when loading from salted table
This was caused by methods to generate an SQL string literal
from a constant value. In some cases it repeated the character
set name introducer.
**Commit 3:**
[TRAFODION-2382] Support tinyint and boolean in tmudfs
Adding support for these new data types in TMUDFs.
- core/sql/generator/LmExpr.cpp
- core/sql/optimizer/UdfDllInteraction.cpp
- core/sql/sqludr/sqludr.cpp
- core/sql/sqludr/sqludr.h
- core/sql/src/main/java/org/trafodion/sql/udr/TupleInfo.java
- core/sql/src/main/java/org/trafodion/sql/udr/TypeInfo.java
- core/sql/src/main/java/org/trafodion/sql/udr/UDRInvocationInfo.java
- core/sql/regress/udr/TEST001
- core/sql/regress/udr/EXPECTED001
TRAFODION-2392 avoid costly sort in reducer TMUDFs
Add a REDUCER_NC function type. The only difference is in the
createContextForAChild function, where we don't generate a required
arrangement for UDFs of this type.
- core/sql/optimizer/OptPhysRelExpr.cpp
- core/sql/sqludr/sqludr.cpp
- core/sql/sqludr/sqludr.h
- core/sql/src/main/java/org/trafodion/sql/udr/UDR.java
- core/sql/src/main/java/org/trafodion/sql/udr/UDRInvocationInfo.java
Misc changes, not related to a JIRA:
- core/sql/regress/compGeneral/EXPECTED071
- core/sql/regress/compGeneral/TEST071
**Commit 4:**
[TRAFODION-2400] wrong results for passthru cols
Wrong results were returned for passthru columns in TMUDFs
when we had equal predicates on the source table. This was
related to VEG rewrite, where the rewritten expression had
the wrong type.
Add a cast node when the type of an expression changes after VEG rewrite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zellerh/incubator-trafodion cses_1026
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/882.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #882
----
commit 91a809b1b7494476e1fb34696ff6496c3b492015
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:23:52Z
[TRAFODION-2382] No plan produced when joining two TMUDFs
A simple join query resulted in no plan. One of the reasons is
the way we handle operator types for TMUDFs. Changed this so that
the operator type indicates the arity of the operator.
commit 1417baf4d15b45dd390c6ff1bfd6078cdf137f53
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:26:13Z
[TRAFODION-2399] Syntax error when loading from salted table
This was caused by methods to generate an SQL string literal
from a constant value. In some cases it repeated the character
set name introducer.
commit 2faae457be9928909a9e84aa5bf9979944324600
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:39:58Z
[TRAFODION-2382] Support tinyint and boolean in tmudfs
Adding support for these new data types in TMUDFs.
[TRAFODION-2392] Avoid costly sort in reducer TMUDFs
Adding a new function type for TMUDFs that can avoid
a costly sort for highly reducing UDFs that implement
an internal hash table (or equivalent).
commit 5097291082615ac4f05b302e3cc5478fb7f3dfb7
Author: Hans Zeller <hz...@apache.org>
Date: 2016-12-20T22:40:55Z
[TRAFODION-2400] wrong results for passthru cols
Wrong results were returned for passthru columns in TMUDFs
when we had equal predicates on the source table. This was
related to VEG rewrite, where the rewritten expression had
the wrong type.
----
> No plan produced when joining two TMUDFs
> ----------------------------------------
>
> Key: TRAFODION-2382
> URL: https://issues.apache.org/jira/browse/TRAFODION-2382
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.1-incubating
> Environment: any
> Reporter: Hans Zeller
> Assignee: Hans Zeller
>
> This problem was reported by Gunnar Tapper and it uncovered three separate issues:
> # The max cardinality for some TMUDFs is set to -1 and this causes the hash join implementation rule to trigger a condition that disables a hash join. This is intended for situations where the left/outer table has at most 1 row, which is not the case here.
> # The merge join rule doesn't fire, because the CAST functions used here are not considered order-preserving expressions. This problem is not addressed as part of this JIRA.
> # The nested join rule produces a filter, but we fail to push the filter predicate into the TMUDF, because the code uses the operator type to determine its arity, and the TMUDF uses the same operator type for different arities.
> In the end, none of the three join algorithms can be applied, and we fail to produce a plan.
> Here is how to reproduce the problem:
> Put this Java code into file join_udr.java:
> {code:java}
> import org.trafodion.sql.udr.*;
> class join_udf extends UDR {
> public join_udf() {}
>
> @Override
> public void describeParamsAndColumns(UDRInvocationInfo info)
> throws UDRException
> {
> // 1st param is vchar size desired
> int vSize = info.par().getInt(0);
> // this param needed when call is exec'd, but not info.out()'d
> info.addFormalParameter(info.par().getColumn(0));
> // For each following param, create an output column
> // Params are json tag names - convert these to SQL column names
> // For output, cannot distinguish data types so everything is VARCHAR
>
> for (int i = 1; i < info.par().getNumColumns(); i++) {
> // make formal param
> info.addFormalParameter(info.par().getColumn(i));
> // use input param string to create output param
> info.out().addVarCharColumn(info.par().getString(i), vSize, true);
> } // for
>
> // set as Mapper so reading can be parallelized
> info.setFuncType(UDRInvocationInfo.FuncType.MAPPER);
> }
>
> @Override
> public void processData(UDRInvocationInfo info,
> UDRPlanInfo plan)
> throws UDRException
> {
> // do nothing
> } // end processData
>
> }
> {code}
> Compile the code:
> {code}
> javac join_udf.java
> jar cvf join_udf.jar join_udf.class
> {code}
> Create the UDF and a source table, then try to compile the query (stay in the same directory and use the sqlci tool):
> {code:sql}
> drop function join_udf;
> drop library join_udf_lib;
> create library join_udf_lib file
> 'join_udf.jar';
> create table_mapping function join_udf()
> external name 'join_udf'
> language java
> library join_udf_lib;
> -- disable semijoin to join transformation, some more complex situations
> -- show this error without this CQD, but we are using it here to keep the
> -- test case simple
> cqd SEMIJOIN_TO_INNERJOIN_TRANSFORMATION 'off';
> create table join_udf_src(a integer);
> prepare s from
> --display
> select a,b
> from
> ( select
> -- Convert the string data to SQL data types.
> cast( a as char(4) ) as a
> , cast( b as date ) as b
> from udf
> ( join_udf(
> table( select * from join_udf_src )
> , 257
> , 'A'
> , 'B'
> )
> )
> )
> where ( a,b ) in
> ( select a,b
> from
> ( select
> cast( a as char(4) ) as a
> , cast( b as date ) as b
> from udf
> ( join_udf(
> 257
> , 'A'
> , 'B'
> )
> )
> )
> where b between date '2016-01-01' and date '2016-02-28'
> )
> ;
> {code}
> This will produce the following error:
> {code}
> *** ERROR[2235] Compiler Internal Error: Pass one skipped, but cannot produce a plan in pass two, originated from file ../optimizer/opt.cpp at line xxxx.
> *** ERROR[8822] The statement was not prepared.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)