[jira] [Commented] (FLINK-3724) testJoinPushThroughJoin fails for TableConfigMode=EFFICIENT

["Fabian Hueske (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234813#comment-15234813 ] 

Fabian Hueske commented on FLINK-3724:
--------------------------------------

This is another issue that requires special case handling for the efficient type mode. I am starting to wonder whether the possible performance gains justify the added code complexity in the program construction and code generation code. 

To resolve this concrete issue, I propose to not allow primitives as types in Table programs but to wrap them in {{Tuple1}} instances. That way we can treat them as composite types and have a common way to handle keys. The added overhead should be insignificant. Serialization and comparisons performance should not decrease (Tuple serializers do not add metadata) and {{Tuple1}} instances should be reused by the generated code.

In the long run, we can also think about to only support {{Row}} and provide nullable and non-nullable serializers and comparators for the same data type. But this is a separate issue and should not be discussed here.

> testJoinPushThroughJoin fails for TableConfigMode=EFFICIENT
> -----------------------------------------------------------
>
>                 Key: FLINK-3724
>                 URL: https://issues.apache.org/jira/browse/FLINK-3724
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API, Tests
>    Affects Versions: 1.1.0
>            Reporter: Vasia Kalavri
>
> Efficient config mode in the Table API also makes the {{JoinITCase.testJoinPushThroughJoin}} test case fail, with the following error:
> {code}
> org.apache.flink.api.common.InvalidProgramException: Specifying keys via field positions is only valid for tuple data types. Type: Integer
> 	at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:217)
> 	at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:208)
> 	at org.apache.flink.api.java.operators.JoinOperator$JoinOperatorSets.where(JoinOperator.java:868)
> 	at org.apache.flink.api.table.plan.nodes.dataset.DataSetJoin.translateToPlan(DataSetJoin.scala:199)
> 	at org.apache.flink.api.table.plan.nodes.dataset.DataSetCalc.translateToPlan(DataSetCalc.scala:95)
> 	at org.apache.flink.api.table.plan.nodes.dataset.DataSetJoin.translateToPlan(DataSetJoin.scala:159)
> 	at org.apache.flink.api.table.plan.nodes.dataset.DataSetCalc.translateToPlan(DataSetCalc.scala:95)
> 	at org.apache.flink.api.java.table.JavaBatchTranslator.translate(JavaBatchTranslator.scala:90)
> 	at org.apache.flink.api.scala.table.ScalaBatchTranslator.translate(ScalaBatchTranslator.scala:51)
> 	at org.apache.flink.api.scala.table.TableConversions.toDataSet(TableConversions.scala:43)
> 	at org.apache.flink.api.scala.table.test.JoinITCase.testJoinPushThroughJoin(JoinITCase.scala:201)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.junit.runners.Suite.runChild(Suite.java:127)
> 	at org.junit.runners.Suite.runChild(Suite.java:26)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)