You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Timo Walther <tw...@apache.org> on 2017/04/25 14:13:51 UTC
TUMBLE/HOP/SESSION_START/END do not resolve time field correctly
Hi all,
I'm working on integrating START and END for TUMBLE/HOP/SESSION in Flink
SQL with logical time indicator columns (e.g. rowtime, proctime). It
seems there is a bug in the resolution logic of SqlToRelConverter. Since
our feature freeze is next week and this feature should be part of Flink
1.3, it would be great if you can help me with at least a hint for a
quick fix.
The problem is as follows:
Input: MyTable(INTEGER a, VARCHAR b, BIGINT c, TIMESTAMP proctime,
TIMESTAMP rowtime)
SQL: SELECT COUNT(*), TUMBLE_START(rowtime, INTERVAL '15' MINUTE),
TUMBLE_END(rowtime, INTERVAL '15' MINUTE) FROM MyTable GROUP BY
TUMBLE(rowtime, INTERVAL '15' MINUTE)
Exception:
java.lang.RuntimeException: while converting
TUMBLE_START(`MyTable`.`rowtime`, INTERVAL '15' MINUTE)
at
org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:134)
at
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:61)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4415)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3783)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:137)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4317)
at
org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:2723)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:2541)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:654)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:616)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2951)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:552)
.....
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:131)
... 42 more
Caused by: java.lang.AssertionError
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.getRootField(SqlToRelConverter.java:4132)
at
org.apache.calcite.sql2rel.SqlToRelConverter.adjustInputRef(SqlToRelConverter.java:3446)
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3421)
at
org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:207)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4424)
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java
The tests in Calcite only cover the case where rowtime is at the
beginning of a row. Once rowtime is somewhere else, the indices are
messed up. I tried to debug it, but I'm stuck at
SqlToRelConverter#convertIdentifier().
Any help is very welcome.
Regards,
Timo
Re: TUMBLE/HOP/SESSION_START/END do not resolve time field correctly
Posted by Timo Walther <tw...@apache.org>.
I created CALCITE-1761 for this issue. If you wanna take a look at the
exception, this is my branch:
https://github.com/twalthr/flink/tree/FLINK-5884
The following test fails right now:
org.apache.flink.table.api.scala.stream.sql.WindowAggregateTest#testTumbleFunction
Once we have a fix for that we can copy the class to Flink until the
next Calcite release. We did that with other issues in the past, too.
Am 25/04/17 um 20:12 schrieb Timo Walther:
> Thanks for your quick response. Flink does not use the monotonicity
> property yet and we are are also not using the STREAM keyword. Could
> this be a problem?
>
>
> Am 25/04/17 um 19:39 schrieb Julian Hyde:
>> I've added a test case (and the test missed in CALCITE-1615) in
>> https://github.com/julianhyde/calcite/tree/xxxx-hop. I cannot
>> reproduce your problem. Please still log a jira case.
>>
>> On Tue, Apr 25, 2017 at 10:13 AM, Julian Hyde <jh...@apache.org> wrote:
>>> I just noticed that in
>>> https://issues.apache.org/jira/browse/CALCITE-1615 tests were added to
>>> SqlToRelConverterTest.xml but not to SqlToRelConverterTest.java. We've
>>> been running without tests.
>>>
>>> On Tue, Apr 25, 2017 at 10:01 AM, Julian Hyde <jh...@apache.org> wrote:
>>>> Can you log a bug please? I will help out if I can.
>>>>
>>>> When this is fixed, I presume you will need a Calcite release at the
>>>> appropriate time so that you can release Flink. Can you start a
>>>> separate email thread when you know that timing?
>>>>
>>>> On Tue, Apr 25, 2017 at 7:13 AM, Timo Walther <tw...@apache.org>
>>>> wrote:
>>>>> Hi all,
>>>>>
>>>>>
>>>>> I'm working on integrating START and END for TUMBLE/HOP/SESSION in
>>>>> Flink SQL
>>>>> with logical time indicator columns (e.g. rowtime, proctime). It
>>>>> seems there
>>>>> is a bug in the resolution logic of SqlToRelConverter. Since our
>>>>> feature
>>>>> freeze is next week and this feature should be part of Flink 1.3,
>>>>> it would
>>>>> be great if you can help me with at least a hint for a quick fix.
>>>>>
>>>>>
>>>>> The problem is as follows:
>>>>>
>>>>>
>>>>> Input: MyTable(INTEGER a, VARCHAR b, BIGINT c, TIMESTAMP proctime,
>>>>> TIMESTAMP
>>>>> rowtime)
>>>>>
>>>>> SQL: SELECT COUNT(*), TUMBLE_START(rowtime, INTERVAL '15' MINUTE),
>>>>> TUMBLE_END(rowtime, INTERVAL '15' MINUTE) FROM MyTable GROUP BY
>>>>> TUMBLE(rowtime, INTERVAL '15' MINUTE)
>>>>>
>>>>> Exception:
>>>>>
>>>>> java.lang.RuntimeException: while converting
>>>>> TUMBLE_START(`MyTable`.`rowtime`, INTERVAL '15' MINUTE)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:134)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:61)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4415)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3783)
>>>>>
>>>>> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:137)
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4317)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:2723)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:2541)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:654)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:616)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2951)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:552)
>>>>>
>>>>> .....
>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>
>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>> at
>>>>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:131)
>>>>>
>>>>> ... 42 more
>>>>> Caused by: java.lang.AssertionError
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.getRootField(SqlToRelConverter.java:4132)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.adjustInputRef(SqlToRelConverter.java:3446)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3421)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:207)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4424)
>>>>>
>>>>> at
>>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java
>>>>>
>>>>>
>>>>>
>>>>> The tests in Calcite only cover the case where rowtime is at the
>>>>> beginning
>>>>> of a row. Once rowtime is somewhere else, the indices are messed
>>>>> up. I tried
>>>>> to debug it, but I'm stuck at SqlToRelConverter#convertIdentifier().
>>>>>
>>>>>
>>>>> Any help is very welcome.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Timo
>>>>>
Re: TUMBLE/HOP/SESSION_START/END do not resolve time field correctly
Posted by Timo Walther <tw...@apache.org>.
Thanks for your quick response. Flink does not use the monotonicity
property yet and we are are also not using the STREAM keyword. Could
this be a problem?
Am 25/04/17 um 19:39 schrieb Julian Hyde:
> I've added a test case (and the test missed in CALCITE-1615) in
> https://github.com/julianhyde/calcite/tree/xxxx-hop. I cannot
> reproduce your problem. Please still log a jira case.
>
> On Tue, Apr 25, 2017 at 10:13 AM, Julian Hyde <jh...@apache.org> wrote:
>> I just noticed that in
>> https://issues.apache.org/jira/browse/CALCITE-1615 tests were added to
>> SqlToRelConverterTest.xml but not to SqlToRelConverterTest.java. We've
>> been running without tests.
>>
>> On Tue, Apr 25, 2017 at 10:01 AM, Julian Hyde <jh...@apache.org> wrote:
>>> Can you log a bug please? I will help out if I can.
>>>
>>> When this is fixed, I presume you will need a Calcite release at the
>>> appropriate time so that you can release Flink. Can you start a
>>> separate email thread when you know that timing?
>>>
>>> On Tue, Apr 25, 2017 at 7:13 AM, Timo Walther <tw...@apache.org> wrote:
>>>> Hi all,
>>>>
>>>>
>>>> I'm working on integrating START and END for TUMBLE/HOP/SESSION in Flink SQL
>>>> with logical time indicator columns (e.g. rowtime, proctime). It seems there
>>>> is a bug in the resolution logic of SqlToRelConverter. Since our feature
>>>> freeze is next week and this feature should be part of Flink 1.3, it would
>>>> be great if you can help me with at least a hint for a quick fix.
>>>>
>>>>
>>>> The problem is as follows:
>>>>
>>>>
>>>> Input: MyTable(INTEGER a, VARCHAR b, BIGINT c, TIMESTAMP proctime, TIMESTAMP
>>>> rowtime)
>>>>
>>>> SQL: SELECT COUNT(*), TUMBLE_START(rowtime, INTERVAL '15' MINUTE),
>>>> TUMBLE_END(rowtime, INTERVAL '15' MINUTE) FROM MyTable GROUP BY
>>>> TUMBLE(rowtime, INTERVAL '15' MINUTE)
>>>>
>>>> Exception:
>>>>
>>>> java.lang.RuntimeException: while converting
>>>> TUMBLE_START(`MyTable`.`rowtime`, INTERVAL '15' MINUTE)
>>>>
>>>> at
>>>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:134)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:61)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4415)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3783)
>>>> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:137)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4317)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:2723)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:2541)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:654)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:616)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2951)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:552)
>>>> .....
>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at
>>>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:131)
>>>> ... 42 more
>>>> Caused by: java.lang.AssertionError
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.getRootField(SqlToRelConverter.java:4132)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.adjustInputRef(SqlToRelConverter.java:3446)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3421)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:207)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4424)
>>>> at
>>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java
>>>>
>>>>
>>>> The tests in Calcite only cover the case where rowtime is at the beginning
>>>> of a row. Once rowtime is somewhere else, the indices are messed up. I tried
>>>> to debug it, but I'm stuck at SqlToRelConverter#convertIdentifier().
>>>>
>>>>
>>>> Any help is very welcome.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Timo
>>>>
Re: TUMBLE/HOP/SESSION_START/END do not resolve time field correctly
Posted by Julian Hyde <jh...@apache.org>.
I've added a test case (and the test missed in CALCITE-1615) in
https://github.com/julianhyde/calcite/tree/xxxx-hop. I cannot
reproduce your problem. Please still log a jira case.
On Tue, Apr 25, 2017 at 10:13 AM, Julian Hyde <jh...@apache.org> wrote:
> I just noticed that in
> https://issues.apache.org/jira/browse/CALCITE-1615 tests were added to
> SqlToRelConverterTest.xml but not to SqlToRelConverterTest.java. We've
> been running without tests.
>
> On Tue, Apr 25, 2017 at 10:01 AM, Julian Hyde <jh...@apache.org> wrote:
>> Can you log a bug please? I will help out if I can.
>>
>> When this is fixed, I presume you will need a Calcite release at the
>> appropriate time so that you can release Flink. Can you start a
>> separate email thread when you know that timing?
>>
>> On Tue, Apr 25, 2017 at 7:13 AM, Timo Walther <tw...@apache.org> wrote:
>>> Hi all,
>>>
>>>
>>> I'm working on integrating START and END for TUMBLE/HOP/SESSION in Flink SQL
>>> with logical time indicator columns (e.g. rowtime, proctime). It seems there
>>> is a bug in the resolution logic of SqlToRelConverter. Since our feature
>>> freeze is next week and this feature should be part of Flink 1.3, it would
>>> be great if you can help me with at least a hint for a quick fix.
>>>
>>>
>>> The problem is as follows:
>>>
>>>
>>> Input: MyTable(INTEGER a, VARCHAR b, BIGINT c, TIMESTAMP proctime, TIMESTAMP
>>> rowtime)
>>>
>>> SQL: SELECT COUNT(*), TUMBLE_START(rowtime, INTERVAL '15' MINUTE),
>>> TUMBLE_END(rowtime, INTERVAL '15' MINUTE) FROM MyTable GROUP BY
>>> TUMBLE(rowtime, INTERVAL '15' MINUTE)
>>>
>>> Exception:
>>>
>>> java.lang.RuntimeException: while converting
>>> TUMBLE_START(`MyTable`.`rowtime`, INTERVAL '15' MINUTE)
>>>
>>> at
>>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:134)
>>> at
>>> org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:61)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4415)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3783)
>>> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:137)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4317)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:2723)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:2541)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:654)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:616)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2951)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:552)
>>> .....
>>> Caused by: java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:131)
>>> ... 42 more
>>> Caused by: java.lang.AssertionError
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.getRootField(SqlToRelConverter.java:4132)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.adjustInputRef(SqlToRelConverter.java:3446)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3421)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:207)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4424)
>>> at
>>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java
>>>
>>>
>>> The tests in Calcite only cover the case where rowtime is at the beginning
>>> of a row. Once rowtime is somewhere else, the indices are messed up. I tried
>>> to debug it, but I'm stuck at SqlToRelConverter#convertIdentifier().
>>>
>>>
>>> Any help is very welcome.
>>>
>>>
>>> Regards,
>>>
>>> Timo
>>>
Re: TUMBLE/HOP/SESSION_START/END do not resolve time field correctly
Posted by Julian Hyde <jh...@apache.org>.
I just noticed that in
https://issues.apache.org/jira/browse/CALCITE-1615 tests were added to
SqlToRelConverterTest.xml but not to SqlToRelConverterTest.java. We've
been running without tests.
On Tue, Apr 25, 2017 at 10:01 AM, Julian Hyde <jh...@apache.org> wrote:
> Can you log a bug please? I will help out if I can.
>
> When this is fixed, I presume you will need a Calcite release at the
> appropriate time so that you can release Flink. Can you start a
> separate email thread when you know that timing?
>
> On Tue, Apr 25, 2017 at 7:13 AM, Timo Walther <tw...@apache.org> wrote:
>> Hi all,
>>
>>
>> I'm working on integrating START and END for TUMBLE/HOP/SESSION in Flink SQL
>> with logical time indicator columns (e.g. rowtime, proctime). It seems there
>> is a bug in the resolution logic of SqlToRelConverter. Since our feature
>> freeze is next week and this feature should be part of Flink 1.3, it would
>> be great if you can help me with at least a hint for a quick fix.
>>
>>
>> The problem is as follows:
>>
>>
>> Input: MyTable(INTEGER a, VARCHAR b, BIGINT c, TIMESTAMP proctime, TIMESTAMP
>> rowtime)
>>
>> SQL: SELECT COUNT(*), TUMBLE_START(rowtime, INTERVAL '15' MINUTE),
>> TUMBLE_END(rowtime, INTERVAL '15' MINUTE) FROM MyTable GROUP BY
>> TUMBLE(rowtime, INTERVAL '15' MINUTE)
>>
>> Exception:
>>
>> java.lang.RuntimeException: while converting
>> TUMBLE_START(`MyTable`.`rowtime`, INTERVAL '15' MINUTE)
>>
>> at
>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:134)
>> at
>> org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:61)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4415)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3783)
>> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:137)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4317)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:2723)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:2541)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:654)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:616)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2951)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:552)
>> .....
>> Caused by: java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:131)
>> ... 42 more
>> Caused by: java.lang.AssertionError
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.getRootField(SqlToRelConverter.java:4132)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.adjustInputRef(SqlToRelConverter.java:3446)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3421)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:207)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4424)
>> at
>> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java
>>
>>
>> The tests in Calcite only cover the case where rowtime is at the beginning
>> of a row. Once rowtime is somewhere else, the indices are messed up. I tried
>> to debug it, but I'm stuck at SqlToRelConverter#convertIdentifier().
>>
>>
>> Any help is very welcome.
>>
>>
>> Regards,
>>
>> Timo
>>
Re: TUMBLE/HOP/SESSION_START/END do not resolve time field correctly
Posted by Julian Hyde <jh...@apache.org>.
Can you log a bug please? I will help out if I can.
When this is fixed, I presume you will need a Calcite release at the
appropriate time so that you can release Flink. Can you start a
separate email thread when you know that timing?
On Tue, Apr 25, 2017 at 7:13 AM, Timo Walther <tw...@apache.org> wrote:
> Hi all,
>
>
> I'm working on integrating START and END for TUMBLE/HOP/SESSION in Flink SQL
> with logical time indicator columns (e.g. rowtime, proctime). It seems there
> is a bug in the resolution logic of SqlToRelConverter. Since our feature
> freeze is next week and this feature should be part of Flink 1.3, it would
> be great if you can help me with at least a hint for a quick fix.
>
>
> The problem is as follows:
>
>
> Input: MyTable(INTEGER a, VARCHAR b, BIGINT c, TIMESTAMP proctime, TIMESTAMP
> rowtime)
>
> SQL: SELECT COUNT(*), TUMBLE_START(rowtime, INTERVAL '15' MINUTE),
> TUMBLE_END(rowtime, INTERVAL '15' MINUTE) FROM MyTable GROUP BY
> TUMBLE(rowtime, INTERVAL '15' MINUTE)
>
> Exception:
>
> java.lang.RuntimeException: while converting
> TUMBLE_START(`MyTable`.`rowtime`, INTERVAL '15' MINUTE)
>
> at
> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:134)
> at
> org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:61)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4415)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3783)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:137)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4317)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:2723)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:2541)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:654)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:616)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2951)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:552)
> .....
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.calcite.sql2rel.ReflectiveConvertletTable$2.convertCall(ReflectiveConvertletTable.java:131)
> ... 42 more
> Caused by: java.lang.AssertionError
> at
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.getRootField(SqlToRelConverter.java:4132)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.adjustInputRef(SqlToRelConverter.java:3446)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3421)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:207)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4424)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java
>
>
> The tests in Calcite only cover the case where rowtime is at the beginning
> of a row. Once rowtime is somewhere else, the indices are messed up. I tried
> to debug it, but I'm stuck at SqlToRelConverter#convertIdentifier().
>
>
> Any help is very welcome.
>
>
> Regards,
>
> Timo
>