You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Lukáš Drbal <lu...@gmail.com> on 2019/02/08 15:32:00 UTC
Monthly calendar window in SQL DSL
Hi everyone,
Is there any way how to use monthly calendar window in SQL DSL? Maybe I
miss something but i don't see any way for this. Is that correct? Thanks.
Currently I can create window before running SQL transform and it's just a
basic test so i'm fine with:
collection.apply("monthly-window", Window.into(CalendarWindows.months(1)))
.apply(SqlTransform.query("<my query>"));
During my tests I also find probably one bug in SQL windowing (filed
BEAM-6632 [1]) and provide PR [2] with fix and related testCase. It will be
nice if someone can take a look on that PR.
Regards,
Lukas
---
[1] https://issues.apache.org/jira/browse/BEAM-6632
[2] https://github.com/apache/beam/pull/7785
Re: Monthly calendar window in SQL DSL
Posted by Rui Wang <ru...@google.com>.
Thanks for the fix of bug. Calcite internally saves milliseconds of the
time interval value into BigDecimal. 31 days = 2678400000 millis which
overflows because of integer type.
-Rui
On Fri, Feb 8, 2019 at 10:54 AM Kenneth Knowles <ke...@apache.org> wrote:
> Your approach is right. The SQL query will be applied to each window.
> Thanks for the report & fix of the bug, too!
> Kenn
>
> On Fri, Feb 8, 2019 at 7:32 AM Lukáš Drbal <lu...@gmail.com> wrote:
>
>> Hi everyone,
>>
>> Is there any way how to use monthly calendar window in SQL DSL? Maybe I
>> miss something but i don't see any way for this. Is that correct? Thanks.
>>
>> Currently I can create window before running SQL transform and it's just
>> a basic test so i'm fine with:
>> collection.apply("monthly-window", Window.into(CalendarWindows.months(1)))
>> .apply(SqlTransform.query("<my query>"));
>>
>>
>> During my tests I also find probably one bug in SQL windowing (filed
>> BEAM-6632 [1]) and provide PR [2] with fix and related testCase. It will be
>> nice if someone can take a look on that PR.
>>
>>
>> Regards,
>> Lukas
>>
>> ---
>> [1] https://issues.apache.org/jira/browse/BEAM-6632
>> [2] https://github.com/apache/beam/pull/7785
>>
>
Re: Monthly calendar window in SQL DSL
Posted by Kenneth Knowles <ke...@apache.org>.
Your approach is right. The SQL query will be applied to each window.
Thanks for the report & fix of the bug, too!
Kenn
On Fri, Feb 8, 2019 at 7:32 AM Lukáš Drbal <lu...@gmail.com> wrote:
> Hi everyone,
>
> Is there any way how to use monthly calendar window in SQL DSL? Maybe I
> miss something but i don't see any way for this. Is that correct? Thanks.
>
> Currently I can create window before running SQL transform and it's just a
> basic test so i'm fine with:
> collection.apply("monthly-window", Window.into(CalendarWindows.months(1)))
> .apply(SqlTransform.query("<my query>"));
>
>
> During my tests I also find probably one bug in SQL windowing (filed
> BEAM-6632 [1]) and provide PR [2] with fix and related testCase. It will be
> nice if someone can take a look on that PR.
>
>
> Regards,
> Lukas
>
> ---
> [1] https://issues.apache.org/jira/browse/BEAM-6632
> [2] https://github.com/apache/beam/pull/7785
>