How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Longdexin <27...@qq.com>]

From my point of view, the value of NOW() function in SQL is certain by the
time when the streaming app is launched and will not change with the process
time. However, as a new Flink user, I'm not so sure of that. By the way, if
my attemp is to keep the time logic to update all the time, what should I
do?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Jark Wu <im...@gmail.com>]

issue created: https://issues.apache.org/jira/browse/FLINK-19861



On Wed, 28 Oct 2020 at 11:00, Danny Chan <da...@apache.org> wrote:

> Our behavior also conflicts with the SQL standard, we should also mention
> this in the document.
>
> Till Rohrmann <tr...@apache.org> 于2020年10月27日周二 下午10:37写道：
>
>> Thanks for the clarification. This improvement would be helpful, I
>> believe.
>>
>> Cheers,
>> Till
>>
>> On Tue, Oct 27, 2020 at 1:19 PM Jark Wu <im...@gmail.com> wrote:
>>
>>> Hi Till,
>>>
>>> The documentation mentions that "this function is not deterministic"
>>> where the "not deterministic" means the value of this function is not
>>> deterministic for every record.
>>> However, this is not very clear for users. I think we can improve the
>>> documentation.
>>>
>>> Best,
>>> Jark
>>>
>>> On Tue, 27 Oct 2020 at 15:59, Till Rohrmann <tr...@apache.org>
>>> wrote:
>>>
>>>> Quick question Jark: Is this difference in behaviour documented? I
>>>> couldn't find it in the docs.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Tue, Oct 27, 2020 at 7:30 AM Jark Wu <im...@gmail.com> wrote:
>>>>
>>>>> Hi Longdexin,
>>>>>
>>>>> In traditional batch sql, NOW() is executed and determined before the
>>>>> job is submitted and will not change for every processed record.
>>>>> However, this doesn't make much sense in streaming sql, therefore,
>>>>> NOW() function in Flink is executed for every record.
>>>>>
>>>>> Best,
>>>>> Jark
>>>>>
>>>>> On Fri, 23 Oct 2020 at 16:30, Till Rohrmann <tr...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Longdexin,
>>>>>>
>>>>>> thanks for reaching out to the Flink community. I am pulling in Jark
>>>>>> who might be able to help you with this question.
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>>>>> On Thu, Oct 22, 2020 at 2:56 PM Longdexin <27...@qq.com> wrote:
>>>>>>
>>>>>>> From my point of view, the value of NOW() function in SQL is certain
>>>>>>> by the
>>>>>>> time when the streaming app is launched and will not change with the
>>>>>>> process
>>>>>>> time. However, as a new Flink user, I'm not so sure of that. By the
>>>>>>> way, if
>>>>>>> my attemp is to keep the time logic to update all the time, what
>>>>>>> should I
>>>>>>> do?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Sent from:
>>>>>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>>>>>>
>>>>>>

Re: How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Danny Chan <da...@apache.org>]

Our behavior also conflicts with the SQL standard, we should also mention
this in the document.

Till Rohrmann <tr...@apache.org> 于2020年10月27日周二 下午10:37写道：

> Thanks for the clarification. This improvement would be helpful, I believe.
>
> Cheers,
> Till
>
> On Tue, Oct 27, 2020 at 1:19 PM Jark Wu <im...@gmail.com> wrote:
>
>> Hi Till,
>>
>> The documentation mentions that "this function is not deterministic"
>> where the "not deterministic" means the value of this function is not
>> deterministic for every record.
>> However, this is not very clear for users. I think we can improve the
>> documentation.
>>
>> Best,
>> Jark
>>
>> On Tue, 27 Oct 2020 at 15:59, Till Rohrmann <tr...@apache.org> wrote:
>>
>>> Quick question Jark: Is this difference in behaviour documented? I
>>> couldn't find it in the docs.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Tue, Oct 27, 2020 at 7:30 AM Jark Wu <im...@gmail.com> wrote:
>>>
>>>> Hi Longdexin,
>>>>
>>>> In traditional batch sql, NOW() is executed and determined before the
>>>> job is submitted and will not change for every processed record.
>>>> However, this doesn't make much sense in streaming sql, therefore,
>>>> NOW() function in Flink is executed for every record.
>>>>
>>>> Best,
>>>> Jark
>>>>
>>>> On Fri, 23 Oct 2020 at 16:30, Till Rohrmann <tr...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Longdexin,
>>>>>
>>>>> thanks for reaching out to the Flink community. I am pulling in Jark
>>>>> who might be able to help you with this question.
>>>>>
>>>>> Cheers,
>>>>> Till
>>>>>
>>>>> On Thu, Oct 22, 2020 at 2:56 PM Longdexin <27...@qq.com> wrote:
>>>>>
>>>>>> From my point of view, the value of NOW() function in SQL is certain
>>>>>> by the
>>>>>> time when the streaming app is launched and will not change with the
>>>>>> process
>>>>>> time. However, as a new Flink user, I'm not so sure of that. By the
>>>>>> way, if
>>>>>> my attemp is to keep the time logic to update all the time, what
>>>>>> should I
>>>>>> do?
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sent from:
>>>>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>>>>>
>>>>>

Re: How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Till Rohrmann <tr...@apache.org>]

Thanks for the clarification. This improvement would be helpful, I believe.

Cheers,
Till

On Tue, Oct 27, 2020 at 1:19 PM Jark Wu <im...@gmail.com> wrote:

> Hi Till,
>
> The documentation mentions that "this function is not deterministic" where
> the "not deterministic" means the value of this function is not
> deterministic for every record.
> However, this is not very clear for users. I think we can improve the
> documentation.
>
> Best,
> Jark
>
> On Tue, 27 Oct 2020 at 15:59, Till Rohrmann <tr...@apache.org> wrote:
>
>> Quick question Jark: Is this difference in behaviour documented? I
>> couldn't find it in the docs.
>>
>> Cheers,
>> Till
>>
>> On Tue, Oct 27, 2020 at 7:30 AM Jark Wu <im...@gmail.com> wrote:
>>
>>> Hi Longdexin,
>>>
>>> In traditional batch sql, NOW() is executed and determined before the
>>> job is submitted and will not change for every processed record.
>>> However, this doesn't make much sense in streaming sql, therefore, NOW()
>>> function in Flink is executed for every record.
>>>
>>> Best,
>>> Jark
>>>
>>> On Fri, 23 Oct 2020 at 16:30, Till Rohrmann <tr...@apache.org>
>>> wrote:
>>>
>>>> Hi Longdexin,
>>>>
>>>> thanks for reaching out to the Flink community. I am pulling in Jark
>>>> who might be able to help you with this question.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Thu, Oct 22, 2020 at 2:56 PM Longdexin <27...@qq.com> wrote:
>>>>
>>>>> From my point of view, the value of NOW() function in SQL is certain
>>>>> by the
>>>>> time when the streaming app is launched and will not change with the
>>>>> process
>>>>> time. However, as a new Flink user, I'm not so sure of that. By the
>>>>> way, if
>>>>> my attemp is to keep the time logic to update all the time, what
>>>>> should I
>>>>> do?
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from:
>>>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>>>>
>>>>

Re: How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Jark Wu <im...@gmail.com>]

Hi Till,

The documentation mentions that "this function is not deterministic" where
the "not deterministic" means the value of this function is not
deterministic for every record.
However, this is not very clear for users. I think we can improve the
documentation.

Best,
Jark

On Tue, 27 Oct 2020 at 15:59, Till Rohrmann <tr...@apache.org> wrote:

> Quick question Jark: Is this difference in behaviour documented? I
> couldn't find it in the docs.
>
> Cheers,
> Till
>
> On Tue, Oct 27, 2020 at 7:30 AM Jark Wu <im...@gmail.com> wrote:
>
>> Hi Longdexin,
>>
>> In traditional batch sql, NOW() is executed and determined before the job
>> is submitted and will not change for every processed record.
>> However, this doesn't make much sense in streaming sql, therefore, NOW()
>> function in Flink is executed for every record.
>>
>> Best,
>> Jark
>>
>> On Fri, 23 Oct 2020 at 16:30, Till Rohrmann <tr...@apache.org> wrote:
>>
>>> Hi Longdexin,
>>>
>>> thanks for reaching out to the Flink community. I am pulling in Jark who
>>> might be able to help you with this question.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Thu, Oct 22, 2020 at 2:56 PM Longdexin <27...@qq.com> wrote:
>>>
>>>> From my point of view, the value of NOW() function in SQL is certain by
>>>> the
>>>> time when the streaming app is launched and will not change with the
>>>> process
>>>> time. However, as a new Flink user, I'm not so sure of that. By the
>>>> way, if
>>>> my attemp is to keep the time logic to update all the time, what should
>>>> I
>>>> do?
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from:
>>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>>>
>>>

Re: How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Till Rohrmann <tr...@apache.org>]

Quick question Jark: Is this difference in behaviour documented? I couldn't
find it in the docs.

Cheers,
Till

On Tue, Oct 27, 2020 at 7:30 AM Jark Wu <im...@gmail.com> wrote:

> Hi Longdexin,
>
> In traditional batch sql, NOW() is executed and determined before the job
> is submitted and will not change for every processed record.
> However, this doesn't make much sense in streaming sql, therefore, NOW()
> function in Flink is executed for every record.
>
> Best,
> Jark
>
> On Fri, 23 Oct 2020 at 16:30, Till Rohrmann <tr...@apache.org> wrote:
>
>> Hi Longdexin,
>>
>> thanks for reaching out to the Flink community. I am pulling in Jark who
>> might be able to help you with this question.
>>
>> Cheers,
>> Till
>>
>> On Thu, Oct 22, 2020 at 2:56 PM Longdexin <27...@qq.com> wrote:
>>
>>> From my point of view, the value of NOW() function in SQL is certain by
>>> the
>>> time when the streaming app is launched and will not change with the
>>> process
>>> time. However, as a new Flink user, I'm not so sure of that. By the way,
>>> if
>>> my attemp is to keep the time logic to update all the time, what should I
>>> do?
>>>
>>>
>>>
>>> --
>>> Sent from:
>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>>
>>

Re: How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Jark Wu <im...@gmail.com>]

Hi Longdexin,

In traditional batch sql, NOW() is executed and determined before the job
is submitted and will not change for every processed record.
However, this doesn't make much sense in streaming sql, therefore, NOW()
function in Flink is executed for every record.

Best,
Jark

On Fri, 23 Oct 2020 at 16:30, Till Rohrmann <tr...@apache.org> wrote:

> Hi Longdexin,
>
> thanks for reaching out to the Flink community. I am pulling in Jark who
> might be able to help you with this question.
>
> Cheers,
> Till
>
> On Thu, Oct 22, 2020 at 2:56 PM Longdexin <27...@qq.com> wrote:
>
>> From my point of view, the value of NOW() function in SQL is certain by
>> the
>> time when the streaming app is launched and will not change with the
>> process
>> time. However, as a new Flink user, I'm not so sure of that. By the way,
>> if
>> my attemp is to keep the time logic to update all the time, what should I
>> do?
>>
>>
>>
>> --
>> Sent from:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>
>

Re: How to understand NOW() in SQL when using Table & SQL API to develop a streaming app?

[Till Rohrmann <tr...@apache.org>]

Hi Longdexin,

thanks for reaching out to the Flink community. I am pulling in Jark who
might be able to help you with this question.

Cheers,
Till

On Thu, Oct 22, 2020 at 2:56 PM Longdexin <27...@qq.com> wrote:

> From my point of view, the value of NOW() function in SQL is certain by the
> time when the streaming app is launched and will not change with the
> process
> time. However, as a new Flink user, I'm not so sure of that. By the way, if
> my attemp is to keep the time logic to update all the time, what should I
> do?
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>