You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Durga Durga <du...@gmail.com> on 2019/02/22 00:59:36 UTC
SinkFunction.Context
HI Folks,
Was following the documentation for
https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html
long currentProcessingTime
<https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#currentProcessingTime-->
()
Returns the current processing time.
long currentWatermark
<https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#currentWatermark-->
()
Returns the current event-time watermark.
Long
<http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true>
timestamp
<https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#timestamp-->
()
Returns the timestamp of the current input record or null if the element
does not have an assigned timestamp.
- CurrentProcessing time - is this is the Event Time ? i.e the time when
the Event Occured ? (or) when the flink window got the message ?.
- timeStamp - is this the time the record is persisted in to the sync ?.
(or) the aggregated data timestamp ?. Say if I have 100 records in my time
window - which time comes in to affect.
- CurrentWaterMark - which time is this - the time the even occured - what
will this value be - when there are 1000 records in my time window
PS - We want to use some ID ( timestamp ) and associate with all the
records that are persisted (aggregated) in a given time window - i.e if
there are 1000 records aggregated - and they resulted in 10 aggregated
records - we want to give these 10 aggregated records the same ID and we
want to use one of the above time stamp
Thanks much.
Re: SinkFunction.Context
Posted by Durga Durga <du...@gmail.com>.
Excellent ! Thanks Rong..
On Thu, Feb 21, 2019 at 7:26 PM Rong Rong <wa...@gmail.com> wrote:
> Hi Durga,
>
> 1. currentProcessingTime: refers to this operator(SinkFunction)'s system
> time at the moment of invoke
> 1a. the time you are referring to as "flink window got the message" is the
> currentProcessingTime() invoked at the window operator (which provided by
> the WindowContext similar to this one [1])
> 2 currentWatermark: refers to the current watermark [2] received by this
> operator(SinkFunction)
> 3. timestamp: is actually the input record's event-time (this "input" is
> referring to the input to the SinkFunction, not to the entire Flink
> topology)
>
> Hope these help.
>
> --
> Rong
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#processwindowfunction
> [2]
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_time.html#event-time-and-watermarks
>
> On Thu, Feb 21, 2019 at 4:59 PM Durga Durga <du...@gmail.com> wrote:
>
>>
>> HI Folks,
>>
>> Was following the documentation for
>>
>>
>> https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html
>>
>>
>>
>> long currentProcessingTime
>> <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#currentProcessingTime-->
>> ()
>> Returns the current processing time.
>> long currentWatermark
>> <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#currentWatermark-->
>> ()
>> Returns the current event-time watermark.
>> Long
>> <http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true>
>> timestamp
>> <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#timestamp-->
>> ()
>> Returns the timestamp of the current input record or null if the element
>> does not have an assigned timestamp.
>>
>> - CurrentProcessing time - is this is the Event Time ? i.e the time when
>> the Event Occured ? (or) when the flink window got the message ?.
>>
>> - timeStamp - is this the time the record is persisted in to the sync ?.
>> (or) the aggregated data timestamp ?. Say if I have 100 records in my time
>> window - which time comes in to affect.
>>
>> - CurrentWaterMark - which time is this - the time the even occured -
>> what will this value be - when there are 1000 records in my time window
>>
>> PS - We want to use some ID ( timestamp ) and associate with all the
>> records that are persisted (aggregated) in a given time window - i.e if
>> there are 1000 records aggregated - and they resulted in 10 aggregated
>> records - we want to give these 10 aggregated records the same ID and we
>> want to use one of the above time stamp
>>
>> Thanks much.
>>
>
--
_Durga Deep
Re: SinkFunction.Context
Posted by Rong Rong <wa...@gmail.com>.
Hi Durga,
1. currentProcessingTime: refers to this operator(SinkFunction)'s system
time at the moment of invoke
1a. the time you are referring to as "flink window got the message" is the
currentProcessingTime() invoked at the window operator (which provided by
the WindowContext similar to this one [1])
2 currentWatermark: refers to the current watermark [2] received by this
operator(SinkFunction)
3. timestamp: is actually the input record's event-time (this "input" is
referring to the input to the SinkFunction, not to the entire Flink
topology)
Hope these help.
--
Rong
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#processwindowfunction
[2]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_time.html#event-time-and-watermarks
On Thu, Feb 21, 2019 at 4:59 PM Durga Durga <du...@gmail.com> wrote:
>
> HI Folks,
>
> Was following the documentation for
>
>
> https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html
>
>
>
> long currentProcessingTime
> <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#currentProcessingTime-->
> ()
> Returns the current processing time.
> long currentWatermark
> <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#currentWatermark-->
> ()
> Returns the current event-time watermark.
> Long
> <http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true>
> timestamp
> <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html#timestamp-->
> ()
> Returns the timestamp of the current input record or null if the element
> does not have an assigned timestamp.
>
> - CurrentProcessing time - is this is the Event Time ? i.e the time when
> the Event Occured ? (or) when the flink window got the message ?.
>
> - timeStamp - is this the time the record is persisted in to the sync ?.
> (or) the aggregated data timestamp ?. Say if I have 100 records in my time
> window - which time comes in to affect.
>
> - CurrentWaterMark - which time is this - the time the even occured - what
> will this value be - when there are 1000 records in my time window
>
> PS - We want to use some ID ( timestamp ) and associate with all the
> records that are persisted (aggregated) in a given time window - i.e if
> there are 1000 records aggregated - and they resulted in 10 aggregated
> records - we want to give these 10 aggregated records the same ID and we
> want to use one of the above time stamp
>
> Thanks much.
>