Flink timestamps

[Davood Rafiei <ra...@gmail.com>]

Hi,

I want to assign timestamps to tuples when they enter source operator ,
get those timestamps in sink operator and extract them from current time
and output.

Basically my goal is to measure latency.

To set timestamps in source operator I used:



*  env.setStreamTimeCharacteristic(TimeCharacteristic.IngestionTime);*
However, I don't know how to get the tuple's previously assigned (in source
operator) timestamp in sink operator. Is it possible? Or should I insert a
separate field to tuple for keeping the timestamp?

Cheers
Davood

Re: Flink timestamps

[Aljoscha Krettek <al...@apache.org>]

Hi Davood,
right now, you can only inspect the timestamps by writing a custom operator
that you would use with DataStream.transform(). Measuring latency this way
has some pitfalls, though. The timestamp might be assigned on a different
machine than the machine that will process the tuple at the sink/custom
operator. If these machines have clocks that are slightly out of sync (as
they surely are) then the latency calculation will only be a rough estimate
(or completely wrong, in the worst case).

Cheers,
Aljoscha

On Sat, 6 Aug 2016 at 13:47 Davood Rafiei <ra...@gmail.com> wrote:

> Hi,
>
> I want to assign timestamps to tuples when they enter source operator ,
> get those timestamps in sink operator and extract them from current time
> and output.
>
> Basically my goal is to measure latency.
>
> To set timestamps in source operator I used:
>
>
>
> *  env.setStreamTimeCharacteristic(TimeCharacteristic.IngestionTime);*
> However, I don't know how to get the tuple's previously assigned (in
> source operator) timestamp in sink operator. Is it possible? Or should I
> insert a separate field to tuple for keeping the timestamp?
>
> Cheers
> Davood
>
>
>
>