Testing Flink Streaming applications - controlling the clock

[Rohit Agarwal <ro...@cloudflare.com>]

Hi,

I am writing tests for my flink streaming application. I mostly use
event-time. But there are some aspects which are still controlled by
wall-clock time. For example, I am using AssignerWithPeriodicWatermarks and
so watermarks are triggered based on wall-clock time. Similarly,
checkpoints are also triggered based on wall-clock time. Is there a way I
can manually control the clock which flink uses from my tests.

--
Rohit Agarwal

Re: Testing Flink Streaming applications - controlling the clock

[Till Rohrmann <tr...@apache.org>]

Hi Rohit,

it depends a little bit on your tests. If you test individual operators you
can use the AbstractStreamOperatorTestHarness class which allows to set the
processing time via AbstractStreamOperatorTestHarness#setProcessingTime.
You can also set the ProcessingTimeService used by a StreamTask via the
StreamTask#setProcessingTimeService method.

If you have an IT case where you want to test the whole topology, there is,
unfortunately, not much tooling around to set the test up so that you can
easily define your own ProcessingTimeService. But if you like, then you
could contribute it to Flink. It will be highly appreciated.

Cheers,
Till
​

On Fri, Dec 9, 2016 at 11:06 PM, Rohit Agarwal <ro...@cloudflare.com> wrote:

> Hi,
>
> I am writing tests for my flink streaming application. I mostly use
> event-time. But there are some aspects which are still controlled by
> wall-clock time. For example, I am using AssignerWithPeriodicWatermarks and
> so watermarks are triggered based on wall-clock time. Similarly,
> checkpoints are also triggered based on wall-clock time. Is there a way I
> can manually control the clock which flink uses from my tests.
>
> --
> Rohit Agarwal
>