You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Marco Villalobos <mv...@kineteque.com> on 2021/01/26 18:44:30 UTC

windows and triggers

I wrote this simple test:

.window(TumblingProcessingTimeWindows.of(Time.minutes(1)))
.trigger(PurgingTrigger.of(CountTrigger.of(5)))

Thinking that if I send 2 elements of data, it would collect them after a
minute.
But that doesn't seem to be happening.

Is my understanding of how windows and triggers work correct?

/**
 * To test this job first in command line make a simple server on a terminal
 *
 * nc -l 8889
 *
 * Then start this job at the command line or in IDE. Then in the
terminal input each value by typing a line of text.
 * Each line of text (excluding the new line) will be picked up by this job.
 */
public class TriggerTestJob {

   public static void main(String args[]) throws Exception {
      final StreamExecutionEnvironment streamEnv =
StreamExecutionEnvironment.getExecutionEnvironment();
      streamEnv.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
      streamEnv.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
      streamEnv.setStreamTimeCharacteristic(TimeCharacteristic.ProcessingTime);

      streamEnv.socketTextStream("localhost", 8889)
            .map(value -> new Tuple2<String, String>("test",
value)).returns(new TypeHint<Tuple2<String, String>>(){})
            .keyBy((KeySelector<Tuple2<String, String>, String>) value
-> value.f0)
            .window(TumblingProcessingTimeWindows.of(Time.minutes(1)))
            .trigger(PurgingTrigger.of(CountTrigger.of(5)))
            .process(new ProcessWindowFunction<Tuple2<String, String>,
Tuple2<String, String>, String, TimeWindow>() {
               @Override
               public void process(String key, Context context,
Iterable<Tuple2<String, String>> elements, Collector<Tuple2<String,
String>> out) throws Exception {
                  for (Tuple2<String, String> element : elements) {
                     out.collect(element);
                  }
               }
            }).name("trigger")
            .print();

      streamEnv.execute("trigger test");
   }
}

Re: windows and triggers

Posted by Marco Villalobos <mv...@kineteque.com>.
Ah, this works as expected, since Flink documentation states this:

By specifying a trigger using trigger() you are overwriting the default
> trigger of a WindowAssigner. For example, if you specify a CountTrigger for
> TumblingEventTimeWindows you will no longer get window firings based on the
> progress of time but only by count. Right now, you have to write your own
> custom trigger if you want to react based on both time and count.
>



On Tue, Jan 26, 2021 at 10:44 AM Marco Villalobos <mv...@kineteque.com>
wrote:

> I wrote this simple test:
>
> .window(TumblingProcessingTimeWindows.of(Time.minutes(1)))
> .trigger(PurgingTrigger.of(CountTrigger.of(5)))
>
> Thinking that if I send 2 elements of data, it would collect them after a
> minute.
> But that doesn't seem to be happening.
>
> Is my understanding of how windows and triggers work correct?
>
> /**
>  * To test this job first in command line make a simple server on a terminal
>  *
>  * nc -l 8889
>  *
>  * Then start this job at the command line or in IDE. Then in the terminal input each value by typing a line of text.
>  * Each line of text (excluding the new line) will be picked up by this job.
>  */
> public class TriggerTestJob {
>
>    public static void main(String args[]) throws Exception {
>       final StreamExecutionEnvironment streamEnv = StreamExecutionEnvironment.getExecutionEnvironment();
>       streamEnv.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
>       streamEnv.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
>       streamEnv.setStreamTimeCharacteristic(TimeCharacteristic.ProcessingTime);
>
>       streamEnv.socketTextStream("localhost", 8889)
>             .map(value -> new Tuple2<String, String>("test", value)).returns(new TypeHint<Tuple2<String, String>>(){})
>             .keyBy((KeySelector<Tuple2<String, String>, String>) value -> value.f0)
>             .window(TumblingProcessingTimeWindows.of(Time.minutes(1)))
>             .trigger(PurgingTrigger.of(CountTrigger.of(5)))
>             .process(new ProcessWindowFunction<Tuple2<String, String>, Tuple2<String, String>, String, TimeWindow>() {
>                @Override
>                public void process(String key, Context context, Iterable<Tuple2<String, String>> elements, Collector<Tuple2<String, String>> out) throws Exception {
>                   for (Tuple2<String, String> element : elements) {
>                      out.collect(element);
>                   }
>                }
>             }).name("trigger")
>             .print();
>
>       streamEnv.execute("trigger test");
>    }
> }
>
>