RE: TopSpeedWindowing - in error: Could not forward element to next operator

[Ovidiu Cristian Marcu <ov...@huawei.com>]

Thank you, I will check this fix in my environment.

Best,
Ovidiu

-----Original Message-----
From: Aljoscha Krettek [mailto:aljoscha@apache.org] 
Sent: Friday, October 21, 2016 5:47 PM
To: dev@flink.apache.org
Subject: Re: TopSpeedWindowing - in error: Could not forward element to next operator

Hi,
the problem is that EvictingWindowOperator uses StreamRecordSerializer to serialise the contents of the windows. This does not serialise timestamps so when the objects are deserialised from RocksDB they all have Long.MIN_VALUE as timestamp. The evictor in the program therefore always evicts all elements and the window function always sees an empty iterable.

I have a fix for this in a recent PR:
https://github.com/apache/flink/pull/2656

Cheers,
Aljoscha

On Fri, 21 Oct 2016 at 16:57 Ovidiu Cristian Marcu < ovidiu.cristian.marcu@huawei.com> wrote:

> Hi
>
> I have the latest source code Master.
> I think the refactoring you were doing on State Backend gives this 
> issue, unless the example is not supposed to work With rocksdb.
> At some point collected records's values are  null, giving the error..
>
> What do you think?
>
> Best,
> Ovidiu
>
> -----Original Message-----
> From: Till Rohrmann [mailto:trohrmann@apache.org]
> Sent: Friday, October 21, 2016 2:09 PM
> To: dev@flink.apache.org
> Subject: Re: TopSpeedWindowing - in error: Could not forward element 
> to next operator
>
> Hi Ovidiu,
>
> which version of Flink are you using?
>
> Cheers,
> Till
>
> On Thu, Oct 20, 2016 at 6:38 PM, Ovidiu Cristian Marcu < 
> ovidiu.cristian.marcu@huawei.com> wrote:
>
> > Could you check the following issue on master?
> >
> > When running this example org.apache.flink.streaming.examples.windowing.
> > TopSpeedWindowing
> > With default configuration I have no errors.
> >
> > When I change the state backend with RocksDB I receive this error:
> >
> > java.lang.RuntimeException: Could not forward element to next operator
> >                 at
> > org.apache.flink.streaming.runtime.tasks.OperatorChain$
> > CopyingChainingOutput.collect(OperatorChain.java:388)
> >                 at
> > org.apache.flink.streaming.runtime.tasks.OperatorChain$
> > ChainingOutput.collect(OperatorChain.java:1)
> >                 at org.apache.flink.streaming.api.operators.
> >
> AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:
> > 393)
> >                 at org.apache.flink.streaming.api.operators.
> > AbstractStreamOperator$CountingOutput.collect(
> > AbstractStreamOperator.java:1)
> >                 at org.apache.flink.streaming.api.operators.
> > TimestampedCollector.collect(TimestampedCollector.java:51)
> >                 at org.apache.flink.streaming.api.functions.windowing.
> > PassThroughWindowFunction.apply(PassThroughWindowFunction.java:32)
> >                 at org.apache.flink.streaming.api.functions.windowing.
> > ReduceApplyWindowFunction.apply(ReduceApplyWindowFunction.java:56)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > functions.InternalIterableWindowFunction.apply(
> > InternalIterableWindowFunction.java:50)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > functions.InternalIterableWindowFunction.apply(
> > InternalIterableWindowFunction.java:1)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > EvictingWindowOperator.fire(EvictingWindowOperator.java:334)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > EvictingWindowOperator.processElement(EvictingWindowOperator.java:199)
> >                 at org.apache.flink.streaming.runtime.io.
> > StreamInputProcessor.processInput(StreamInputProcessor.java:177)
> >                 at org.apache.flink.streaming.runtime.tasks.
> > OneInputStreamTask.run(OneInputStreamTask.java:66)
> >                 at org.apache.flink.streaming.runtime.tasks.StreamTask.
> > invoke(StreamTask.java:270)
> >                 at org.apache.flink.runtime.taskmanager.Task.run(Task.
> > java:609)
> >                 at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.NullPointerException
> >                 at org.apache.flink.api.java.typeutils.runtime.
> > TupleSerializer.copy(TupleSerializer.java:103)
> >                 at org.apache.flink.api.java.typeutils.runtime.
> > TupleSerializer.copy(TupleSerializer.java:30)
> >                 at
> > org.apache.flink.streaming.runtime.tasks.OperatorChain$
> > CopyingChainingOutput.collect(OperatorChain.java:383)
> >                 ... 15 more
> >
>