You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by "Ganelin, Ilya" <Il...@capitalone.com> on 2016/03/17 21:00:45 UTC
Obscure buffer server error
Hello all – could use some help understanding the error below:
This shows up when I run the Apex program in local mode. Things run fine for a short time, then I get a pause in my output and see these messages ultimately terminating in an exception:
2016-03-17 12:54:14,458 [ProcessWideEventLoop] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@18b788b4{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000001, ending_window=56eb0b5e00000012, refCount=3, uniqueIdentifier=0, next=11.latenciesOut.2, future=null} in memory due to 3 references.
2016-03-17 12:54:14,459 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@18b788b4{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000001, ending_window=56eb0b5e00000012, refCount=2, uniqueIdentifier=0, next=11.latenciesOut.2, future=null} in memory due to 2 references.
2016-03-17 12:54:14,459 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@18b788b4{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000001, ending_window=56eb0b5e00000012, refCount=1, uniqueIdentifier=0, next=11.latenciesOut.2, future=null} in memory due to 1 references.
2016-03-17 12:54:14,486 [ProcessWideEventLoop] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@7abb74fb{identifier=1.output.1, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000011, ending_window=56eb0b5e00000012, refCount=3, uniqueIdentifier=0, next=1.output.1, future=null} in memory due to 3 references.
2016-03-17 12:54:14,486 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@7abb74fb{identifier=1.output.1, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000011, ending_window=56eb0b5e00000012, refCount=2, uniqueIdentifier=0, next=1.output.1, future=null} in memory due to 2 references.
2016-03-17 12:54:14,487 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@7abb74fb{identifier=1.output.1, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000011, ending_window=56eb0b5e00000012, refCount=1, uniqueIdentifier=0, next=1.output.1, future=null} in memory due to 1 references.
2016-03-17 12:55:18,708 [ProcessWideEventLoop] WARN netlet.DefaultEventLoop run -
Unexpected termination of {id=ProcessWideEventLoop, head=66, tail=66, capacity=1024}
Exception in thread "ProcessWideEventLoop" java.lang.IllegalStateException: Discarded block com.datatorrent.bufferserver.internal.DataList$Block@20436d87{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=386752, starting_window=56eb0b5e00000012, ending_window=56eb0b5e00000014, refCount=5, uniqueIdentifier=0, next=null, future=Done} has positive reference count!
at com.datatorrent.bufferserver.internal.DataList.rewind(DataList.java:122)
at com.datatorrent.bufferserver.server.Server.handlePublisherRequest(Server.java:321)
at com.datatorrent.bufferserver.server.Server$UnidentifiedClient.onMessage(Server.java:408)
at com.datatorrent.netlet.AbstractLengthPrependerClient.read(AbstractLengthPrependerClient.java:149)
at com.datatorrent.netlet.AbstractClient.read(AbstractClient.java:117)
at com.datatorrent.netlet.DefaultEventLoop.handleSelectedKey(DefaultEventLoop.java:314)
at com.datatorrent.netlet.OptimizedEventLoop$SelectedSelectionKeySet.forEach(OptimizedEventLoop.java:58)
at com.datatorrent.netlet.OptimizedEventLoop.runEventLoop(OptimizedEventLoop.java:188)
at com.datatorrent.netlet.OptimizedEventLoop.runEventLoop(OptimizedEventLoop.java:156)
at com.datatorrent.netlet.DefaultEventLoop.run(DefaultEventLoop.java:119)
at java.lang.Thread.run(Thread.java:745)
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: Obscure buffer server error
Posted by Vlad Rozov <v....@datatorrent.com>.
This exception is raised when buffer server is requested to discard a
block as a result of purge, reset or rewind operation that has not zero
reference count, meaning that there is a subscriber that consumes the
block to be discarded. There was a recent fix to a bug caused by a
publisher iterator not being properly closed, thus leaving positive
reference count behind. Please see APEXCORE-374.
Thank you,
Vlad
On 3/17/16 13:00, Ganelin, Ilya wrote:
> Hello all – could use some help understanding the error below:
>
> This shows up when I run the Apex program in local mode. Things run fine for a short time, then I get a pause in my output and see these messages ultimately terminating in an exception:
>
> 2016-03-17 12:54:14,458 [ProcessWideEventLoop] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@18b788b4{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000001, ending_window=56eb0b5e00000012, refCount=3, uniqueIdentifier=0, next=11.latenciesOut.2, future=null} in memory due to 3 references.
> 2016-03-17 12:54:14,459 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@18b788b4{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000001, ending_window=56eb0b5e00000012, refCount=2, uniqueIdentifier=0, next=11.latenciesOut.2, future=null} in memory due to 2 references.
> 2016-03-17 12:54:14,459 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@18b788b4{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000001, ending_window=56eb0b5e00000012, refCount=1, uniqueIdentifier=0, next=11.latenciesOut.2, future=null} in memory due to 1 references.
> 2016-03-17 12:54:14,486 [ProcessWideEventLoop] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@7abb74fb{identifier=1.output.1, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000011, ending_window=56eb0b5e00000012, refCount=3, uniqueIdentifier=0, next=1.output.1, future=null} in memory due to 3 references.
> 2016-03-17 12:54:14,486 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@7abb74fb{identifier=1.output.1, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000011, ending_window=56eb0b5e00000012, refCount=2, uniqueIdentifier=0, next=1.output.1, future=null} in memory due to 2 references.
> 2016-03-17 12:54:14,487 [ServerHelper-1-1] DEBUG internal.DataList release - Holding com.datatorrent.bufferserver.internal.DataList$Block@7abb74fb{identifier=1.output.1, data=1048576, readingOffset=0, writingOffset=1048576, starting_window=56eb0b5e00000011, ending_window=56eb0b5e00000012, refCount=1, uniqueIdentifier=0, next=1.output.1, future=null} in memory due to 1 references.
>
>
> 2016-03-17 12:55:18,708 [ProcessWideEventLoop] WARN netlet.DefaultEventLoop run -
> Unexpected termination of {id=ProcessWideEventLoop, head=66, tail=66, capacity=1024}
>
> Exception in thread "ProcessWideEventLoop" java.lang.IllegalStateException: Discarded block com.datatorrent.bufferserver.internal.DataList$Block@20436d87{identifier=11.latenciesOut.2, data=1048576, readingOffset=0, writingOffset=386752, starting_window=56eb0b5e00000012, ending_window=56eb0b5e00000014, refCount=5, uniqueIdentifier=0, next=null, future=Done} has positive reference count!
> at com.datatorrent.bufferserver.internal.DataList.rewind(DataList.java:122)
> at com.datatorrent.bufferserver.server.Server.handlePublisherRequest(Server.java:321)
> at com.datatorrent.bufferserver.server.Server$UnidentifiedClient.onMessage(Server.java:408)
> at com.datatorrent.netlet.AbstractLengthPrependerClient.read(AbstractLengthPrependerClient.java:149)
> at com.datatorrent.netlet.AbstractClient.read(AbstractClient.java:117)
> at com.datatorrent.netlet.DefaultEventLoop.handleSelectedKey(DefaultEventLoop.java:314)
> at com.datatorrent.netlet.OptimizedEventLoop$SelectedSelectionKeySet.forEach(OptimizedEventLoop.java:58)
> at com.datatorrent.netlet.OptimizedEventLoop.runEventLoop(OptimizedEventLoop.java:188)
> at com.datatorrent.netlet.OptimizedEventLoop.runEventLoop(OptimizedEventLoop.java:156)
> at com.datatorrent.netlet.DefaultEventLoop.run(DefaultEventLoop.java:119)
> at java.lang.Thread.run(Thread.java:745)
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.