[jira] [Comment Edited] (NIFI-4638) Drop events disordering

["Netanel Bitan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-4638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304758#comment-16304758 ] 

Netanel Bitan edited comment on NIFI-4638 at 12/27/17 7:53 PM:
---------------------------------------------------------------

In my attempts to try to get to the bottom of the events disorder bug  I have written a small script that finds the disorders. I hope you might find this useful in trying to recreate the bug, and find out if you are experiencing it in any of your own nifi flows.

The script is attached to this ticket and can be found as a gist in this link: https://gist.github.com/bitanetanel/808cb3a0204aad5830f31d6d7ff89299 as well

To run the script simply configure the name of the nifi host and the amount of time back you want the script to check at the top of the document.

Its interesting to note that we have not been able to reproduce this bug in any of our more simple flows, and it seems to happen only in a very complicated flow consisting of 75+ processors.


was (Author: bitanetanel):
In my attempts to try to get to the bottom of the events disorder bug  I have written a small script that finds the disorders. I hope you might find this useful in trying to recreate the bug, and find out if you are experiencing it in any of your own nifi flows.

To run the script simply configure the name of the nifi host and the amount of time back you want the script to check at the top of the document.

Its interesting to note that we have not been able to reproduce this bug in any of our more simple flows, and it seems to happen only in a very complicated flow consisting of 75+ processors.

> Drop events disordering
> -----------------------
>
>                 Key: NIFI-4638
>                 URL: https://issues.apache.org/jira/browse/NIFI-4638
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: Omer Hadari
>         Attachments: nifi-script.py
>
>
> Drop events that are generated by terminated relationships alongside other event types are sometimes registered before them. This happens since the drop event’s timestamp is lower by a millisecond compared to the timestamp of the other event, even though the ordinal is higher. This causes the lineage graph to look as if the other event comes out of the drop event.
> The one millisecond difference and the fact that this bug is sporadic suggests a rollover to the next millisecond when registering the event before the drop event.
> For example this was encountered in the split json processor, when the original flowfile receives a fragment count and then dropped. This could be a bug in the processor itself, though it seems to be more generic.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)