[jira] [Commented] (NIFI-3213) ListFile always skips files with the latest timestamp in an iteration even if the files have existed a while ago

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753922#comment-15753922 ] 

ASF GitHub Bot commented on NIFI-3213:
--------------------------------------

GitHub user ijokarumawak opened a pull request:

    https://github.com/apache/nifi/pull/1335

    NIFI-3213: Do not skip obviously old files.

    Thank you for submitting a contribution to Apache NiFi.
    
    In order to streamline the review of the contribution we ask you
    to ensure the following steps have been taken:
    
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
         in the commit message?
    
    - [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    
    - [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    - [x] Is your initial contribution a single, squashed commit?
    
    ### For code changes:
    - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
    - [x] Have you written or updated unit tests to verify your changes?
    
    ### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
    
    Before this fix, files with the latest timestamp within a listing
    iteration are always be held back one cycle no matter how old it is.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ijokarumawak/nifi nifi-3213

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/nifi/pull/1335.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1335
    
----
commit 9bfae4dda5f6e1fa37242207ee3b284da79dffaf
Author: Koji Kawamura <ij...@apache.org>
Date:   2016-12-16T08:48:06Z

    NIFI-3213: Do not skip obviously old files.
    
    Before this fix, files with the latest timestamp within a listing
    iteration are always be held back one cycle no matter how old it is.

----


> ListFile always skips files with the latest timestamp in an iteration even if the files have existed a while ago
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-3213
>                 URL: https://issues.apache.org/jira/browse/NIFI-3213
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.0.0, 0.5.0, 0.6.0, 0.5.1, 0.7.0, 0.6.1, 1.1.0, 0.7.1
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>
> NIFI-1484 add few lines of code to avoid files to be emitted if those have the latest timestamp within an iteration of listing, because it may still be written at the same time.
> While it doesn't affect much if ListFiles processor is scheduled with a short period of time, such as few ms, but it does affect negatively if an user scheduled it with longer run schedule such as "1 day" or with cron scheduler. 
> For example, user would expect to process list of files per daily basis. Even if a file is saved few hours ago, the processor will skip this, because the file has the latest timestamp within the iteration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)