[jira] [Updated] (NIFI-2946) Nifi can skip data in List processors and never ingest it afterwards

["Nicolas Dupont (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nicolas Dupont updated NIFI-2946:
---------------------------------
    Issue Type: Bug  (was: Improvement)

> Nifi can skip data in List processors and never ingest it afterwards
> --------------------------------------------------------------------
>
>                 Key: NIFI-2946
>                 URL: https://issues.apache.org/jira/browse/NIFI-2946
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.0.0
>         Environment: All
>            Reporter: Nicolas Dupont
>              Labels: errorhandling, listFiles, nifi, skip
>
> With current design for Listing processors, it seems like Nifi may skip some folder's contents which will never be ingested later.
> {code:title=Example from FTPTransfer.java|borderStyle=solid}
> if (recurse && file.isDirectory()) 
> {
> try 
> {
> listing.addAll(getListing(newFullForwardPath, depth + 1, maxResults - count));
> } 
> catch (final IOException e) 
> {
> logger.error("Unable to get listing from " + newFullForwardPath + "; skipping this subdirectory");
>  }
>  }
> {code}
> With this design, you can encounter a situation in which you can't list one or more subfolder's contents (for any reason like a sudden time-out) BUT still update the component state timestamps : at next runs previous "forgotten" files will never be pushed down because of their timestamps + they can't be reinjected through an error relationship.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)