You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2020/05/21 09:48:47 UTC

[GitHub] [nifi-minifi-cpp] fgerlits opened a new pull request #790: MINIFICPP-1177 Improvements to the TailFile processor

fgerlits opened a new pull request #790:
URL: https://github.com/apache/nifi-minifi-cpp/pull/790


   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   ### Description of PR
   
   This PR is not completely done, submitting for early feedback.
   
   Done:
   - handle rotation of log files correctly -- instead of the dodgy logic of doing one file at a time in onTrigger and trying to keep track using the persisted state, we follow the same logic as NiFi: only look at rolled files if the input file got truncated, only look at new rotated files matching the pattern, and stream all new content in one onTrigger call
   - find and pick up new files in Multiple file mode
   - yield when no flow files were generated, and only then (used to fail in multiple and rotation cases)
   - lots of minor cleanup, unit test fixes, and added more unit tests
   
   Still todo:
    - include the delimiter at the end of the flow file, as this is what NiFi does and it would work better on Windows
   - make '\n' the default delimiter? (currently the default is no delimiter)
   - support Recursive lookup in Multiple files mode
   - support Lookup frequency in Multiple files mode (we do a lookup in every onTrigger at the moment)
   - at startup, compare the persisted checksum with the current contents of the file, and rewind to the beginning of the file if they don't match
   - (maybe) support NiFi's Initial Start Position setting (Beginning of Time | Beginning of File | Current Time); we always use Beginning of File at the moment
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? Is it referenced
        in the commit message?
   
   - [x] Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
   
   - [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi-minifi-cpp] fgerlits commented on pull request #790: MINIFICPP-1177 Improvements to the TailFile processor

Posted by GitBox <gi...@apache.org>.
fgerlits commented on pull request #790:
URL: https://github.com/apache/nifi-minifi-cpp/pull/790#issuecomment-631997593


   I have found than some commits were rebased incorrectly; closing this PR and will open a new one.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #790: MINIFICPP-1177 Improvements to the TailFile processor

Posted by GitBox <gi...@apache.org>.
fgerlits closed pull request #790:
URL: https://github.com/apache/nifi-minifi-cpp/pull/790


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org