[jira] [Updated] (MINIFICPP-1515) Add integration tests verifying that MiNiFi handles large flowfiles as expected

["Adam Hunyadi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MINIFICPP-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adam Hunyadi updated MINIFICPP-1515:
------------------------------------
    Labels: MiNiFi-CPP-Hygiene  (was: )

> Add integration tests verifying that MiNiFi handles large flowfiles as expected
> -------------------------------------------------------------------------------
>
>                 Key: MINIFICPP-1515
>                 URL: https://issues.apache.org/jira/browse/MINIFICPP-1515
>             Project: Apache NiFi MiNiFi C++
>          Issue Type: Improvement
>    Affects Versions: 1.0.0, 0.9.0
>            Reporter: Adam Hunyadi
>            Assignee: Adam Hunyadi
>            Priority: Minor
>              Labels: MiNiFi-CPP-Hygiene
>             Fix For: 1.0.0
>
>
> *Background:*
> Issues with big flowfiles were reported here:
> https://stackoverflow.com/questions/66330866/minifi-getfile-processor-fails-to-get-large-files/66334615?noredirect=1#comment117275399_66334615
> Seems like this is related to twi issues, one a [narrowing exception happens|https://github.com/apache/nifi-minifi-cpp/blob/minifi-cpp-0.9.0-RC2/libminifi/src/core/ContentSession.cpp#L75] when trying to determine the length of the file to be written into the content repository, and another is that we use _stat on windows even though we should be using _stat64 to query files larger than 2GB. 
> *Proposal:*
> MiNiFi should be tested with large files.
> *Acceptance criteria:*
> {code:python|title=Feature definition proposal}
>   Scenario Outline: MiNiFi is capable of manipulating flowfiles of different sizes
>     Given a GetFile processor with the "Input Directory" property set to "/tmp/input"
>     And a file with <file size> of content is present in "/tmp/input"
>     And a PutFile processor with the "Directory" property set to "/tmp/output"
>     And the "success" relationship of the GetFile processor is connected to the PutFile
>     When the MiNiFi instance starts up
>     Then a flowfile with matching content is placed in the monitored directory in less than 20 seconds
>   Examples: File size
>     | file size |
>     | 10 B      |
>     | 1.5 KiB   |
>     | 10 MiB    |
>     | 1.0 GiB   |
>     | 2.1 GiB   |
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)