[jira] [Comment Edited] (NIFI-4658) MergeContent Max Number of Entries resetting to default value

["Brian Ghigiarelli (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16278619#comment-16278619 ] 

Brian Ghigiarelli edited comment on NIFI-4658 at 12/5/17 2:27 PM:
------------------------------------------------------------------

Adding another split before it seems to work (sometimes?) when you save off the original fragments to merge later, though it requires careful consideration for the queue priorities in order to make sure that the multiple levels of splits don't overload MergeContent.

For example:
* Get(File/Http/etc.)
* Split(Text/Avro/etc.)
* UpdateAttribute
** set first.fragment.index to fragment.index
** set first.fragment.count to fragment.count
** set first.fragment.identifier to fragment.identifier
* Split(Text/Avro/etc.)
* NiftyProcessors(Group)
* MergeContent (defragments based on second split's fragment.* attributes)
* UpdateAttribute
** set fragment.index to first.fragment.index
** set fragment.count to first.fragment.count
** set fragment.identifier to first.fragment.identifier
* MergeContent (defragments based on the first split's fragment.* attributes)
* ProcessFinishedBatch

Though, am I making this unnecessarily complicated at this point?


was (Author: brianghig):
Adding another split before it seems to work (sometimes?) when you save off the original fragments to merge later, though it requires careful consideration for the queue priorities in order to make sure that the multiple levels of splits don't overload MergeContent.

For example:
* Get(File/Http/etc.)
* Split(Text/Avro/etc.)
* UpdateAttribute
** set first.fragment.index to fragment.index
** set first.fragment.count to fragment.count
** set first.fragment.identifier to fragment.identifier
* Split(Text/Avro/etc.)
* NiftyProcessors(Group)
* MergeContent (defragments based on second split's fragment.* attributes)
* UpdateAttribute
** set fragment.index to first.fragment.index
** set fragment.count to first.fragment.count
** set fragment.identifier to first.fragment.identifier
* MergeContent (defragments based on the first split's fragment.* attributes)

Though, am I making this unnecessarily complicated at this point?

> MergeContent Max Number of Entries resetting to default value
> -------------------------------------------------------------
>
>                 Key: NIFI-4658
>                 URL: https://issues.apache.org/jira/browse/NIFI-4658
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Brian Ghigiarelli
>
> Prior to and including 1.4.0, the MergeContent processor supports a property called "Maximum Number of Entries". It has a default value of 1,000. Prior to 1.4.0 and in the description of this property, if the property is not set, there will be no maximum number of files to include in a bundle.
> However, with the release of 1.4.0, if you clear the value of this property in order to have an unlimited number of files in the bundle and "Apply" the change, the next time that you open the configuration of the processor, it will again be set to the default value of 1,000. The expectation is that the cleared value will remain cleared, maintaining a configuration for an unlimited number of files in a merge bundle.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)