[jira] [Commented] (NIFI-3649) Parallelize Consumption of Cluster Replication Responses

["Mark Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945521#comment-15945521 ] 

Mark Payne commented on NIFI-3649:
----------------------------------

Looking more into the code, it looks like doing the actual merge inline will be a pretty massive change. There are mergers for dozens of types of DTO objects. Each would have to be updated. It's unclear if all can be updated cleanly or if that would require even more effort to refactor things.

I will leave this ticket to buffer the responses, up to some max amount of heap. Then will create a new JIRA for merging responses inline, if we need to do so.

> Parallelize Consumption of Cluster Replication Responses
> --------------------------------------------------------
>
>                 Key: NIFI-3649
>                 URL: https://issues.apache.org/jira/browse/NIFI-3649
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>
> Currently, when we replicate a REST API call to all nodes in the cluster, we wait until we get back a Response object for each node. We then go to the Response Mapper and merge those responses into 1 NodeResponse. This merging is done serially, and this is where we actually do the reading of the response. We need to instead do this merging in parallel or buffer responses in parallel and then merge then when done.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)