You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@nifi.apache.org by 李磊 <le...@ruisdata.com> on 2018/03/14 15:09:21 UTC

issue with nifi startup performance

Hi,

I use KYLO with NIFI-1.3.0 now. In KYLO, I create 100 feed with data-ingest template to transform data form RDBMS to HDFS, which provided by kylo.Finally, more than 1000 processors were created and there are 29 processors in the reuseable_templates. NIFI uses 6g memory.

However, when I restart the NIFI, more than 7 mins cost. By modify the source code and write log, I find that nearly 6 min costs in org.apache.nifi.controller.StandardFlowSynchronizer Class<sync>, especially method <addProcessGroup>. By read the source code, I think the function of <addProcessGroup> is analyze flow.xml.gz and restore the NIFI status before turned off.

Is there any ways to improve the performance of <addProcessGroup>? We will create more than 1000 feeds in KYLO in production, means more than 10000 processors will be existed in NIFI.

Thanks!