You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "David Handermann (Jira)" <ji...@apache.org> on 2025/01/18 14:29:00 UTC
[jira] [Commented] (NIFI-14155) JsonParserFactory will create an new ObjectMapper when process every flowfile, and the performance drops by more than 50%.
[ https://issues.apache.org/jira/browse/NIFI-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914342#comment-17914342 ]
David Handermann commented on NIFI-14155:
-----------------------------------------
Thanks for highlighting the poor performance associated with unnecessary object creation [~castile]. I have submitted a pull request to reuse ObjectMapper instances for JSON Record Readers and components that use the JsonParserFactory.
> JsonParserFactory will create an new ObjectMapper when process every flowfile, and the performance drops by more than 50%.
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-14155
> URL: https://issues.apache.org/jira/browse/NIFI-14155
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Examples
> Affects Versions: 1.26.0, 2.1.0
> Reporter: castile
> Assignee: David Handermann
> Priority: Critical
> Labels: Record, json, performance
> Attachments: image-2025-01-14-22-32-30-460.png, image-2025-01-14-22-39-57-860.png, image-2025-01-14-22-41-56-719.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Our team is using nifi1.26.0. When using UpdateRecord processor or other processor sets that inherit org.apache.nifi.processors.standard.AbstractRecordProcessor, we found that its processing performance dropped by more than 50%. After our analysis, we found that in the Parser object created by org.apache.nifi.json.JsonParserFactory, an ObjectMapper object is created when processing each data, which consumes a lot of performance.
> !image-2025-01-14-22-32-30-460.png|width=522,height=258!
>
> ObjectMapper is a *{color:#ff0000}heavyweight{color}* object and is {color:#ff0000}*thread-safe*{color}. It seems that a static global or singleton instance should be created.
> We have run the performance tests and found that it was useful，here is the compare results：
> BEFORE：
> !image-2025-01-14-22-41-56-719.png|width=447,height=120!
> AFTER：
> !image-2025-01-14-22-39-57-860.png|width=631,height=172!
> When each JSON data is large, the gap will be more obvious！！！！
> ------------------------------------------------------------------------
> *{color:#ff0000}No matter what，the ObjectMapper object should be unique in the controller service（For example, JsonTreeReader
> ）,and should not be created when processing each flowfile.{color}*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)