You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Jonathan Eagles (JIRA)" <ji...@apache.org> on 2018/10/05 19:27:00 UTC
[jira] [Commented] (HADOOP-15550) Avoid static initialization of
ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640251#comment-16640251 ]
Jonathan Eagles commented on HADOOP-15550:
------------------------------------------
I have some interest in back-porting this improvement to 2.8 (Mainly to get rid of the new ObjectMapper creation for KMSJSONWriter). Seems like a simple cherry-pick to 3.1 is possible when JsonSerialization library was added as part of HADOOP-13786. Back-porting to earlier lines either means partial back-porting HADOOP-13786 or creating a small KMSJSONWriter custom performance patch to remove the ObjectMapper creation.
[~stevel@apache.org], [~tlipcon] do you have preference on this? It seems the most straight forward to 1) back-port this jira to 3.1. and 2) Create a separate KMSJSONWriter jira for earlier lines, but I am open to suggestions.
> Avoid static initialization of ObjectMappers
> --------------------------------------------
>
> Key: HADOOP-15550
> URL: https://issues.apache.org/jira/browse/HADOOP-15550
> Project: Hadoop Common
> Issue Type: Bug
> Components: performance
> Affects Versions: 3.2.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Fix For: 3.2.0
>
> Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt
>
>
> Various classes statically initialize an ObjectMapper READER instance. This ends up doing a bunch of class-loading of Jackson libraries that can add up to a fair amount of CPU, even if the reader ends up not being used. This is particularly the case with WebHdfsFileSystem, which is class-loaded by a serviceloader even when unused in a particular job. We should lazy-init these members instead of doing so as a static class member.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org