You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Colvin Cowie <co...@gmail.com> on 2019/12/19 17:30:31 UTC

Jackson ObjectMapper reuse

Hi,

Looking through the source code to check up on a vulnerability in Jackson,
I saw that there's a couple of places on master where new *ObjectMapper*s
are being created and not reused.

I've found through experience that not reusing ObjectMappers (especially in
loops) can have a very detrimental impact on performance. Reuse is
recommended:
https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway
- "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
ObjectMapper is thread safe once configured.

The older uses just create them statically:

   - org.apache.solr.analytics.AnalyticsRequestParser
   - org.apache.solr.prometheus.scraper.SolrScraper

The new ones I see currently are in

   -
   org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
   - org.apache.solr.packagemanager.PackageUtils.getMapper()
   -
   org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()

I don't know whether these uses are necessarily going to cause performance
issues or not, but it's generally best not to new them up inline.

Colvin

Re: Jackson ObjectMapper reuse

Posted by Jan Høydahl <ja...@cominvent.com>.
AuditLoggerPlugin: https://issues.apache.org/jira/browse/SOLR-14129 <https://issues.apache.org/jira/browse/SOLR-14129>

Jan

> 20. des. 2019 kl. 01:28 skrev Noble Paul <no...@gmail.com>:
> 
> SolrJacksonAnnotationInspector
> 
> is OK, because it is created and given to the original caller who
> creates it. The caller is supposed to cache and reuse it
> 
> 
> On Fri, Dec 20, 2019 at 4:41 AM Ishan Chattopadhyaya
> <ic...@gmail.com> wrote:
>> 
>> Thanks Colvin for the analysis. Very helpful!
>> 
>> I added the one here:
>> org.apache.solr.packagemanager.PackageUtils.getMapper()
>> Now, that you mentioned, I'll change it to reuse it. However, this
>> particular one is not performance critical as it is called from the
>> SolrCLI (when "bin/solr package" support is used), and its life is as
>> long as a single command.
>> 
>> @Jan Høydahl can you please review the AuditLogger one?
>> @Noble Paul നോബിള്‍ नोब्ळ् can you please review the
>> SolrJacksonAnnotationInspector one?
>> 
>> On Thu, Dec 19, 2019 at 11:00 PM Colvin Cowie
>> <co...@gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> Looking through the source code to check up on a vulnerability in Jackson, I saw that there's a couple of places on master where new ObjectMappers are being created and not reused.
>>> 
>>> I've found through experience that not reusing ObjectMappers (especially in loops) can have a very detrimental impact on performance. Reuse is recommended:
>>> https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway - "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
>>> ObjectMapper is thread safe once configured.
>>> 
>>> The older uses just create them statically:
>>> 
>>> org.apache.solr.analytics.AnalyticsRequestParser
>>> org.apache.solr.prometheus.scraper.SolrScraper
>>> 
>>> The new ones I see currently are in
>>> 
>>> org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
>>> org.apache.solr.packagemanager.PackageUtils.getMapper()
>>> org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()
>>> 
>>> I don't know whether these uses are necessarily going to cause performance issues or not, but it's generally best not to new them up inline.
>>> 
>>> Colvin
>>> 
> 
> 
> 
> --
> -----------------------------------------------------
> Noble Paul


Re: Jackson ObjectMapper reuse

Posted by Noble Paul <no...@gmail.com>.
SolrJacksonAnnotationInspector

is OK, because it is created and given to the original caller who
creates it. The caller is supposed to cache and reuse it


On Fri, Dec 20, 2019 at 4:41 AM Ishan Chattopadhyaya
<ic...@gmail.com> wrote:
>
> Thanks Colvin for the analysis. Very helpful!
>
> I added the one here:
> org.apache.solr.packagemanager.PackageUtils.getMapper()
> Now, that you mentioned, I'll change it to reuse it. However, this
> particular one is not performance critical as it is called from the
> SolrCLI (when "bin/solr package" support is used), and its life is as
> long as a single command.
>
> @Jan Høydahl can you please review the AuditLogger one?
> @Noble Paul നോബിള്‍ नोब्ळ् can you please review the
> SolrJacksonAnnotationInspector one?
>
> On Thu, Dec 19, 2019 at 11:00 PM Colvin Cowie
> <co...@gmail.com> wrote:
> >
> > Hi,
> >
> > Looking through the source code to check up on a vulnerability in Jackson, I saw that there's a couple of places on master where new ObjectMappers are being created and not reused.
> >
> > I've found through experience that not reusing ObjectMappers (especially in loops) can have a very detrimental impact on performance. Reuse is recommended:
> > https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway - "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
> > ObjectMapper is thread safe once configured.
> >
> > The older uses just create them statically:
> >
> > org.apache.solr.analytics.AnalyticsRequestParser
> > org.apache.solr.prometheus.scraper.SolrScraper
> >
> > The new ones I see currently are in
> >
> > org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
> > org.apache.solr.packagemanager.PackageUtils.getMapper()
> > org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()
> >
> > I don't know whether these uses are necessarily going to cause performance issues or not, but it's generally best not to new them up inline.
> >
> > Colvin
> >



--
-----------------------------------------------------
Noble Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: Jackson ObjectMapper reuse

Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
Thanks Colvin for the analysis. Very helpful!

I added the one here:
org.apache.solr.packagemanager.PackageUtils.getMapper()
Now, that you mentioned, I'll change it to reuse it. However, this
particular one is not performance critical as it is called from the
SolrCLI (when "bin/solr package" support is used), and its life is as
long as a single command.

@Jan Høydahl can you please review the AuditLogger one?
@Noble Paul നോബിള്‍ नोब्ळ् can you please review the
SolrJacksonAnnotationInspector one?

On Thu, Dec 19, 2019 at 11:00 PM Colvin Cowie
<co...@gmail.com> wrote:
>
> Hi,
>
> Looking through the source code to check up on a vulnerability in Jackson, I saw that there's a couple of places on master where new ObjectMappers are being created and not reused.
>
> I've found through experience that not reusing ObjectMappers (especially in loops) can have a very detrimental impact on performance. Reuse is recommended:
> https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway - "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
> ObjectMapper is thread safe once configured.
>
> The older uses just create them statically:
>
> org.apache.solr.analytics.AnalyticsRequestParser
> org.apache.solr.prometheus.scraper.SolrScraper
>
> The new ones I see currently are in
>
> org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
> org.apache.solr.packagemanager.PackageUtils.getMapper()
> org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()
>
> I don't know whether these uses are necessarily going to cause performance issues or not, but it's generally best not to new them up inline.
>
> Colvin
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org