You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Colvin Cowie <co...@gmail.com> on 2019/12/19 17:30:31 UTC
Jackson ObjectMapper reuse
Hi,
Looking through the source code to check up on a vulnerability in Jackson,
I saw that there's a couple of places on master where new *ObjectMapper*s
are being created and not reused.
I've found through experience that not reusing ObjectMappers (especially in
loops) can have a very detrimental impact on performance. Reuse is
recommended:
https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway
- "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
ObjectMapper is thread safe once configured.
The older uses just create them statically:
- org.apache.solr.analytics.AnalyticsRequestParser
- org.apache.solr.prometheus.scraper.SolrScraper
The new ones I see currently are in
-
org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
- org.apache.solr.packagemanager.PackageUtils.getMapper()
-
org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()
I don't know whether these uses are necessarily going to cause performance
issues or not, but it's generally best not to new them up inline.
Colvin
Re: Jackson ObjectMapper reuse
Posted by Jan Høydahl <ja...@cominvent.com>.
AuditLoggerPlugin: https://issues.apache.org/jira/browse/SOLR-14129 <https://issues.apache.org/jira/browse/SOLR-14129>
Jan
> 20. des. 2019 kl. 01:28 skrev Noble Paul <no...@gmail.com>:
>
> SolrJacksonAnnotationInspector
>
> is OK, because it is created and given to the original caller who
> creates it. The caller is supposed to cache and reuse it
>
>
> On Fri, Dec 20, 2019 at 4:41 AM Ishan Chattopadhyaya
> <ic...@gmail.com> wrote:
>>
>> Thanks Colvin for the analysis. Very helpful!
>>
>> I added the one here:
>> org.apache.solr.packagemanager.PackageUtils.getMapper()
>> Now, that you mentioned, I'll change it to reuse it. However, this
>> particular one is not performance critical as it is called from the
>> SolrCLI (when "bin/solr package" support is used), and its life is as
>> long as a single command.
>>
>> @Jan Høydahl can you please review the AuditLogger one?
>> @Noble Paul നോബിള് नोब्ळ् can you please review the
>> SolrJacksonAnnotationInspector one?
>>
>> On Thu, Dec 19, 2019 at 11:00 PM Colvin Cowie
>> <co...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> Looking through the source code to check up on a vulnerability in Jackson, I saw that there's a couple of places on master where new ObjectMappers are being created and not reused.
>>>
>>> I've found through experience that not reusing ObjectMappers (especially in loops) can have a very detrimental impact on performance. Reuse is recommended:
>>> https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway - "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
>>> ObjectMapper is thread safe once configured.
>>>
>>> The older uses just create them statically:
>>>
>>> org.apache.solr.analytics.AnalyticsRequestParser
>>> org.apache.solr.prometheus.scraper.SolrScraper
>>>
>>> The new ones I see currently are in
>>>
>>> org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
>>> org.apache.solr.packagemanager.PackageUtils.getMapper()
>>> org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()
>>>
>>> I don't know whether these uses are necessarily going to cause performance issues or not, but it's generally best not to new them up inline.
>>>
>>> Colvin
>>>
>
>
>
> --
> -----------------------------------------------------
> Noble Paul
Re: Jackson ObjectMapper reuse
Posted by Noble Paul <no...@gmail.com>.
SolrJacksonAnnotationInspector
is OK, because it is created and given to the original caller who
creates it. The caller is supposed to cache and reuse it
On Fri, Dec 20, 2019 at 4:41 AM Ishan Chattopadhyaya
<ic...@gmail.com> wrote:
>
> Thanks Colvin for the analysis. Very helpful!
>
> I added the one here:
> org.apache.solr.packagemanager.PackageUtils.getMapper()
> Now, that you mentioned, I'll change it to reuse it. However, this
> particular one is not performance critical as it is called from the
> SolrCLI (when "bin/solr package" support is used), and its life is as
> long as a single command.
>
> @Jan Høydahl can you please review the AuditLogger one?
> @Noble Paul നോബിള് नोब्ळ् can you please review the
> SolrJacksonAnnotationInspector one?
>
> On Thu, Dec 19, 2019 at 11:00 PM Colvin Cowie
> <co...@gmail.com> wrote:
> >
> > Hi,
> >
> > Looking through the source code to check up on a vulnerability in Jackson, I saw that there's a couple of places on master where new ObjectMappers are being created and not reused.
> >
> > I've found through experience that not reusing ObjectMappers (especially in loops) can have a very detrimental impact on performance. Reuse is recommended:
> > https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway - "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
> > ObjectMapper is thread safe once configured.
> >
> > The older uses just create them statically:
> >
> > org.apache.solr.analytics.AnalyticsRequestParser
> > org.apache.solr.prometheus.scraper.SolrScraper
> >
> > The new ones I see currently are in
> >
> > org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
> > org.apache.solr.packagemanager.PackageUtils.getMapper()
> > org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()
> >
> > I don't know whether these uses are necessarily going to cause performance issues or not, but it's generally best not to new them up inline.
> >
> > Colvin
> >
--
-----------------------------------------------------
Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Jackson ObjectMapper reuse
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
Thanks Colvin for the analysis. Very helpful!
I added the one here:
org.apache.solr.packagemanager.PackageUtils.getMapper()
Now, that you mentioned, I'll change it to reuse it. However, this
particular one is not performance critical as it is called from the
SolrCLI (when "bin/solr package" support is used), and its life is as
long as a single command.
@Jan Høydahl can you please review the AuditLogger one?
@Noble Paul നോബിള് नोब्ळ् can you please review the
SolrJacksonAnnotationInspector one?
On Thu, Dec 19, 2019 at 11:00 PM Colvin Cowie
<co...@gmail.com> wrote:
>
> Hi,
>
> Looking through the source code to check up on a vulnerability in Jackson, I saw that there's a couple of places on master where new ObjectMappers are being created and not reused.
>
> I've found through experience that not reusing ObjectMappers (especially in loops) can have a very detrimental impact on performance. Reuse is recommended:
> https://github.com/FasterXML/jackson-docs/wiki/Presentation:-Jackson-Performance#basics-things-you-should-do-anyway - "1. Reuse heavy-weight objects: ObjectMapper (data-binding)"
> ObjectMapper is thread safe once configured.
>
> The older uses just create them statically:
>
> org.apache.solr.analytics.AnalyticsRequestParser
> org.apache.solr.prometheus.scraper.SolrScraper
>
> The new ones I see currently are in
>
> org.apache.solr.security.AuditLoggerPlugin.JSONAuditEventFormatter.formatEvent(AuditEvent)
> org.apache.solr.packagemanager.PackageUtils.getMapper()
> org.apache.solr.util.SolrJacksonAnnotationInspector.createObjectMapper()
>
> I don't know whether these uses are necessarily going to cause performance issues or not, but it's generally best not to new them up inline.
>
> Colvin
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org