You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Kay Kay (JIRA)" <ji...@apache.org> on 2008/12/14 18:15:44 UTC
[jira] Created: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
--------------------------------------------------------------------------------
Key: SOLR-913
URL: https://issues.apache.org/jira/browse/SOLR-913
Project: Solr
Issue Type: Improvement
Components: clients - java
Environment: Tomcat 6, JRE 6
Reporter: Kay Kay
Fix For: 1.4
In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
static Integer readInterval(String interval) ;
Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SOLR-913) org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Ok, makes sense. Done.
On Sun, Dec 14, 2008 at 11:23 PM, Noble Paul നോബിള് नोब्ळ् <
noble.paul@gmail.com> wrote:
> I guess so. static Objects live in the permgen
> keep as few as possible
>
> On Sun, Dec 14, 2008 at 11:21 PM, Shalin Shekhar Mangar (JIRA)
> <ji...@apache.org> wrote:
> >
> > [
> https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656422#action_12656422]
> >
> > Shalin Shekhar Mangar commented on SOLR-913:
> > --------------------------------------------
> >
> > bq. this object is used only in the startup a couple of times It is
> better to use it and throw away rather than holding on to the reference
> forever
> >
> > I just committed this, too trigger happy you see :)
> >
> > But I see your point, this method is used on startup only. Is it worth
> reverting this change?
> >
> >> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
> static
> >>
> --------------------------------------------------------------------------------
> >>
> >> Key: SOLR-913
> >> URL: https://issues.apache.org/jira/browse/SOLR-913
> >> Project: Solr
> >> Issue Type: Improvement
> >> Components: clients - java
> >> Environment: Tomcat 6, JRE 6
> >> Reporter: Kay Kay
> >> Fix For: 1.4
> >>
> >> Attachments: SOLR-913.patch
> >>
> >> Original Estimate: 2h
> >> Remaining Estimate: 2h
> >>
> >> In the class - org.apache.solr.handler.SnapPuller - there seems to be
> an expensive Pattern object created locally in the method
> >> static Integer readInterval(String interval) ;
> >> Pattern instances are better created as static objects and reused.
> >
> > --
> > This message is automatically generated by JIRA.
> > -
> > You can reply to this email to add a comment to the issue online.
> >
> >
>
>
>
> --
> --Noble Paul
>
--
Regards,
Shalin Shekhar Mangar.
Re: [jira] Commented: (SOLR-913) org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
I guess so. static Objects live in the permgen
keep as few as possible
On Sun, Dec 14, 2008 at 11:21 PM, Shalin Shekhar Mangar (JIRA)
<ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656422#action_12656422 ]
>
> Shalin Shekhar Mangar commented on SOLR-913:
> --------------------------------------------
>
> bq. this object is used only in the startup a couple of times It is better to use it and throw away rather than holding on to the reference forever
>
> I just committed this, too trigger happy you see :)
>
> But I see your point, this method is used on startup only. Is it worth reverting this change?
>
>> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
>> --------------------------------------------------------------------------------
>>
>> Key: SOLR-913
>> URL: https://issues.apache.org/jira/browse/SOLR-913
>> Project: Solr
>> Issue Type: Improvement
>> Components: clients - java
>> Environment: Tomcat 6, JRE 6
>> Reporter: Kay Kay
>> Fix For: 1.4
>>
>> Attachments: SOLR-913.patch
>>
>> Original Estimate: 2h
>> Remaining Estimate: 2h
>>
>> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
>> static Integer readInterval(String interval) ;
>> Pattern instances are better created as static objects and reused.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
--
--Noble Paul
Re: [jira] Updated: (SOLR-913) org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Kay, analysis refers to the Analyzers inside Solr. Your original components
description was correct.
On Tue, Dec 16, 2008 at 1:03 AM, Kay Kay (JIRA) <ji...@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Kay Kay updated SOLR-913:
> -------------------------
>
> Component/s: (was: replication (java))
> Analysis
> Priority: Minor (was: Major)
>
> > org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
> static
> >
> --------------------------------------------------------------------------------
> >
> > Key: SOLR-913
> > URL: https://issues.apache.org/jira/browse/SOLR-913
> > Project: Solr
> > Issue Type: Improvement
> > Components: Analysis
> > Environment: Tomcat 6, JRE 6
> > Reporter: Kay Kay
> > Priority: Minor
> > Fix For: 1.4
> >
> > Attachments: SOLR-913.patch
> >
> > Original Estimate: 2h
> > Remaining Estimate: 2h
> >
> > In the class - org.apache.solr.handler.SnapPuller - there seems to be an
> expensive Pattern object created locally in the method
> > static Integer readInterval(String interval) ;
> > Pattern instances are better created as static objects and reused.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
--
Regards,
Shalin Shekhar Mangar.
[jira] Reopened: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay reopened SOLR-913:
--------------------------
The bigger problem that we are trying to address on our end is a fast SolrCore shutdown and restart and this is part of the bigger picture.
Pattern objects are notoriously heavy to create.
If necessary - they could be wrapped around as a resource handler that can set the static reference to null after the last use so that the static reference does not take up the space.
Allowing this to persist because we have 2 calls might affect the scalability as some other piece of code , inadvertently uses the same.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656534#action_12656534 ]
Noble Paul commented on SOLR-913:
---------------------------------
bq.As mentioned earlier - this is a bottleneck got from profiling
Profiling tools alone cannot be used to make a judgment because they are dumb and just spits out suggestions on the basis of a few rules. While it is useful, we need to study the environment in which the code is run and a it is good to have better understanding of a system to make relevant changes.
It is also worthwhile to run the system and see the real-world impact .
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656429#action_12656429 ]
Shalin Shekhar Mangar commented on SOLR-913:
--------------------------------------------
bq. The bigger problem that we are trying to address on our end is a fast SolrCore shutdown and restart and this is part of the bigger picture.
This does not cause longer shutdowns. If it is used only once in the lifetime of a core, why would we need to keep it around all the time?
bq. If necessary - they could be wrapped around as a resource handler that can set the static reference to null after the last use so that the static reference does not take up the space.
As Noble pointed out, it is not used more than once. Keeping it static and then nulling the reference after the only use, serves no purpose.
bq. Allowing this to persist because we have 2 calls might affect the scalability as some other piece of code , inadvertently uses the same.
I do not understand this point at all.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static (HttpClient object too )
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated SOLR-913:
-------------------------
Summary: org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static (HttpClient object too ) (was: org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static )
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static (HttpClient object too )
> ---------------------------------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch, SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
> The same is true for HttpClient instances. These are one per core right now. We can make that static too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656419#action_12656419 ]
Noble Paul commented on SOLR-913:
---------------------------------
this object is used only in the startup a couple of times It is better to use it and throw away rather than holding on to the reference forever
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656485#action_12656485 ]
Ryan McKinley commented on SOLR-913:
------------------------------------
How many cores how often? Does changing it to static make any mesurable difference?
A bottleneck has to be *somewhere* -- perhaps it is not really an issue (i don't know if it is or is not) -- it sounds like this was designed thinking the function is called infrequently while your use case is different...
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656872#action_12656872 ]
Noble Paul commented on SOLR-913:
---------------------------------
OK. In that case this is a valid concern . It is better off to make it a static object so that every core startup can reuse the same instance.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: Analysis
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar resolved SOLR-913.
----------------------------------------
Resolution: Won't Fix
Marking as won't fix because the readInterval method is called only on core startup and it is better to create and throw away Pattern object rather then keep it in static.
Thanks for going through the code Kay, please keep the suggestions coming :)
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656469#action_12656469 ]
Kay Kay commented on SOLR-913:
------------------------------
As mentioned earlier - this is a bottleneck got from profiling. By providing better abstractions - we can move make complex systems more tractable. Performance advantage, for something as big as a regex object, is not something that should not be sacrificed here in this case.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656422#action_12656422 ]
Shalin Shekhar Mangar commented on SOLR-913:
--------------------------------------------
bq. this object is used only in the startup a couple of times It is better to use it and throw away rather than holding on to the reference forever
I just committed this, too trigger happy you see :)
But I see your point, this method is used on startup only. Is it worth reverting this change?
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656711#action_12656711 ]
Kay Kay commented on SOLR-913:
------------------------------
> Is it true that you have a large no:of cores and you start/stop them very frequently and the start/stop performance is a concern for you?
Unfortunately yes. Hence my focus has been primarily on profiling the code path to start the solr system and hence to stress this. I agree with your concern about having static objects being live (after their intended use ) affecting memory but may be that could be grouped under ResourceMap but recreating the Pattern object , internally with all the NFA graph compilation would be better to be optimized as against the memory constraint.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static (HttpClient object too )
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay closed SOLR-913.
------------------------
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static (HttpClient object too )
> ---------------------------------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch, SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
> The same is true for HttpClient instances. These are one per core right now. We can make that static too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-913:
---------------------------------------
Component/s: (was: Analysis)
replication (java)
Description:
In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
static Integer readInterval(String interval) ;
Pattern instances are better created as static objects and reused.
The same is true for HttpClient instances. These are one per core right now. We can make that static too.
was:
In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
static Integer readInterval(String interval) ;
Pattern instances are better created as static objects and reused.
Assignee: Shalin Shekhar Mangar
HttpClient can be made static too in that case.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
> The same is true for HttpClient instances. These are one per core right now. We can make that static too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657017#action_12657017 ]
Kay Kay commented on SOLR-913:
------------------------------
Shalin - the new patch looks good. Thanks for helping with it.
Thanks everybody in the thread for the discussions regarding this.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch, SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
> The same is true for HttpClient instances. These are one per core right now. We can make that static too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656432#action_12656432 ]
Kay Kay commented on SOLR-913:
------------------------------
It causes longer time to restart in terms of recreating the heavy Pattern object once again.
There is a difference between keeping a static reference pointing to a live object and setting it to null . In the case of the latter - it definitely frees up the memory on a good implementation.
I meant to say that - if there were to be another piece of code , that inadvertently - calls this piece of code once again by creating additional Pattern objects would be suicidal. These are not random starting pain points that I am encountering in the startup but rather in the descending order of the time from profiling the shutdown / startup process.
While I believe the concern about static Pattern object is misplaced - we can implement a global ResourceMap with Key as String, Object and the users of the Resources removing entries from the Map after they are used . This is pretty standard across projects in the sense it addresses the issue of duplicate creation of resources and the issue of releasing the resources after the last use.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656571#action_12656571 ]
Noble Paul commented on SOLR-913:
---------------------------------
Is it true that you have a large no:of cores and you start/stop them very frequently and the start/stop performance is a concern for you?
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static (HttpClient object too )
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar resolved SOLR-913.
----------------------------------------
Resolution: Fixed
Committed revision 727354.
Thanks Kay!
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static (HttpClient object too )
> ---------------------------------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch, SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
> The same is true for HttpClient instances. These are one per core right now. We can make that static too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-913)
org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made
static
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656434#action_12656434 ]
Noble Paul commented on SOLR-913:
---------------------------------
bq.if there were to be another piece of code , that inadvertently - calls this piece of code once again by creating additional Pattern objects would be suicidal
there is no other piece of code currently calling it inside Solr.
There is a tradeoff of complexity Vs. perf advantage
Maintaining a large system such as Solr is hard as it is. So we have to keep in mind that we keep the code simpler. Unless there is a known issue let us not solve it now and add complexity
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-913:
---------------------------------------
Attachment: SOLR-913.patch
Attached patch changes HttpClient to static as well so that connections may be shared in cases of multiple cores.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch, SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
> The same is true for HttpClient instances. These are one per core right now. We can make that static too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated SOLR-913:
-------------------------
Attachment: SOLR-913.patch
Pattern object is a static object instead of being created locally in a function since the regex pattern object is very expensive to create.
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated SOLR-913:
-------------------------
Component/s: (was: clients - java)
replication (java)
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: replication (java)
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-913) org/apache/solr/handler/SnapPuller.java
- Expensive Pattern object made static
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated SOLR-913:
-------------------------
Component/s: (was: replication (java))
Analysis
Priority: Minor (was: Major)
> org/apache/solr/handler/SnapPuller.java - Expensive Pattern object made static
> --------------------------------------------------------------------------------
>
> Key: SOLR-913
> URL: https://issues.apache.org/jira/browse/SOLR-913
> Project: Solr
> Issue Type: Improvement
> Components: Analysis
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-913.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> In the class - org.apache.solr.handler.SnapPuller - there seems to be an expensive Pattern object created locally in the method
> static Integer readInterval(String interval) ;
> Pattern instances are better created as static objects and reused.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.