You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by janne mattila <ja...@gmail.com> on 2012/04/03 17:46:27 UTC
Weird memory leak problem with dataimporthandler scheduling
I have implemented dataimporthandler scheduling based on
http://wiki.apache.org/solr/DataImportHandler#Scheduling. It
periodically triggers full and delta updates. I'm unpacking the
original solr.war, adding a few scheduling-related classes such as
ApplicationListener etc (I have modified the example a lot) and
repacking the web application.
The scheduling works fine, but when I undeploy solr web application,
Tomcat gives errors about ThreadLocals that were not cleared:
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [org.apache.solr.handler.dataimport.DataImporter$2] (value
[org.apache.solr.
handler.dataimport.DataImporter$2@b0e2096]) and a value of type
[java.util.concurrent.atomic.AtomicLong] (value [2]) but failed to
remove it when the web applic
ation was stopped. Threads are going to be renewed over time to try
and avoid a probable memory leak.
3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [org.apache.solr.handler.dataimport.DataImporter$3] (value
[org.apache.solr.
handler.dataimport.DataImporter$3@4c7d5d85]) and a value of type
[java.text.SimpleDateFormat] (value
[java.text.SimpleDateFormat@4f76f1a0]) but failed to remove
it when the web application was stopped. Threads are going to be
renewed over time to try and avoid a probable memory leak.
3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [java.lang.ThreadLocal] (value
[java.lang.ThreadLocal@3a86edfe]) and a value
of type [org.apache.solr.handler.dataimport.ContextImpl] (value
[org.apache.solr.handler.dataimport.ContextImpl@7072dcb6]) but failed
to remove it when the web
application was stopped. Threads are going to be renewed over time to
try and avoid a probable memory leak.
3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [org.apache.solr.schema.DateField.ThreadLocalDateFormat]
(value [org.apache.
solr.schema.DateField$ThreadLocalDateFormat@4f86a67]) and a value of
type [org.apache.solr.schema.DateField.ISO8601CanonicalDateFormat]
(value [org.apache.solr.
schema.DateField$ISO8601CanonicalDateFormat@6b2ed43a]) but failed to
remove it when the web application was stopped. Threads are going to
be renewed over time t
o try and avoid a probable memory leak.
3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [org.apache.solr.handler.dataimport.DataImporter$2] (value
[org.apache.solr.
handler.dataimport.DataImporter$2@b0e2096]) and a value of type
[java.util.concurrent.atomic.AtomicLong] (value [2]) but failed to
remove it when the web applic
ation was stopped. Threads are going to be renewed over time to try
and avoid a probable memory leak.
3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [java.lang.ThreadLocal] (value
[java.lang.ThreadLocal@3a86edfe]) and a value
of type [org.apache.solr.handler.dataimport.ContextImpl] (value
[org.apache.solr.handler.dataimport.ContextImpl@511192bd]) but failed
to remove it when the web
application was stopped. Threads are going to be renewed over time to
try and avoid a probable memory leak.
3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [org.apache.solr.handler.dataimport.DataImporter$3] (value
[org.apache.solr.
handler.dataimport.DataImporter$3@4c7d5d85]) and a value of type
[java.text.SimpleDateFormat] (value
[java.text.SimpleDateFormat@4f76f1a0]) but failed to remove
it when the web application was stopped. Threads are going to be
renewed over time to try and avoid a probable memory leak.
3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks
SEVERE: The web application [/my-solr] created a ThreadLocal with key
of type [org.apache.solr.schema.DateField.ThreadLocalDateFormat]
(value [org.apache.
solr.schema.DateField$ThreadLocalDateFormat@4f86a67]) and a value of
type [org.apache.solr.schema.DateField.ISO8601CanonicalDateFormat]
(value [org.apache.solr.
schema.DateField$ISO8601CanonicalDateFormat@6b2ed43a]) but failed to
remove it when the web application was stopped. Threads are going to
be renewed over time t
o try and avoid a probable memory leak.
I have rechecked my code to make sure it should not have any memory
leaks. I have identified the cause to method:
private void sendHttpPost(String completeUrl, String coreName) {
HttpClient client = new HttpClient();
PostMethod method = new PostMethod(completeUrl);
try {
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
System.err.println("Method failed: " + method.getStatusLine());
}
method.getResponseBody();
} catch (HttpException e) {
System.err.println("Fatal protocol violation: " + e.getMessage());
e.printStackTrace();
} catch (IOException e) {
System.err.println("Fatal transport error: " + e.getMessage());
e.printStackTrace();
} finally {
// Release the connection.
method.releaseConnection();
}
}
If I comment out that code, the errors about memory leaks disappear.
This method simply starts solr indexing operation by calling localhost
url such as my-solr/mycore/dataimport?command=full-import
The method is rewritten to use HttpClient but I got the same mem leak
errors when I used the original HTTPPostScheduler class from
http://wiki.apache.org/solr/DataImportHandler#Scheduling
So it looks like the mem leak errors do not come from my added code
but from original Solr code (which receives the http requests).
What is even more confusing is that if I disable the sendHttpPost(),
the errors disappear as I mentioned - but if I manually request the
same urls to initiate the full / delta imports, the mem leak errors do
not show up when webapp is undeployed....??
Re: Weird memory leak problem with dataimporthandler scheduling
Posted by janne mattila <ja...@gmail.com>.
OK. Just typing out the question fixed it.
Changing from post to get:
GetMethod method = new GetMethod(completeUrl);
removed the errors. The reason, I cannot explain...
On Tue, Apr 3, 2012 at 6:46 PM, janne mattila
<ja...@gmail.com> wrote:
> I have implemented dataimporthandler scheduling based on
> http://wiki.apache.org/solr/DataImportHandler#Scheduling. It
> periodically triggers full and delta updates. I'm unpacking the
> original solr.war, adding a few scheduling-related classes such as
> ApplicationListener etc (I have modified the example a lot) and
> repacking the web application.
>
> The scheduling works fine, but when I undeploy solr web application,
> Tomcat gives errors about ThreadLocals that were not cleared:
>
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [org.apache.solr.handler.dataimport.DataImporter$2] (value
> [org.apache.solr.
> handler.dataimport.DataImporter$2@b0e2096]) and a value of type
> [java.util.concurrent.atomic.AtomicLong] (value [2]) but failed to
> remove it when the web applic
> ation was stopped. Threads are going to be renewed over time to try
> and avoid a probable memory leak.
> 3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
> checkThreadLocalMapForLeaks
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [org.apache.solr.handler.dataimport.DataImporter$3] (value
> [org.apache.solr.
> handler.dataimport.DataImporter$3@4c7d5d85]) and a value of type
> [java.text.SimpleDateFormat] (value
> [java.text.SimpleDateFormat@4f76f1a0]) but failed to remove
> it when the web application was stopped. Threads are going to be
> renewed over time to try and avoid a probable memory leak.
> 3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
> checkThreadLocalMapForLeaks
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [java.lang.ThreadLocal] (value
> [java.lang.ThreadLocal@3a86edfe]) and a value
> of type [org.apache.solr.handler.dataimport.ContextImpl] (value
> [org.apache.solr.handler.dataimport.ContextImpl@7072dcb6]) but failed
> to remove it when the web
> application was stopped. Threads are going to be renewed over time to
> try and avoid a probable memory leak.
> 3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
> checkThreadLocalMapForLeaks
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [org.apache.solr.schema.DateField.ThreadLocalDateFormat]
> (value [org.apache.
> solr.schema.DateField$ThreadLocalDateFormat@4f86a67]) and a value of
> type [org.apache.solr.schema.DateField.ISO8601CanonicalDateFormat]
> (value [org.apache.solr.
> schema.DateField$ISO8601CanonicalDateFormat@6b2ed43a]) but failed to
> remove it when the web application was stopped. Threads are going to
> be renewed over time t
> o try and avoid a probable memory leak.
> 3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
> checkThreadLocalMapForLeaks
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [org.apache.solr.handler.dataimport.DataImporter$2] (value
> [org.apache.solr.
> handler.dataimport.DataImporter$2@b0e2096]) and a value of type
> [java.util.concurrent.atomic.AtomicLong] (value [2]) but failed to
> remove it when the web applic
> ation was stopped. Threads are going to be renewed over time to try
> and avoid a probable memory leak.
> 3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
> checkThreadLocalMapForLeaks
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [java.lang.ThreadLocal] (value
> [java.lang.ThreadLocal@3a86edfe]) and a value
> of type [org.apache.solr.handler.dataimport.ContextImpl] (value
> [org.apache.solr.handler.dataimport.ContextImpl@511192bd]) but failed
> to remove it when the web
> application was stopped. Threads are going to be renewed over time to
> try and avoid a probable memory leak.
> 3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
> checkThreadLocalMapForLeaks
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [org.apache.solr.handler.dataimport.DataImporter$3] (value
> [org.apache.solr.
> handler.dataimport.DataImporter$3@4c7d5d85]) and a value of type
> [java.text.SimpleDateFormat] (value
> [java.text.SimpleDateFormat@4f76f1a0]) but failed to remove
> it when the web application was stopped. Threads are going to be
> renewed over time to try and avoid a probable memory leak.
> 3.4.2012 18:36:49 org.apache.catalina.loader.WebappClassLoader
> checkThreadLocalMapForLeaks
> SEVERE: The web application [/my-solr] created a ThreadLocal with key
> of type [org.apache.solr.schema.DateField.ThreadLocalDateFormat]
> (value [org.apache.
> solr.schema.DateField$ThreadLocalDateFormat@4f86a67]) and a value of
> type [org.apache.solr.schema.DateField.ISO8601CanonicalDateFormat]
> (value [org.apache.solr.
> schema.DateField$ISO8601CanonicalDateFormat@6b2ed43a]) but failed to
> remove it when the web application was stopped. Threads are going to
> be renewed over time t
> o try and avoid a probable memory leak.
>
> I have rechecked my code to make sure it should not have any memory
> leaks. I have identified the cause to method:
>
> private void sendHttpPost(String completeUrl, String coreName) {
> HttpClient client = new HttpClient();
> PostMethod method = new PostMethod(completeUrl);
> try {
> int statusCode = client.executeMethod(method);
> if (statusCode != HttpStatus.SC_OK) {
> System.err.println("Method failed: " + method.getStatusLine());
> }
> method.getResponseBody();
> } catch (HttpException e) {
> System.err.println("Fatal protocol violation: " + e.getMessage());
> e.printStackTrace();
> } catch (IOException e) {
> System.err.println("Fatal transport error: " + e.getMessage());
> e.printStackTrace();
> } finally {
> // Release the connection.
> method.releaseConnection();
> }
> }
>
> If I comment out that code, the errors about memory leaks disappear.
> This method simply starts solr indexing operation by calling localhost
> url such as my-solr/mycore/dataimport?command=full-import
>
> The method is rewritten to use HttpClient but I got the same mem leak
> errors when I used the original HTTPPostScheduler class from
> http://wiki.apache.org/solr/DataImportHandler#Scheduling
>
> So it looks like the mem leak errors do not come from my added code
> but from original Solr code (which receives the http requests).
>
> What is even more confusing is that if I disable the sendHttpPost(),
> the errors disappear as I mentioned - but if I manually request the
> same urls to initiate the full / delta imports, the mem leak errors do
> not show up when webapp is undeployed....??