You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Gambier (JIRA)" <ji...@apache.org> on 2014/05/22 16:55:01 UTC
[jira] [Created] (SOLR-6110) DIH increase tcp connections so
full-import failed
Gambier created SOLR-6110:
-----------------------------
Summary: DIH increase tcp connections so full-import failed
Key: SOLR-6110
URL: https://issues.apache.org/jira/browse/SOLR-6110
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 4.7.2
Environment: Debian Lenny
Reporter: Gambier
Priority: Blocker
Before i used solr 4.3.1 and i had no problem. Since i use solr 4.7.2 i have a problem with the DIH.
When i do a full import with DIH the number of TCP connection increase.
For example i have 240000 documents to index and the full-import stop randomly between 6000 and 9000 docs indexed with the error :
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
With the command (while full-import is running) :
netstat | grep localdo
I saw that there are many connections in status "TIME_WAIT"
The problem is that the OS don't free those connections quickly. So the tcp connections queue increase to its limit and crash ---> full-import failed.
To solve this problem i found a solution :
If i change OS settings "/proc/sys/net/ipv4/tcp_tw_reuse" to 1 and "/proc/sys/net/ipv4/tcp_tw_recycle" to 1 my full-import works to the end.
So why connections are not closed in this new version ?
And why the java socket option SO_REUSEADDR is not used ?
I'm not a Java developer so maybe i miss something but the only way for me to do a success full-import was to use the solution above.
Thank you by advance for your answers to help me solve this problem with a better solution than mine.
Thomas
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org