You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Cook, Jeryl" <JC...@innodata-isogen.com> on 2007/02/26 20:08:40 UTC
RE: [jira] Updated: (SOLR-173) "too many open files" with posting
toupdate handler
I remember when using Lucene directly in a project, I noticed this error
only comes up if the system is deployed on UNIX boxes..in our case
SOLARIS.
You can increase the number of allowed "open files" on unix ..i forget
the command but the default is 256. We increased it, and the problem
went away.(but the patch looks to address it programmatically as well.)
Jeryl Cook
-----Original Message-----
From: Ryan McKinley (JIRA) [mailto:jira@apache.org]
Sent: Saturday, February 24, 2007 7:43 PM
To: solr-dev@lucene.apache.org
Subject: [jira] Updated: (SOLR-173) "too many open files" with posting
toupdate handler
[
https://issues.apache.org/jira/browse/SOLR-173?page=com.atlassian.jira.p
lugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley updated SOLR-173:
-------------------------------
Attachment: SOLR-173-open-files-bug.patch
modifying for problem #2 was easier then i expected, it only touches
JSONResponseWriter.java
TextResponseWriter.java
XMLWriter.java
rather then initalize the searcher in the constructor, they load a local
SolrIndexSearcher inside the writeDocList() function
> "too many open files" with posting to update handler
> ----------------------------------------------------
>
> Key: SOLR-173
> URL: https://issues.apache.org/jira/browse/SOLR-173
> Project: Solr
> Issue Type: Bug
> Components: update
> Affects Versions: 1.2
> Reporter: Ryan McKinley
> Attachments: SOLR-173-open-files-bug.patch,
SOLR-173-open-files-bug.patch, SOLR-173-open-files-bug.patch
>
>
> From brian:
> 1) Download trunk/nightly
> 2) Change line 347 of example/solr/conf/solrconfig.xml to
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler">
> 3) java -jar start.jar...
> 3) Run post.sh a bunch of times on the same xml file... (in a shell
script or whatever)
> 4) After a few seconds/minutes jetty will crash with "too many open
files"
> - - - - -
> all you've got to do is
> apache-solr-nightly/example/exampledocs ryan$ while [ 0 -lt 1 ]; do
./post.sh hd.xml; done
> with the request handler pointing to /update. Use
> # lsof | grep solr | wc -l
> to watch the fdescs fly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (SOLR-173) "too many open files" with posting toupdate handler
Posted by Ryan McKinley <ry...@gmail.com>.
This problem is different then the normal situation where you can
increase it with "ulimit -n XXX".
This is a real bug introduced by SOLR-104. As Yonik pointed out, the
filter does not close its request and when the searcher is opened for
an update (unnecessary) it hangs on to closed files indefinitely.
I think the patch on SOLR-173 is good to go - if anyone has a chance
to look at it, that would be great.
thanks
ryan
On 2/26/07, Cook, Jeryl <JC...@innodata-isogen.com> wrote:
> I remember when using Lucene directly in a project, I noticed this error
> only comes up if the system is deployed on UNIX boxes..in our case
> SOLARIS.
>
> You can increase the number of allowed "open files" on unix ..i forget
> the command but the default is 256. We increased it, and the problem
> went away.(but the patch looks to address it programmatically as well.)
>
> Jeryl Cook
>
> -----Original Message-----
> From: Ryan McKinley (JIRA) [mailto:jira@apache.org]
> Sent: Saturday, February 24, 2007 7:43 PM
> To: solr-dev@lucene.apache.org
> Subject: [jira] Updated: (SOLR-173) "too many open files" with posting
> toupdate handler
>
>
> [
> https://issues.apache.org/jira/browse/SOLR-173?page=com.atlassian.jira.p
> lugin.system.issuetabpanels:all-tabpanel ]
>
> Ryan McKinley updated SOLR-173:
> -------------------------------
>
> Attachment: SOLR-173-open-files-bug.patch
>
> modifying for problem #2 was easier then i expected, it only touches
> JSONResponseWriter.java
> TextResponseWriter.java
> XMLWriter.java
>
> rather then initalize the searcher in the constructor, they load a local
> SolrIndexSearcher inside the writeDocList() function
>
> > "too many open files" with posting to update handler
> > ----------------------------------------------------
> >
> > Key: SOLR-173
> > URL: https://issues.apache.org/jira/browse/SOLR-173
> > Project: Solr
> > Issue Type: Bug
> > Components: update
> > Affects Versions: 1.2
> > Reporter: Ryan McKinley
> > Attachments: SOLR-173-open-files-bug.patch,
> SOLR-173-open-files-bug.patch, SOLR-173-open-files-bug.patch
> >
> >
> > From brian:
> > 1) Download trunk/nightly
> > 2) Change line 347 of example/solr/conf/solrconfig.xml to
> <requestHandler name="/update" class="solr.XmlUpdateRequestHandler">
> > 3) java -jar start.jar...
> > 3) Run post.sh a bunch of times on the same xml file... (in a shell
> script or whatever)
> > 4) After a few seconds/minutes jetty will crash with "too many open
> files"
> > - - - - -
> > all you've got to do is
> > apache-solr-nightly/example/exampledocs ryan$ while [ 0 -lt 1 ]; do
> ./post.sh hd.xml; done
> > with the request handler pointing to /update. Use
> > # lsof | grep solr | wc -l
> > to watch the fdescs fly.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>