Re: [jira] Updated: (SOLR-173) "too many open files" with posting toupdate handler

[Ryan McKinley <ry...@gmail.com>]

This problem is different then the normal situation where you can
increase it with "ulimit -n XXX".

This is a real bug introduced by SOLR-104.  As Yonik pointed out, the
filter does not close its request and when the searcher is opened for
an update (unnecessary) it hangs on to closed files indefinitely.

I think the patch on SOLR-173 is good to go - if anyone has a chance
to look at it, that would be great.

thanks
ryan


On 2/26/07, Cook, Jeryl <JC...@innodata-isogen.com> wrote:
> I remember when using Lucene directly in a project, I noticed this error
> only comes up if the system is deployed on UNIX boxes..in our case
> SOLARIS.
>
> You can increase the number of allowed "open files" on unix ..i forget
> the command but the default is 256.   We increased it, and the problem
> went away.(but the patch looks to address it programmatically as well.)
>
> Jeryl Cook
>
> -----Original Message-----
> From: Ryan McKinley (JIRA) [mailto:jira@apache.org]
> Sent: Saturday, February 24, 2007 7:43 PM
> To: solr-dev@lucene.apache.org
> Subject: [jira] Updated: (SOLR-173) "too many open files" with posting
> toupdate handler
>
>
>      [
> https://issues.apache.org/jira/browse/SOLR-173?page=com.atlassian.jira.p
> lugin.system.issuetabpanels:all-tabpanel ]
>
> Ryan McKinley updated SOLR-173:
> -------------------------------
>
>     Attachment: SOLR-173-open-files-bug.patch
>
> modifying for problem #2 was easier then i expected, it only touches
>  JSONResponseWriter.java
>  TextResponseWriter.java
>  XMLWriter.java
>
> rather then initalize the searcher in the constructor, they load a local
> SolrIndexSearcher inside the writeDocList() function
>
> > "too many open files" with posting to update handler
> > ----------------------------------------------------
> >
> >                 Key: SOLR-173
> >                 URL: https://issues.apache.org/jira/browse/SOLR-173
> >             Project: Solr
> >          Issue Type: Bug
> >          Components: update
> >    Affects Versions: 1.2
> >            Reporter: Ryan McKinley
> >         Attachments: SOLR-173-open-files-bug.patch,
> SOLR-173-open-files-bug.patch, SOLR-173-open-files-bug.patch
> >
> >
> > From brian:
> > 1) Download trunk/nightly
> > 2) Change line 347 of example/solr/conf/solrconfig.xml to
> <requestHandler name="/update" class="solr.XmlUpdateRequestHandler">
> > 3) java -jar start.jar...
> > 3) Run post.sh a bunch of times on the same xml file... (in a shell
> script or whatever)
> > 4) After a few seconds/minutes jetty will crash with "too many open
> files"
> > - - - - -
> > all you've got to do is
> > apache-solr-nightly/example/exampledocs ryan$ while [ 0 -lt 1 ]; do
> ./post.sh hd.xml; done
> > with the request handler pointing to /update. Use
> > # lsof | grep solr | wc -l
> > to watch the fdescs fly.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>