You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Solr User <so...@gmail.com> on 2011/03/03 23:02:32 UTC
Out of memory while creating indexes
Hi All,
I am trying to create indexes out of a 400MB XML file using the following
command and I am running into out of memory exception.
$JAVA_HOME/bin/java -Xms768m -Xmx1024m -*Durl*=http://$SOLR_HOST
SOLR_PORT/solr/customercarecore/update -jar
$SOLRBASEDIR/*dataconvertor*/common/lib/post.jar
$SOLRBASEDIR/dataconvertor/customercare/xml/CustomerData.xml
I am planning to bump up the memory and try again.
Did any one ran into similar issue? Any inputs would be very helpful to
resolve the out of memory exception.
I was able to create indexes with small file but not with large file. I am
not using Solr J.
Thanks,
Solr User
Re: Out of memory while creating indexes
Posted by Gora Mohanty <go...@mimirtech.com>.
On Fri, Mar 4, 2011 at 3:32 AM, Solr User <so...@gmail.com> wrote:
> Hi All,
>
> I am trying to create indexes out of a 400MB XML file using the following
> command and I am running into out of memory exception.
Is this a single record in the XML file? If it is more than one, breaking
it up into separate XML files, say one per record, should help.
> $JAVA_HOME/bin/java -Xms768m -Xmx1024m -*Durl*=http://$SOLR_HOST
> SOLR_PORT/solr/customercarecore/update -jar
> $SOLRBASEDIR/*dataconvertor*/common/lib/post.jar
> $SOLRBASEDIR/dataconvertor/customercare/xml/CustomerData.xml
>
> I am planning to bump up the memory and try again.
[...]
If you give Solr enough memory this should work, but IMHO, it would
be better to break up your input XML files if you can.
Regards,
Gora
Re: Out of memory while creating indexes
Posted by Praveen Parameswaran <bu...@gmail.com>.
Hi ,
post.sh is using curl as I see , will that be helpful?
On Fri, Mar 4, 2011 at 1:24 PM, Upayavira <uv...@odoko.co.uk> wrote:
> post.jar is intended for demo purposes, not production use, so it
> doesn;t surprise me you've managed to break it.
>
> Have you tried using curl to do the post?
>
> Upayavira
>
> On Thu, 03 Mar 2011 17:02 -0500, "Solr User" <so...@gmail.com> wrote:
> > Hi All,
> >
> > I am trying to create indexes out of a 400MB XML file using the following
> > command and I am running into out of memory exception.
> >
> > $JAVA_HOME/bin/java -Xms768m -Xmx1024m -*Durl*=http://$SOLR_HOST
> > SOLR_PORT/solr/customercarecore/update -jar
> > $SOLRBASEDIR/*dataconvertor*/common/lib/post.jar
> > $SOLRBASEDIR/dataconvertor/customercare/xml/CustomerData.xml
> >
> > I am planning to bump up the memory and try again.
> >
> > Did any one ran into similar issue? Any inputs would be very helpful to
> > resolve the out of memory exception.
> >
> > I was able to create indexes with small file but not with large file. I
> > am
> > not using Solr J.
> >
> > Thanks,
> > Solr User
> >
> ---
> Enterprise Search Consultant at Sourcesense UK,
> Making Sense of Open Source
>
>
Re: Out of memory while creating indexes
Posted by Upayavira <uv...@odoko.co.uk>.
post.jar is intended for demo purposes, not production use, so it
doesn;t surprise me you've managed to break it.
Have you tried using curl to do the post?
Upayavira
On Thu, 03 Mar 2011 17:02 -0500, "Solr User" <so...@gmail.com> wrote:
> Hi All,
>
> I am trying to create indexes out of a 400MB XML file using the following
> command and I am running into out of memory exception.
>
> $JAVA_HOME/bin/java -Xms768m -Xmx1024m -*Durl*=http://$SOLR_HOST
> SOLR_PORT/solr/customercarecore/update -jar
> $SOLRBASEDIR/*dataconvertor*/common/lib/post.jar
> $SOLRBASEDIR/dataconvertor/customercare/xml/CustomerData.xml
>
> I am planning to bump up the memory and try again.
>
> Did any one ran into similar issue? Any inputs would be very helpful to
> resolve the out of memory exception.
>
> I was able to create indexes with small file but not with large file. I
> am
> not using Solr J.
>
> Thanks,
> Solr User
>
---
Enterprise Search Consultant at Sourcesense UK,
Making Sense of Open Source