You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Quan Nguyen Anh <b3...@gmail.com> on 2010/03/08 15:09:25 UTC
Import database
Hi,
I have started using Solr. I had a problem when I insert a database with
2 million rows . I hav
The server encounters error: java.lang.OutOfMemoryError: Java heap space
I searched around but can't find the solution.
Any hep regarding this will be appreciated.
Thanks in advance
Re: Import database
Posted by Quan Nguyen Anh <b3...@gmail.com>.
On 3/8/2010 11:05 PM, Shawn Heisey wrote:
> What database are you using? Many of the JDBC drivers try to pull the
> entire resultset into RAM before feeding it to the application that
> requested the data. If it's MySQL, I can show you how to fix it. The
> batchSize parameter below tells it to stream the data rather than
> buffer it. With other databases, I don't know how to do this.
>
> <dataConfig>
> <dataSource type="JdbcDataSource"
> driver="com.mysql.jdbc.Driver"
> encoding="UTF-8"
>
> url="jdbc:mysql://[SERVER]:3306/[SCHEMA]?zeroDateTimeBehavior=convertToNull"
>
> batchSize="-1"
> user="[REMOVED]"
> password="[REMOVED]"/>
> <document>
> <entity name="[TABLE]"
> query="select * from [TABLE] where (did mod 6) = 0">
> </entity>
> </document>
> </dataConfig>
>
> Shawn
>
>
> On 3/8/2010 7:09 AM, Quan Nguyen Anh wrote:
>> Hi,
>> I have started using Solr. I had a problem when I insert a database
>> with 2 million rows . I hav
>> The server encounters error: java.lang.OutOfMemoryError: Java heap space
>> I searched around but can't find the solution.
>> Any hep regarding this will be appreciated.
>> Thanks in advance
I 'm using MySQL.
This solution fixed my problem. Thanks for your help .
Re: Import database
Posted by Quan Nguyen Anh <b3...@gmail.com>.
On 3/8/2010 11:05 PM, Shawn Heisey wrote:
> What database are you using? Many of the JDBC drivers try to pull the
> entire resultset into RAM before feeding it to the application that
> requested the data. If it's MySQL, I can show you how to fix it. The
> batchSize parameter below tells it to stream the data rather than
> buffer it. With other databases, I don't know how to do this.
>
> <dataConfig>
> <dataSource type="JdbcDataSource"
> driver="com.mysql.jdbc.Driver"
> encoding="UTF-8"
>
> url="jdbc:mysql://[SERVER]:3306/[SCHEMA]?zeroDateTimeBehavior=convertToNull"
>
> batchSize="-1"
> user="[REMOVED]"
> password="[REMOVED]"/>
> <document>
> <entity name="[TABLE]"
> query="select * from [TABLE] where (did mod 6) = 0">
> </entity>
> </document>
> </dataConfig>
>
> Shawn
>
>
> On 3/8/2010 7:09 AM, Quan Nguyen Anh wrote:
>> Hi,
>> I have started using Solr. I had a problem when I insert a database
>> with 2 million rows . I hav
>> The server encounters error: java.lang.OutOfMemoryError: Java heap space
>> I searched around but can't find the solution.
>> Any hep regarding this will be appreciated.
>> Thanks in advance
I 'm using MySQL.
This solution fixed my problem. Thanks for your help .
Re: Import database
Posted by Shawn Heisey <so...@elyograg.org>.
What database are you using? Many of the JDBC drivers try to pull the
entire resultset into RAM before feeding it to the application that
requested the data. If it's MySQL, I can show you how to fix it. The
batchSize parameter below tells it to stream the data rather than buffer
it. With other databases, I don't know how to do this.
<dataConfig>
<dataSource type="JdbcDataSource"
driver="com.mysql.jdbc.Driver"
encoding="UTF-8"
url="jdbc:mysql://[SERVER]:3306/[SCHEMA]?zeroDateTimeBehavior=convertToNull"
batchSize="-1"
user="[REMOVED]"
password="[REMOVED]"/>
<document>
<entity name="[TABLE]"
query="select * from [TABLE] where (did mod 6) = 0">
</entity>
</document>
</dataConfig>
Shawn
On 3/8/2010 7:09 AM, Quan Nguyen Anh wrote:
> Hi,
> I have started using Solr. I had a problem when I insert a database
> with 2 million rows . I hav
> The server encounters error: java.lang.OutOfMemoryError: Java heap space
> I searched around but can't find the solution.
> Any hep regarding this will be appreciated.
> Thanks in advance
>
>
>
Re: Import database
Posted by Quan Nguyen Anh <b3...@gmail.com>.
On 3/8/2010 9:21 PM, Lee Smith wrote:
> I had same issue with Jetty
>
> Adding extra memory resolved my issue ie: java -Xms=512M -Xmx=1024M -jar start.jar
>
> Its in the manual, but cant seem to find the link
>
>
> On 8 Mar 2010, at 14:09, Quan Nguyen Anh wrote:
>
>
>> Hi,
>> I have started using Solr. I had a problem when I insert a database with 2 million rows . I hav
>> The server encounters error: java.lang.OutOfMemoryError: Java heap space
>> I searched around but can't find the solution.
>> Any hep regarding this will be appreciated.
>> Thanks in advance
>>
>>
>>
>>
>
>
I tried this solution but the server has the same error. I think that
the heap size is not large enough to contain data from mysql.
Re: Import database
Posted by Lee Smith <le...@weblee.co.uk>.
I had same issue with Jetty
Adding extra memory resolved my issue ie: java -Xms=512M -Xmx=1024M -jar start.jar
Its in the manual, but cant seem to find the link
On 8 Mar 2010, at 14:09, Quan Nguyen Anh wrote:
> Hi,
> I have started using Solr. I had a problem when I insert a database with 2 million rows . I hav
> The server encounters error: java.lang.OutOfMemoryError: Java heap space
> I searched around but can't find the solution.
> Any hep regarding this will be appreciated.
> Thanks in advance
>
>
>