You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Bojan Vukojevic <em...@gmail.com> on 2010/08/24 07:07:27 UTC
SolrJ addField with Reader
I am using SolrJ with embedded Solr server and some documents have a lot of
text. Solr will be running on a small device with very limited memory. In my
tests I cannot process more than 3MB of text (in a body) with 64MB heap.
According to Java there is about 30MB free memory before I call server.add
and with 5MB of text it runs out of memory.
Is there a way around this?
Is there a plan to enhance SolrJ to allow a reader to be passed in instead
of a string?
thx!
b
Re: SolrJ addField with Reader
Posted by Lance Norskog <go...@gmail.com>.
There are a couple of options here. Solr can fetch text from a file or
from HTTP given an url. Look at the stream.file and stream.url
parameters. You can use these from EmbeddedSolr.
Also, there are 'ContentStream' objects in the SolrJ API which you can
also use. Look at
http://lucene.apache.org/solr/api/org/apache/solr/common/util/ContentStreamBase.FileStream.html.
The unit tests have a few examples of how to use it.
Lance
On Wed, Aug 25, 2010 at 12:43 AM, Shalin Shekhar Mangar
<sh...@gmail.com> wrote:
> On Tue, Aug 24, 2010 at 10:37 AM, Bojan Vukojevic <em...@gmail.com>wrote:
>
>> I am using SolrJ with embedded Solr server and some documents have a lot
>> of
>> text. Solr will be running on a small device with very limited memory. In
>> my
>> tests I cannot process more than 3MB of text (in a body) with 64MB heap.
>> According to Java there is about 30MB free memory before I call server.add
>> and with 5MB of text it runs out of memory.
>>
>> Is there a way around this?
>>
>> Is there a plan to enhance SolrJ to allow a reader to be passed in instead
>> of a string?
>>
>>
> Can you please open a Jira issue?
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
--
Lance Norskog
goksron@gmail.com
Re: SolrJ addField with Reader
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Aug 24, 2010 at 10:37 AM, Bojan Vukojevic <em...@gmail.com>wrote:
> I am using SolrJ with embedded Solr server and some documents have a lot
> of
> text. Solr will be running on a small device with very limited memory. In
> my
> tests I cannot process more than 3MB of text (in a body) with 64MB heap.
> According to Java there is about 30MB free memory before I call server.add
> and with 5MB of text it runs out of memory.
>
> Is there a way around this?
>
> Is there a plan to enhance SolrJ to allow a reader to be passed in instead
> of a string?
>
>
Can you please open a Jira issue?
--
Regards,
Shalin Shekhar Mangar.