You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by longsan <lo...@sina.com> on 2014/01/21 11:37:12 UTC
CloudSolrServer has thread safe issue?
Hi ,
i'm using SolrJ to do some indexing work with CloudSolrServer class. It's
strange that when i start several threads (each thread add 10000 documents)
to add documents, the result is just only 10000 can be indexed finally. But
if i change the thread num as 1, everything is ok. Even if it's same as i
start seveval processes at same time.
i use solr 4.6.0.
Here is some code:
public class SolrUpdater implements Runnable {
private final UpdateRequest req = new UpdateRequest();
private final CloudSolrServer solr;
private final AtomicLong id;
public SolrUpdater(CloudSolrServer solr, AtomicLong id) {
this.solr = solr;
this.id = id;
}
@Override
public void run() {
List<SolrInputDocument> docList = buildDocList();
System.out.println(String.format("Start to send updater%d", id.get()));
long start = System.currentTimeMillis();
try {
UpdateResponse rep = solr.add(docList);
System.out.println(rep.toString());
} catch (SolrServerException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
long end = System.currentTimeMillis();
System.out.println(String.format("End to send updater%d, time is [%d]",
id.get(), end - start));
}
}
--
View this message in context: http://lucene.472066.n3.nabble.com/CloudSolrServer-has-thread-safe-issue-tp4112423.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: CloudSolrServer has thread safe issue?
Posted by longsan <lo...@sina.com>.
Thanks. You are right. It's the key.
--
View this message in context: http://lucene.472066.n3.nabble.com/CloudSolrServer-has-thread-safe-issue-tp4112423p4112618.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: CloudSolrServer has thread safe issue?
Posted by Erick Erickson <er...@gmail.com>.
I suspect that each thread is indexing the _same_ 10,000 documents,
and any document with the same <uniqueKey> will replace earlier
docs with the same ID. If that is true, on the admin page you
should see numDocs as 10,000 and maxDoc as (number of threads) * 10,000
Best
Erick
On Tue, Jan 21, 2014 at 5:37 AM, longsan <lo...@sina.com> wrote:
> Hi ,
>
> i'm using SolrJ to do some indexing work with CloudSolrServer class. It's
> strange that when i start several threads (each thread add 10000 documents)
> to add documents, the result is just only 10000 can be indexed finally. But
> if i change the thread num as 1, everything is ok. Even if it's same as i
> start seveval processes at same time.
>
> i use solr 4.6.0.
>
> Here is some code:
>
> public class SolrUpdater implements Runnable {
> private final UpdateRequest req = new UpdateRequest();
> private final CloudSolrServer solr;
> private final AtomicLong id;
>
> public SolrUpdater(CloudSolrServer solr, AtomicLong id) {
> this.solr = solr;
> this.id = id;
> }
>
> @Override
> public void run() {
> List<SolrInputDocument> docList = buildDocList();
> System.out.println(String.format("Start to send updater%d", id.get()));
> long start = System.currentTimeMillis();
>
> try {
> UpdateResponse rep = solr.add(docList);
> System.out.println(rep.toString());
> } catch (SolrServerException e) {
> e.printStackTrace();
> } catch (IOException e) {
> e.printStackTrace();
> }
>
> long end = System.currentTimeMillis();
> System.out.println(String.format("End to send updater%d, time is [%d]",
> id.get(), end - start));
> }
> }
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/CloudSolrServer-has-thread-safe-issue-tp4112423.html
> Sent from the Solr - User mailing list archive at Nabble.com.