You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jorge Moreira <j....@gmail.com> on 2014/08/28 10:49:51 UTC
Indexing documents with ContentStreamUpdateRequest (SolrJ) asynchronously
I am using SolrJ API 4.8 to index rich documents to solr. But i want
to index these documents asynchronously. The function that I made send
documents synchronously but i don't know how to change it to make it
asynchronously. Any idea?
Function:
public Boolean indexDocument(HttpSolrServer server, String PathFile,
InputReader external)
{
ContentStreamUpdateRequest up = new
ContentStreamUpdateRequest("/update/extract");
try {
up.addFile(new File(PathFile), "text");
} catch (IOException e) {
Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null,
e);
return false;
}
up.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true);
try {
server.request(up);
} catch (SolrServerException e) {
Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null,
e);
return false;
} catch (IOException e) {
Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null,
e);
return false;
}
return true;
}
Solr server: version 4.8.
RE: Indexing documents with ContentStreamUpdateRequest (SolrJ)
asynchronously
Posted by Tomer Levi <To...@nice.com>.
Hi Jorge,
In my indexing code I've created the following Callable class:
public class IndexerThread implements Callable< UpdateResponse > {
private SolrServer solrServer;
private Collection<SolrInputDocument> documentsToIndex;
public IndexerThread(SolrServer solr, Collection<SolrInputDocument> documentsToIndex){
this.solrServer = solr;
this. documentsToIndex = documentsToIndex;
}
@Override
public UpdateResponse call() throws Exception {
return solr.add(documentsToIndex);
}
}
Then I used the code below to create threads:
SolrServer solrServer = createSolrServer();
List<Callable<Long>> threads = createThreads(solrServer, documentsToIndex);
ExecutorService executor = Executors.newFixedThreadPool(threads.size());
List<Future<Long>> futures = executor.invokeAll(threads);
while (true){
int complete = 0;
for(Future<Long> result : futures){
if(result.isDone()) {
UpdateResponse resp = result.get();
//do something with the response
}
}
if(complete == threads.size()) {
break;
}else Thread.sleep(1000);
}
Hope it helps,
Tomer
-----Original Message-----
From: Jorge Moreira [mailto:j.moreira222@gmail.com]
Sent: Thursday, August 28, 2014 11:50 AM
To: solr-user@lucene.apache.org
Subject: Indexing documents with ContentStreamUpdateRequest (SolrJ) asynchronously
I am using SolrJ API 4.8 to index rich documents to solr. But i want to index these documents asynchronously. The function that I made send documents synchronously but i don't know how to change it to make it asynchronously. Any idea?
Function:
public Boolean indexDocument(HttpSolrServer server, String PathFile, InputReader external) {
ContentStreamUpdateRequest up = new ContentStreamUpdateRequest("/update/extract");
try {
up.addFile(new File(PathFile), "text");
} catch (IOException e) {
Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
return false;
}
up.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true);
try {
server.request(up);
} catch (SolrServerException e) {
Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
return false;
} catch (IOException e) {
Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
return false;
}
return true;
}
Solr server: version 4.8.