You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Shamik Bandopadhyay <sh...@gmail.com> on 2013/09/26 09:48:18 UTC
Not able to index documents using CloudSolrServer
Hi,
I've recently started exploring SolrCloud and is trying to index
documents using CloudSolrServer client. The issue I'm seeing is if I don't
fire an explicit commit on CloudSolrServer object, the documents are not
getting indexed. Here's my code snippet :
CloudSolrServer server = new CloudSolrServer("localhost:2181");
server.setDefaultCollection("collection1");
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", "http://test.com/akn/test6.html");
doc.addField("Source2", "aknsource");
doc.addField("url", "http://test.com/akn/test6.html");
doc.addField("title", "SolrCloud rocks");
doc.addField("text", "This is a sample text");
UpdateResponse resp = server.add(doc);
//UpdateResponse res = server.commit();
I've 2 shards with 1 replica each and a single zookeeper instance.
Once I run this test code, I'm able to see the request hitting the nodes.
Here's the output from the log :
INFO - 2013-09-26 03:19:04.981;
org.apache.solr.update.processor.LogUpdateProcessor; [collection1]
webapp=/solr path=/update params={distrib.from=
http://ec2-1-2-3-4.us-west-1.compute.amazonaws.com:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2}
{add=[http://test.com/akn/test6.html (1447223565945405440)]} 0 42
INFO - 2013-09-26 03:19:19.943;
org.apache.solr.update.DirectUpdateHandler2; start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO - 2013-09-26 03:19:20.249; org.apache.solr.core.SolrDeletionPolicy;
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ebs2/AutoDeskSolr44/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@36ddc581;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_7,generation=7}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ebs2/AutoDeskSolr44/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@36ddc581;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_8,generation=8}
INFO - 2013-09-26 03:19:20.250; org.apache.solr.core.SolrDeletionPolicy;
newest commit generation = 8
INFO - 2013-09-26 03:19:20.252; org.apache.solr.search.SolrIndexSearcher;
Opening Searcher@c324b85 realtime
INFO - 2013-09-26 03:19:20.254;
org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
>From the log, it looked like that the commit has gone through successfully.
But then if I query the servers, none of the entries are showing up.
Now, if I turn on UpdateResponse res = server.commit(); , I do the see the
data indexed. Here's the log :
INFO - 2013-09-26 03:41:24.433;
org.apache.solr.update.processor.LogUpdateProcessor; [collection1]
webapp=/solr path=/update params={wt=javabin&version=2} {add=[
http://autodesk.com/akn/test6.html (1447224970494083072)]} 0 12
INFO - 2013-09-26 03:41:24.490;
org.apache.solr.update.DirectUpdateHandler2; start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO - 2013-09-26 03:41:24.788; org.apache.solr.core.SolrDeletionPolicy;
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ebs2/AutoDeskSolr44/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@36ddc581;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_8,generation=8}
commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ebs2/AutoDeskSolr44/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@36ddc581;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_9,generation=9}
INFO - 2013-09-26 03:41:24.788; org.apache.solr.core.SolrDeletionPolicy;
newest commit generation = 9
INFO - 2013-09-26 03:41:24.792; org.apache.solr.search.SolrIndexSearcher;
Opening Searcher@138ba593 main
INFO - 2013-09-26 03:41:24.794;
org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
INFO - 2013-09-26 03:41:24.794; org.apache.solr.core.QuerySenderListener;
QuerySenderListener sending requests to
Searcher@138ba593main{StandardDirectoryReader(segments_9:21:nrt
_0(4.4):C1 _1(4.4):C1
_3(4.4):C1 _4(4.4):C1 _5(4.4):C1 _7(4.4):C1)}
INFO - 2013-09-26 03:41:24.795; org.apache.solr.core.QuerySenderListener;
QuerySenderListener done.
INFO - 2013-09-26 03:41:24.798; org.apache.solr.core.SolrCore;
[collection1] Registered new searcher
Searcher@138ba593main{StandardDirectoryReader(segments_9:21:nrt
_0(4.4):C1 _1(4.4):C1
_3(4.4):C1 _4(4.4):C1 _5(4.4):C1 _7(4.4):C1)}
INFO - 2013-09-26 03:41:24.798;
org.apache.solr.update.processor.LogUpdateProcessor; [collection1]
webapp=/solr path=/update
params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false}
{commit=} 0 308
Here's what I've in config file :
<autoCommit>
<maxTime>30000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>1000</maxTime>
</autoSoftCommit>
Not sure what I'm missing here. I've been using ConcurrentUpdateSolrServer
before in 4.2 , I never needed to use an explicit commit.
Any pointers will be appreciated.
Thanks,
Shamik
Re: Not able to index documents using CloudSolrServer
Posted by Erick Erickson <er...@gmail.com>.
this is key: openSearcher=false
That means your commit did, indeed, fire but no new
searcher was opened so the document remains invisible.
If you configure softCommit, that will make the doc visible
after the interval, or if you set openSearcher=true in your
commit configuration.
Or, as you noted, if you issue an explicit commit.
You might find this useful:
http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
Best,
Erick
On Thu, Sep 26, 2013 at 3:22 PM, shamik <sh...@gmail.com> wrote:
> Just an update, I finally saw the documents getting indexed. But it happened
> after 4-5 hours since I had used CloudServer to send the documents to Solr.
> Is there any configuration change required ? I've having 2 nodes with a
> replica each and a single zookeeper instance.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-index-documents-using-CloudSolrServer-tp4092074p4092238.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to index documents using CloudSolrServer
Posted by shamik <sh...@gmail.com>.
Just an update, I finally saw the documents getting indexed. But it happened
after 4-5 hours since I had used CloudServer to send the documents to Solr.
Is there any configuration change required ? I've having 2 nodes with a
replica each and a single zookeeper instance.
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-index-documents-using-CloudSolrServer-tp4092074p4092238.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Not able to index documents using CloudSolrServer
Posted by shamik <sh...@gmail.com>.
Anyone suggestion ?
--
View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-index-documents-using-CloudSolrServer-tp4092074p4092185.html
Sent from the Solr - User mailing list archive at Nabble.com.