You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org> on 2014/01/23 11:58:39 UTC
[jira] [Commented] (SOLR-5658) commitWithin does not reflect the
new documents added
[ https://issues.apache.org/jira/browse/SOLR-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13879844#comment-13879844 ]
Shalin Shekhar Mangar commented on SOLR-5658:
---------------------------------------------
Thanks for reporting this Varun.
The thing to note between the 4.5 logs and the 4.6.x logs is that in 4.5, there are two commit statements per node (1 soft commit with openSearcher=true and another hard commit with openSearcher=false) whereas in 4.6.x there is only one commit (hard commit with openSearcher=false). We need to find out why.
> commitWithin does not reflect the new documents added
> -----------------------------------------------------
>
> Key: SOLR-5658
> URL: https://issues.apache.org/jira/browse/SOLR-5658
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.6, 5.0
> Reporter: Varun Thacker
> Assignee: Shalin Shekhar Mangar
>
> I start 4 nodes using the setup mentioned on - https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
>
> I added a document using -
> curl http://localhost:8983/solr/update?commitWithin=10000 -H "Content-Type: text/xml" --data-binary '<add><doc><field name="id">testdoc</field></doc></add>'
> In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit with openSearcher=false
> In Solr 4.6.x there is there is only one commit hard commit with openSearcher=false
>
> So even after 10 seconds queries on none of the shards reflect the added document.
> This was also reported on the solr-user list ( http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html )
> Here are the relevant logs
> Logs from Solr 4.5.1
> Node 1:
> {code}
> 420021 [qtp619011445-12] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 45
> {code}
>
> Node 2:
> {code}
> 119896 [qtp1608701025-10] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458003295513608192)]} 0 348
> 129648 [commitScheduler-8-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> 129679 [commitScheduler-8-thread-1] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@e174f70 main
> 129680 [commitScheduler-8-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> 129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@e174f70 main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
> 129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener done.
> 129681 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – [collection1] Registered new searcher Searcher@e174f70 main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
> 134648 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
> 134658 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – newest commit generation = 4
> 134660 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
>
> Node 3:
>
> Node 4:
> {code}
> 374545 [qtp1608701025-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458002133233172480)]} 0 20
> 384545 [commitScheduler-8-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> 384552 [commitScheduler-8-thread-1] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@36137e08 main
> 384553 [commitScheduler-8-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> 384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@36137e08 main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
> 384553 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener done.
> 384554 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – [collection1] Registered new searcher Searcher@36137e08 main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
> 389545 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 389549 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
> 389550 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – newest commit generation = 3
> 389551 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
>
>
> Using Solr 4.6
>
> Node 1:
> {code}
> 124513 [qtp1314570047-13] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 348
> {code}
> Node 2:
> {code}
> 101586 [qtp1608701025-13] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458003613357965312)]} 0 217
> 116407 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 116429 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> 116430 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – newest commit generation = 2
> 116444 [commitScheduler-7-thread-1] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@75e32318 realtime
> 116445 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
>
> Node 3:
>
>
> Node 4:
> {code}
> 68183 [qtp1338008566-14] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458003613357965312)]} 0 43
> 83183 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 83207 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> 83208 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – newest commit generation = 2
> 83220 [commitScheduler-7-thread-1] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@326f944c realtime
> 83220 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
>
> Using Solr 4.6.1
>
> Node 1:
> {code}
> 301363 [qtp619011445-15] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 32
> {code}
> Node 2:
> {code}
> 207000 [qtp619011445-17] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458004563169640448)]} 0 28
> 221974 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 221987 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
> 221987 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – newest commit generation = 3
> 221989 [commitScheduler-7-thread-1] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@132713fa realtime
> 221990 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
> Node 3:
>
> Node 4:
> {code}
> 193133 [qtp1608701025-16] INFO org.apache.solr.update.processor.LogUpdateProcessor – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458004563169640448)]} 0 23
> 208133 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 208141 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
> 208141 [commitScheduler-7-thread-1] INFO org.apache.solr.core.SolrCore – newest commit generation = 3
> 208144 [commitScheduler-7-thread-1] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@3171c7df realtime
> 208146 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org