You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2014/06/16 19:00:05 UTC
[jira] [Updated] (SOLR-6162) Softcommit does not open a new
searcher if nrtMode is false
[ https://issues.apache.org/jira/browse/SOLR-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-6162:
---------------------------
Description:
We just upgraded from 4.2 to 4.8.1 and now run into an issue where added documents are not visible after doing a soft commit. Seems to be introduced by SOLR-5783.
Problem manifests if conig explicitly sets nrtMode to false
The setup of the index is fairly simple; single core, no cloud, no replication, java program that adds a few rows, concludes with a softcommit, hardcommit is done via autoCommit few times an hour. Another (similar) index that uses autoSoftCommit instead of softcommitting via code has the same issue. Here is a snippet of the log:
{code}
07:34:01.867 INFO [qtp1520582600-1445 ] : o.a.s.u.processor.LogUpdateProcessor - [PRD_KlwrDocAttrs] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id-04114dd2-339e-4d2f-bb9f-539415799b1a (1470681570716155904), id-05eac67d-4f84-4c6f-b78f-714fe31eab40 (1470681570716155905), id-07559ff2-246c-4fde-9b97-0c5926348f62 (1470681570716155906), id-07cc0feb-b92d-41b8-9755-7677c9589af8 (1470681570716155907), id-11214665-9640-459d-804b-d9e3b069e8c2 (1470681570716155908), id-1147abeb-3dd5-4cda-b469-a99ebce913b4 (1470681570716155909), id-143f50b4-c195-480e-ab44-0ee4e7fbdc5c (1470681570716155910), id-14ad5b3e-1558-4bc6-a7a7-d2f03d1cd040 (1470681570717204480), id-168e810c-9a75-4e4f-84f4-946866cf0d26 (1470681570717204481), id-16de425b-0cd8-4bcd-98c1-6699c3de5112 (1470681570717204482), ... (87 adds)]} 0 264
07:34:01.869 INFO [qtp1520582600-1445 ] : org.apache.solr.update.UpdateHandler - start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
07:34:01.872 INFO [qtp1520582600-1445 ] : org.apache.solr.core.SolrCore - SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
07:34:01.872 INFO [qtp1520582600-1445 ] : org.apache.solr.update.UpdateHandler - end_commit_flush
07:34:01.873 INFO [qtp1520582600-1445 ] : o.a.s.u.processor.LogUpdateProcessor - [PRD_KlwrDocAttrs] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=true} {commit=} 0 4
0
{code}
was:
We just upgraded from 4.2 to 4.8.1 and now run into an issue where added documents are not visible after doing a soft commit. Seems to be introduced by SOLR-5783. The setup of the index is fairly simple; single core, no cloud, no replication, java program that adds a few rows, concludes with a softcommit, hardcommit is done via autoCommit few times an hour. Another (similar) index that uses autoSoftCommit instead of softcommitting via code has the same issue. Here is a snippet of the log:
{code}
07:34:01.867 INFO [qtp1520582600-1445 ] : o.a.s.u.processor.LogUpdateProcessor - [PRD_KlwrDocAttrs] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id-04114dd2-339e-4d2f-bb9f-539415799b1a (1470681570716155904), id-05eac67d-4f84-4c6f-b78f-714fe31eab40 (1470681570716155905), id-07559ff2-246c-4fde-9b97-0c5926348f62 (1470681570716155906), id-07cc0feb-b92d-41b8-9755-7677c9589af8 (1470681570716155907), id-11214665-9640-459d-804b-d9e3b069e8c2 (1470681570716155908), id-1147abeb-3dd5-4cda-b469-a99ebce913b4 (1470681570716155909), id-143f50b4-c195-480e-ab44-0ee4e7fbdc5c (1470681570716155910), id-14ad5b3e-1558-4bc6-a7a7-d2f03d1cd040 (1470681570717204480), id-168e810c-9a75-4e4f-84f4-946866cf0d26 (1470681570717204481), id-16de425b-0cd8-4bcd-98c1-6699c3de5112 (1470681570717204482), ... (87 adds)]} 0 264
07:34:01.869 INFO [qtp1520582600-1445 ] : org.apache.solr.update.UpdateHandler - start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
07:34:01.872 INFO [qtp1520582600-1445 ] : org.apache.solr.core.SolrCore - SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
07:34:01.872 INFO [qtp1520582600-1445 ] : org.apache.solr.update.UpdateHandler - end_commit_flush
07:34:01.873 INFO [qtp1520582600-1445 ] : o.a.s.u.processor.LogUpdateProcessor - [PRD_KlwrDocAttrs] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=true} {commit=} 0 4
0
{code}
Summary: Softcommit does not open a new searcher if nrtMode is false (was: Softcommit does not open a new searcher after adding docs)
tweaked issue summary/description for posteity to make it clear what combination of thigns causes bug.
Pieter: your process/explanation of how you got into this situation makes total sense -- part of my initial confusion was in not realizing/remembering that softCommit+openSearcher even existed prior to nrtMode. (off the top of my head, i can't even remember what that did under the covers before we had an nrtMode to open the searcher directly off the writer)
obviously we'll need to get to the bottom of what the "Right" thing to do is moving forward, but it's good to know that removing nrtMode=false from your configs seems to be working with your master->slave setup and gives us a viable workarround in the mean time.
if you run into any other problems using this setup, please file new bugs, but make sure to reference this one as well to help keep in mind that it may in fact be a replication+nrtMode+softcommit related issue.
> Softcommit does not open a new searcher if nrtMode is false
> -----------------------------------------------------------
>
> Key: SOLR-6162
> URL: https://issues.apache.org/jira/browse/SOLR-6162
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.8
> Reporter: Pieter
> Attachments: KlwrDocAttrs.zip
>
>
> We just upgraded from 4.2 to 4.8.1 and now run into an issue where added documents are not visible after doing a soft commit. Seems to be introduced by SOLR-5783.
> Problem manifests if conig explicitly sets nrtMode to false
> The setup of the index is fairly simple; single core, no cloud, no replication, java program that adds a few rows, concludes with a softcommit, hardcommit is done via autoCommit few times an hour. Another (similar) index that uses autoSoftCommit instead of softcommitting via code has the same issue. Here is a snippet of the log:
> {code}
> 07:34:01.867 INFO [qtp1520582600-1445 ] : o.a.s.u.processor.LogUpdateProcessor - [PRD_KlwrDocAttrs] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id-04114dd2-339e-4d2f-bb9f-539415799b1a (1470681570716155904), id-05eac67d-4f84-4c6f-b78f-714fe31eab40 (1470681570716155905), id-07559ff2-246c-4fde-9b97-0c5926348f62 (1470681570716155906), id-07cc0feb-b92d-41b8-9755-7677c9589af8 (1470681570716155907), id-11214665-9640-459d-804b-d9e3b069e8c2 (1470681570716155908), id-1147abeb-3dd5-4cda-b469-a99ebce913b4 (1470681570716155909), id-143f50b4-c195-480e-ab44-0ee4e7fbdc5c (1470681570716155910), id-14ad5b3e-1558-4bc6-a7a7-d2f03d1cd040 (1470681570717204480), id-168e810c-9a75-4e4f-84f4-946866cf0d26 (1470681570717204481), id-16de425b-0cd8-4bcd-98c1-6699c3de5112 (1470681570717204482), ... (87 adds)]} 0 264
> 07:34:01.869 INFO [qtp1520582600-1445 ] : org.apache.solr.update.UpdateHandler - start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> 07:34:01.872 INFO [qtp1520582600-1445 ] : org.apache.solr.core.SolrCore - SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
> 07:34:01.872 INFO [qtp1520582600-1445 ] : org.apache.solr.update.UpdateHandler - end_commit_flush
> 07:34:01.873 INFO [qtp1520582600-1445 ] : o.a.s.u.processor.LogUpdateProcessor - [PRD_KlwrDocAttrs] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=true} {commit=} 0 4
> 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org