You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Cassandra Targett (JIRA)" <ji...@apache.org> on 2018/01/11 20:19:00 UTC
[jira] [Updated] (SOLR-6065) Solr should give you clear error if
you try to add too many docs
[ https://issues.apache.org/jira/browse/SOLR-6065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cassandra Targett updated SOLR-6065:
------------------------------------
Component/s: update
> Solr should give you clear error if you try to add too many docs
> ----------------------------------------------------------------
>
> Key: SOLR-6065
> URL: https://issues.apache.org/jira/browse/SOLR-6065
> Project: Solr
> Issue Type: Bug
> Components: update
> Reporter: Hoss Man
>
> LUCENE-5843 recently fixed IndexWriter to throw a specific exception in the event that an index grows too big, preventing people from creating indexes they can't later search. That issue also added a new test-only setter that can be used to lower the limit for hte purposes of testing.
> For the Solr side of things, we should primarily focus on making sure we have some cloud based tests that set the lower limit and then verify that users get clean error messages when they exceed it for a single shard.
> {panel:title=original bug report}
> yamazaki reported an error on solr-user where, on opening a new searcher, he got an IAE from BaseCompositeReader because the numDocs was greater then Integer.MAX_VALUE.
> I'm surprised that in a straight forward setup (ie: no "AddIndex" merging) IndexWriter will even let you add more docs then max int. We should investigate if this makes sense and either add logic in IndexWriter to prevent this from happening, or add logic to Solr's UpdateHandler to prevent things from getting that far.
> ie: we should be failing to "add" too many documents, and leaving the index usable -- not accepting the add and leaving hte index in an unusable state.
> stack trace reported by user...
> {noformat}
> ERROR org.apache.solr.core.CoreContainer – Unable to create core: collection1
> org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:821)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:618)
> at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
> at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
> at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1438)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1550)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
> ... 13 more
> Caused by: org.apache.solr.common.SolrException: Error opening Reader
> at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172)
> at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:183)
> at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:179)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1414)
> ... 15 more
> Caused by: java.lang.IllegalArgumentException: Too many documents,
> composite IndexReaders cannot exceed 2147483647
> at org.apache.lucene.index.BaseCompositeReader.<init>(BaseCompositeReader.java:77)
> at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:368)
> at org.apache.lucene.index.StandardDirectoryReader.<init>(StandardDirectoryReader.java:42)
> at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:71)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
> at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88)
> at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
> at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169)
> ... 18 more
> {noformat}
> {panel}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org