You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Jack Krupansky (JIRA)" <ji...@apache.org> on 2012/06/03 23:36:22 UTC
[jira] [Created] (SOLR-3504) Clearly document the limit for the
maximum number of documents in a single index
Jack Krupansky created SOLR-3504:
------------------------------------
Summary: Clearly document the limit for the maximum number of documents in a single index
Key: SOLR-3504
URL: https://issues.apache.org/jira/browse/SOLR-3504
Project: Solr
Issue Type: Improvement
Components: update
Affects Versions: 3.6
Reporter: Jack Krupansky
Priority: Minor
Although the actual limit to the number of documents supported by a Solr implementation depends on the number of shards, unless the user is intimately familiar with the implementation of Lucene, they may not realize that a single Solr index (single shard, single core) is limited to approximately 2.14 billion documents regardless of their processing power or memory. This limit should be clearly documented for the Solr user.
Granted, users should be strongly discouraged from attempting to create a single, unsharded index of that size, but they certainly should have to find out about the Lucene limit by accident.
A subsequent issue will recommend that Solr detect and appropriately report to the user when and if this limit is hit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3504) Clearly document the limit for the
maximum number of documents in a single index
Posted by "Jack Krupansky (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288266#comment-13288266 ]
Jack Krupansky commented on SOLR-3504:
--------------------------------------
Solr has a number of interfaces for adding documents, such as Update XML, Update CSV, SolrCell, Data Import Handler, SolrCloud, etc. Generally, each has a wiki page, to which the following section should be added:
"Limitations
Although a Solr implementation can scale into the billions of documents by using a number of shards, each individual shard or Solr core is limited by the Lucene limit for an index which is approximately 2.14 billion documents (2,147,483,647 to be exact) in the current implementation of Lucene. In practice, it is unlikely that such a large number of documents would fit and perform well in a single index. In extreme cases it may be possible, but in no case can the number of documents in a single index exceed that number."
This limitation could also be added to the Solr tutorial page.
There are probably a few other locations in the Solr docs when this limitation should be noted.
> Clearly document the limit for the maximum number of documents in a single index
> --------------------------------------------------------------------------------
>
> Key: SOLR-3504
> URL: https://issues.apache.org/jira/browse/SOLR-3504
> Project: Solr
> Issue Type: Improvement
> Components: update
> Affects Versions: 3.6
> Reporter: Jack Krupansky
> Priority: Minor
>
> Although the actual limit to the number of documents supported by a Solr implementation depends on the number of shards, unless the user is intimately familiar with the implementation of Lucene, they may not realize that a single Solr index (single shard, single core) is limited to approximately 2.14 billion documents regardless of their processing power or memory. This limit should be clearly documented for the Solr user.
> Granted, users should be strongly discouraged from attempting to create a single, unsharded index of that size, but they certainly should have to find out about the Lucene limit by accident.
> A subsequent issue will recommend that Solr detect and appropriately report to the user when and if this limit is hit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org