You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Dmitry Kan (Jira)" <ji...@apache.org> on 2021/04/08 18:56:00 UTC
[jira] [Commented] (SOLR-4903) Solr sends all doc ids to all shards
in the query counting facets
[ https://issues.apache.org/jira/browse/SOLR-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317427#comment-17317427 ]
Dmitry Kan commented on SOLR-4903:
----------------------------------
[~ctargett] thanks for bringing this up. I believe, the issue was a result of several components:
1. we were using Apache Tomcat to deploy Solr
2. deployment was done using manual sharding, no SolrCloud
3. the shards had tens of millions or even hundreds of millions of small documents
This setup no longer exists: the system has moved to SolrCloud, under k8s with completely overhauled way of document storage model. So from where I sit -- this is no longer an issue. Would you like me to close it?
> Solr sends all doc ids to all shards in the query counting facets
> -----------------------------------------------------------------
>
> Key: SOLR-4903
> URL: https://issues.apache.org/jira/browse/SOLR-4903
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 3.4, 4.3, 4.3.1
> Reporter: Dmitry Kan
> Priority: Major
>
> Setup: front end solr and shards.
> Summary: solr frontend sends all doc ids received from QueryComponent to all shards which causes POST request buffer size overflow.
> Symptoms:
> The query is: http://pastebin.com/0DndK1Cs
> I have omitted the shards parameter.
> The router log: http://pastebin.com/FTVH1WF3
> Notice the port of a shard, that is affected. That port changes all the time, even for the same request
> The log entry is prepended with lines:
> SEVERE: org.apache.solr.common.SolrException: Internal Server Error
> Internal Server Error
> (they are not in the pastebin link)
> The shard log: http://pastebin.com/exwCx3LX
> Suggestion: change the data structure in FacetComponent to send only doc ids that belong to a shard and not a concatenation of all doc ids.
> Why is this important: for scaling. Adding more shards will result in overflowing the POST request buffer at some point anyway.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org