You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Dave Seltzer <ds...@tveyes.com> on 2016/11/01 20:18:14 UTC
Poor Solr Cloud Query Performance against a Small Dataset
Hello!
I'm trying to utilize Solr Cloud to help with a hash search problem. The
record set has only 4,300 documents.
When I run my search against a single core I get results on the order of
10ms. When I run the same search against Solr Cloud results take about
5,000 ms.
Is there something about this particular query which makes it perform
poorly in a Cloud environment? The query looks like this (linebreaks added
for readability):
{!frange+l%3D5+u%3D25}sum(
termfreq(hashTable_0,'225706351'),
termfreq(hashTable_1,'17664000'),
termfreq(hashTable_2,'86447642'),
termfreq(hashTable_3,'134816033'),
termfreq(hashTable_4,'1061820218'),
termfreq(hashTable_5,'543627850'),
termfreq(hashTable_6,'-1828379348'),
termfreq(hashTable_7,'423236759'),
termfreq(hashTable_8,'522192943'),
termfreq(hashTable_9,'572537937'),
termfreq(hashTable_10,'286991887'),
termfreq(hashTable_11,'789711386'),
termfreq(hashTable_12,'235801909'),
termfreq(hashTable_13,'67109911'),
termfreq(hashTable_14,'609628285'),
termfreq(hashTable_15,'1796472850'),
termfreq(hashTable_16,'202312085'),
termfreq(hashTable_17,'306200840'),
termfreq(hashTable_18,'85657669'),
termfreq(hashTable_19,'671548727'),
termfreq(hashTable_20,'71309060'),
termfreq(hashTable_21,'1125848323'),
termfreq(hashTable_22,'1077548043'),
termfreq(hashTable_23,'117638159'),
termfreq(hashTable_24,'-1408039642'))
The schema looks like this:
<field name="subFingerprintId" type="string" indexed="true"
stored="true" required="true" multiValued="false"
omitTermFreqAndPositions="true" omitNorms="true" />
<field name="trackId" type="string" indexed="true" stored="true"
required="true" multiValued="false" omitTermFreqAndPositions="true"
omitNorms="true" />
<dynamicField name="hashTable_*" type="long" indexed="true"
stored="true" omitTermFreqAndPositions="true" omitNorms="true" />
<field name="sequenceAt" type="double" indexed="false" stored="true"
required="false" multiValued="false" omitTermFreqAndPositions="true"
omitNorms="true" />
<field name="sequenceNumber" type="int" indexed="false" stored="true"
required="false" multiValued="false" omitTermFreqAndPositions="true"
omitNorms="true" />
<uniqueKey>subFingerprintId</uniqueKey>
I've included some sample output below. I wasn't sure if this was a matter
of changing the routing key in the collections system, or if this is a more
fundamental problem with the way Term Frequencies are counted in a Solr
Cloud environment.
Many thanks!
-Dave
---------------------- Single Core Example Query:
{
"responseHeader":{
"status":0,
"QTime":13,
"params":{
"q":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
"indent":"on",
"wt":"json",
"debugQuery":"on",
"_":"1478024378680"}},
"response":{"numFound":1,"start":0,"docs":[
{
"subFingerprintId":"f6c9093e-e8e9-4c0f-aa2a-387b46e7ef2a",
"trackId":"5207095a-0126-4c41-8787-16d41165158a",
"sequenceNumber":136,
"sequenceAt":12.5399129172714,
"hashTable_0":354749018,
"hashTable_1":287779841,
"hashTable_2":1797994010,
"hashTable_3":151854851,
"hashTable_4":375260422,
"hashTable_5":441911360,
"hashTable_6":68120837,
"hashTable_7":420158535,
"hashTable_8":16979004,
"hashTable_9":1443304209,
"hashTable_10":1732468239,
"hashTable_11":455215642,
"hashTable_12":135990559,
"hashTable_13":123093271,
"hashTable_14":1444029969,
"hashTable_15":788988675,
"hashTable_16":53480196,
"hashTable_17":488255035,
"hashTable_18":505809973,
"hashTable_19":201814293,
"hashTable_20":70208520,
"hashTable_21":805503541,
"hashTable_22":658713904,
"hashTable_23":302387775,
"hashTable_24":194970957,
"_version_":1549818240561053696}]
},
"debug":{
"rawquerystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
"querystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
"parsedquery":"FunctionRangeQuery(ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]))",
"parsedquery_toString":"ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25])",
"explain":{
"f6c9093e-e8e9-4c0f-aa2a-387b46e7ef2a":"\n1.0 =
ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]), product of:\n 1.0 = boost\n 1.0 = queryNorm\n"},
"QParser":"",
"timing":{
"time":13.0,
"prepare":{
"time":0.0,
"query":{
"time":0.0},
"facet":{
"time":0.0},
"facet_module":{
"time":0.0},
"mlt":{
"time":0.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"expand":{
"time":0.0},
"terms":{
"time":0.0},
"debug":{
"time":0.0}},
"process":{
"time":12.0,
"query":{
"time":10.0},
"facet":{
"time":0.0},
"facet_module":{
"time":0.0},
"mlt":{
"time":0.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"expand":{
"time":0.0},
"terms":{
"time":0.0},
"debug":{
"time":2.0}}}}}
---------------------- Solr Cloud Example Query:
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":5023,
"params":{
"q":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
"indent":"on",
"wt":"json",
"debugQuery":"on",
"_":"1478024068169"}},
"response":{"numFound":1,"start":0,"maxScore":1.0,"docs":[
{
"subFingerprintId":"98839fbe-7986-440d-89aa-559a24219d87",
"trackId":"3652cc99-767e-4210-b242-d49730957c31",
"sequenceNumber":136,
"sequenceAt":12.5399129172714,
"hashTable_0":354749018,
"hashTable_1":287779841,
"hashTable_2":1797994010,
"hashTable_3":151854851,
"hashTable_4":375260422,
"hashTable_5":441911360,
"hashTable_6":68120837,
"hashTable_7":420158535,
"hashTable_8":16979004,
"hashTable_9":1443304209,
"hashTable_10":1732468239,
"hashTable_11":455215642,
"hashTable_12":135990559,
"hashTable_13":123093271,
"hashTable_14":1444029969,
"hashTable_15":788988675,
"hashTable_16":53480196,
"hashTable_17":488255035,
"hashTable_18":505809973,
"hashTable_19":201814293,
"hashTable_20":70208520,
"hashTable_21":805503541,
"hashTable_22":658713904,
"hashTable_23":302387775,
"hashTable_24":194970957,
"_version_":1549806667971952640}]
},
"debug":{
"track":{
"rid":"crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3",
"EXECUTE_QUERY":{
"
http://crompcore02:8983/solr/sf_fingerprints_shard1_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard1_replica1/
":{
"QTime":"4",
"ElapsedTime":"5012",
"RequestPurpose":"GET_TOP_IDS",
"NumFound":"0",
"Response":"{responseHeader={zkConnected=true,status=0,QTime=4,params={df=text,distrib=false,debug=[false,
timing, track],fl=[subFingerprintId,
score],shards.purpose=4,start=0,fsv=true,shard.url=
http://crompcore02:8983/solr/sf_fingerprints_shard1_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard1_replica1/,rows=10,rid=crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3,version=2,q={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),requestPurpose=GET_TOP_IDS,NOW=1478024295225,isShard=true,wt=javabin,debugQuery=false,_=1478024068169}},response={numFound=0,start=0,maxScore=0.0,docs=[]},sort_values={},debug={timing={time=4.0,prepare={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=3.0,query={time=3.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}}}}}"},
"
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/
":{
"QTime":"4",
"ElapsedTime":"5013",
"RequestPurpose":"GET_TOP_IDS",
"NumFound":"1",
"Response":"{responseHeader={zkConnected=true,status=0,QTime=4,params={df=text,distrib=false,debug=[false,
timing, track],fl=[subFingerprintId,
score],shards.purpose=4,start=0,fsv=true,shard.url=
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/,rows=10,rid=crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3,version=2,q={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),requestPurpose=GET_TOP_IDS,NOW=1478024295225,isShard=true,wt=javabin,debugQuery=false,_=1478024068169}},response={numFound=1,start=0,maxScore=1.0,docs=[SolrDocument{subFingerprintId=98839fbe-7986-440d-89aa-559a24219d87,
score=1.0}]},sort_values={},debug={timing={time=4.0,prepare={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=3.0,query={time=3.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}}}}}"}},
"GET_FIELDS":{
"
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/
":{
"QTime":"2",
"ElapsedTime":"6",
"RequestPurpose":"GET_FIELDS,GET_DEBUG",
"NumFound":"1",
"Response":"{responseHeader={zkConnected=true,status=0,QTime=2,params={df=text,distrib=false,debug=[timing,
track],shards.purpose=320,shard.url=
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/,rows=10,rid=crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3,version=2,q={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),requestPurpose=GET_FIELDS,GET_DEBUG,NOW=1478024295225,ids=98839fbe-7986-440d-89aa-559a24219d87,isShard=true,wt=javabin,debugQuery=true,_=1478024068169}},response={numFound=1,start=0,docs=[SolrDocument{subFingerprintId=98839fbe-7986-440d-89aa-559a24219d87,
trackId=3652cc99-767e-4210-b242-d49730957c31, sequenceNumber=136,
sequenceAt=12.5399129172714, hashTable_0=354749018, hashTable_1=287779841,
hashTable_2=1797994010, hashTable_3=151854851, hashTable_4=375260422,
hashTable_5=441911360, hashTable_6=68120837, hashTable_7=420158535,
hashTable_8=16979004, hashTable_9=1443304209, hashTable_10=1732468239,
hashTable_11=455215642, hashTable_12=135990559, hashTable_13=123093271,
hashTable_14=1444029969, hashTable_15=788988675, hashTable_16=53480196,
hashTable_17=488255035, hashTable_18=505809973, hashTable_19=201814293,
hashTable_20=70208520, hashTable_21=805503541, hashTable_22=658713904,
hashTable_23=302387775, hashTable_24=194970957,
_version_=1549806667971952640}]},debug={rawquerystring={!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),querystring={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),parsedquery=FunctionRangeQuery(ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO
25])),parsedquery_toString=ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]),explain={98839fbe-7986-440d-89aa-559a24219d87=\n1.0 =
ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]), product of:\n 1.0 = boost\n 1.0 =
queryNorm\n},QParser=,timing={time=2.0,prepare={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=1.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=1.0}}}}}"}}},
"timing":{
"time":10.0,
"prepare":{
"time":0.0,
"query":{
"time":0.0},
"facet":{
"time":0.0},
"facet_module":{
"time":0.0},
"mlt":{
"time":0.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"expand":{
"time":0.0},
"terms":{
"time":0.0},
"debug":{
"time":0.0}},
"process":{
"time":7.0,
"query":{
"time":6.0},
"facet":{
"time":0.0},
"facet_module":{
"time":0.0},
"mlt":{
"time":0.0},
"highlight":{
"time":0.0},
"stats":{
"time":0.0},
"expand":{
"time":0.0},
"terms":{
"time":0.0},
"debug":{
"time":1.0}}},
"rawquerystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
"querystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
"parsedquery":"FunctionRangeQuery(ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]))",
"parsedquery_toString":"ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25])",
"QParser":"",
"explain":{
"98839fbe-7986-440d-89aa-559a24219d87":"\n1.0 =
ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]), product of:\n 1.0 = boost\n 1.0 = queryNorm\n"}}}
Re: Poor Solr Cloud Query Performance against a Small Dataset
Posted by Dave Seltzer <ds...@tveyes.com>.
Good tip Rick,
I'll dig in and make sure everything is set up correctly.
Thanks!
-D
Dave Seltzer <ds...@tveyes.com>
Chief Systems Architect
TVEyes
(203) 254-3600 x222
On Wed, Nov 2, 2016 at 9:05 PM, Rick Leir <rl...@leirtech.com> wrote:
> Here is a wild guess. Whenever I see a 5 second delay in networking, I
> think DNS timeouts. YMMV, good luck.
>
> cheers -- Rick
>
> On 2016-11-01 04:18 PM, Dave Seltzer wrote:
>
>> Hello!
>>
>> I'm trying to utilize Solr Cloud to help with a hash search problem. The
>> record set has only 4,300 documents.
>>
>> When I run my search against a single core I get results on the order of
>> 10ms. When I run the same search against Solr Cloud results take about
>> 5,000 ms.
>>
>> Is there something about this particular query which makes it perform
>> poorly in a Cloud environment? The query looks like this (linebreaks added
>> for readability):
>>
>> {!frange+l%3D5+u%3D25}sum(
>> termfreq(hashTable_0,'225706351'),
>> termfreq(hashTable_1,'17664000'),
>> termfreq(hashTable_2,'86447642'),
>> termfreq(hashTable_3,'134816033'),
>>
>
>
Re: Poor Solr Cloud Query Performance against a Small Dataset
Posted by Rick Leir <rl...@leirtech.com>.
Here is a wild guess. Whenever I see a 5 second delay in networking, I
think DNS timeouts. YMMV, good luck.
cheers -- Rick
On 2016-11-01 04:18 PM, Dave Seltzer wrote:
> Hello!
>
> I'm trying to utilize Solr Cloud to help with a hash search problem. The
> record set has only 4,300 documents.
>
> When I run my search against a single core I get results on the order of
> 10ms. When I run the same search against Solr Cloud results take about
> 5,000 ms.
>
> Is there something about this particular query which makes it perform
> poorly in a Cloud environment? The query looks like this (linebreaks added
> for readability):
>
> {!frange+l%3D5+u%3D25}sum(
> termfreq(hashTable_0,'225706351'),
> termfreq(hashTable_1,'17664000'),
> termfreq(hashTable_2,'86447642'),
> termfreq(hashTable_3,'134816033'),