You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Markus Jelsma <ma...@openindex.io> on 2012/12/12 13:37:25 UTC
SolrCloud breaks distributed query strings
Hi,
We're starting to see issues on a test cluster where Solr breaks up query string parameters that are either defined in the request handler or are passed in the URL in the initial request.
In our request handler we have an SF parameter for edismax (SOLR-3925):
<str name="sf">
title_general~2^4
title_nl~2^4
title_en~2^4
title_de~2^4
</str>
Almost all queries pass without issue but some fail because the parameter arrives in an incorrect format, i've logged several occurences:
2012-12-12 12:01:12,159 ERROR [solr.core.SolrCore] - [http-8080-exec-23] - : org
.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid a
rguments for sf, must be sf=FIELD~DISTANCE^BOOST, got
title_general~2^4
title_nl~2^4
title_en~2^4
title_de~2
4
at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
nt.java:154)
....
2012-12-12 12:00:57,164 ERROR [solr.core.SolrCore] - [http-8080-exec-1] - : org.
apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
guments for sf, must be sf=FIELD~DISTANCE^BOOST, got
title_general~2^4
title_nl~2
4
title_en~2^4
title_de~2^4
at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
nt.java:154)
....
2012-12-12 12:01:11,223 ERROR [solr.core.SolrCore] - [http-8080-exec-8] - : org.
apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
guments for sf, must be sf=FIELD~DISTANCE^BOOST, got ^ title_general~2^4
title_nl~2^4
title_en~2^4
title_de~2^4
at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
nt.java:154)
....
This seems crazy! For some reason, some times, the parameter get corrupted in some manner! We've also seen this with a function query in the edismax boost parameter where for some reasons a comma is replaced by a newline:
2012-12-12 11:11:45,527 ERROR [solr.core.SolrCore] - [http-8080-exec-16] - : org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
.9),.8),.7)'
at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:154)
...
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
.9),.8),.7)'
Accompanying these errors is a number of AIOOBexceptions without stack trace and Spellchecker NPE's (SOLR-4049). I'm completely puzzled here because it queries get randomly mangled in some manner. The SF parameter seems to get mangled only by replacing ^ with a newline. The boost query seems to be mangled in the same way if it fails. Only about 6% of all queries fired to the cluster end in such an error.
We're also seeing strange facets returned where two constraints seem to appear in a single returned value for a field, completely messed up :)
2012-12-12 12:00:56,341 ERROR [handler.component.FacetComponent] - [http-8080-exec-11] - : Unexpected term returned for facet refining. key=host term='aandeanderekant.domain.ext^aanoukk.domain.ext'
request params=spellcheck=false&facet=true&sort=score+desc&tie=0.35&spellcheck.maxCollationTries=2&ps3=5&facet.limit=8&hl.simple.pre=%3Cem%3E&q.alt=*%3A*&distrib=true&facet.method=enum&hl=false&shards.tolerant=true&omitHeader=true&echoParams=none&fl=md_*+title_*+id+type+subcollection+host+cat+date+size+lang&ps2=10&hl.simple.post=%3C%2Fem%3E&spellcheck.count=1&qs=9&spellcheck.alternativeTermCount=1&hl.fragsize=192&mm=80%25&spellcheck.maxResultsForSuggest=12&facet.mincount=1&spellcheck.extendedResults=true&uf=-*&f.host.facet.method=fc&qf=%0A++++++++domain_grams%5E3.7%0A++++++++domain_idx%5E15.9%0A++++++++host_idx%5E2.8%0A++++++++url%5E3.64%0A++++++++content_general%5E1.6+title_general%5E6.4+h1_general%5E5.4+h2_general%5E2.3%0A++++++++content_nl%5E1.6+title_nl%5E6.4+h1_nl%5E5.4+h2_nl%5E2.3%0A++++++++content_en%5E1.6+title_en%5E6.4+h1_en%5E5.4+h2_en%5E2.3%0A++++++++content_de%5E1.6+title_de%5E6.4+h1_de%5E5.4+h2_d
e%5E2.3%0A%0A++++++&sf=%0A++++++++title_general%7E2%5E4%0A++++++++title_nl%7E2%5E4%0A++++++++title_en%7E2%5E4%0A++++++++title_de%7E2%5E4%0A%0A++++++&hl.fl=content_*&json.nl=map&spellcheck.collate=true&wt=json&rows=0&defType=edismax&pf=%0A++++++++content_general%0A++++++++content_nl%0A++++++++content_en%0A++++++++content_de%0A%0A++++++&hl.useFastVectorHighlighter=true&hl.snippets=1&hl.maxAlternateFieldLength=192&facet.sort=count&spellcheck.dictionary=default&hl.alternateField=content_*&ps=25&spellcheck.q=aanvraagformulier+sterabonnement&spellcheck.q=aanvraagformulier+sterabonnement&bqhomepage=homepage%3Atrue&ssi=5b4fc43933c03c4f&ssi=5b4fc43933c03c4f&boost=def%28query%28%24bqhomepage%29%2C.68%29&boost=if%28exists%28date%29%2Cmax%28recip%28ms%28NOW%2FDAY%2Cdate%29%2C3.17e-8%2C143%2C.9%29%2C.8%29%2C.7%29&q=aanvraagformulier+sterabonnement&q=aanvraagformulier+sterabonnement&facet.field=%7B%21terms%3D%24host__terms+ex%3Dhost%7Dhost
&host__terms=nl.wikipedia.org%2Ccorenwilvandee.domain.ext%2Cannevandervelde.domain.ext%2Cbramvantriest.domain.ext%2Cdewilepelaar.domain.ext%2Cdirkenjasper.domain.ext%2Cverslootinqatar.domain.ext%2Cwww.kieskeurig.nl%2Cyvonnetromp.domain.ext%2C2goglobal.domain.ext%2C888888.domain.ext%2Caafkezuidervliet.domain.ext%2Caandeanderekant.domain.ext%2Caanoukk.domain.ext%2Cagdoun.domain.ext%2Caishakuipers.domain.ext%2Cakkelyn.domain.ext%2Calbertinev.domain.ext%2Calexandra1987.domain.ext%2Calva.domain.ext%2Canamika.domain.ext%2Cangelicawibmer.domain.ext%2Cangeliquetijs.domain.ext%2Caniekduyverman.domain.ext%2Cankeroeffen.domain.ext%2Cannekegouw.domain.ext%2Canneliesvanheek.domain.ext%2Canneman.domain.ext%2Cannemiekenlisette.domain.ext%2Cannevelthorst.domain.ext%2Cannewieke.domain.ext%2Canoek82.domain.ext%2Canoukenjulie.domain.ext%2Canoukensjoerd.domain.ext%2Carendenbarbara.domain.ext%2Carieenruubje.domain.ext%2Carindafamilie.domain.ext%2Carjankreeft.domain.ext%2Carletontour.domain.ext
%2Casiatraveller.domain.ext%2Cassistanatlille.domain.ext%2Caukeenannemieke.domain.ext%2Cavontuurlijkediana.domain.ext%2Cbabeth.domain.ext%2Cbartschillings.domain.ext%2Cbartvanhoorn.domain.ext%2Cbasbronsing.domain.ext%2Cbasenalberto.domain.ext%2Cbasinzuidafrika.domain.ext%2Cbelanda.domain.ext%2Cbenjealintokyo.domain.ext%2Cbertenbeasoest.domain.ext%2Cbianca2007.domain.ext%2Cbiancaenleeinmalta.domain.ext%2Cbiancaf.domain.ext%2Cbienekeenrene.domain.ext%2Cbobhendriks.domain.ext%2Cbonke.domain.ext%2Cboontje.domain.ext%2Cbramvanworkum.domain.ext%2Ccaatjevanrijswijk.domain.ext%2Ccelinevd.domain.ext%2Ccharlyn.domain.ext%2Cchelsey.domain.ext%2Cchina2006.domain.ext%2Cchrisenmarleen.domain.ext%2Cchrisvanhaften.domain.ext%2Ccindyverheij.domain.ext%2Cconnyengeorge.domain.ext%2Cconstantijnbrouwer.domain.ext%2Ccostapiet.domain.ext%2Cdaaninzuidafrika.domain.ext%2Cdaniellewillems.domain.ext%2Cdappel.domain.ext%2Cdebbyatbulamu.domain.ext%2Cdiannehofenk.domain.ext%2Cdickenellenoppad.domain.ext%
2Cdickenirene.domain.ext%2Cdidikortekaas.domain.ext%2Cdielange1982.domain.ext%2Cdigigee.domain.ext%2Cdikhuizing.domain.ext%2Cdirkopdefiets.domain.ext%2Cdocent.domain.ext%2Cdorienstraathof.domain.ext%2Cdorijnsuriname.domain.ext%2Cdutchpebble.domain.ext%2Ceddienicole.domain.ext%2Cedenheleen.domain.ext%2Ceelcokarin.domain.ext%2Ceelkeeva.domain.ext%2Celianneansems.domain.ext%2Celkeheurkens.domain.ext%2Cellenenhenkjan.domain.ext%2Celleninnz.domain.ext%2Cellenvandenberg.domain.ext%2Cellisscheinhardt.domain.ext%2Cellyenhans.domain.ext%2Cericenyvonchina.domain.ext%2Cericschuit.domain.ext%2Cestherinoeganda.domain.ext%2Cevamagre.domain.ext%2Ceveliendelijster.domain.ext%2Ceveliensombekke.domain.ext%2Cfajalobi.domain.ext%2Cfeemeel.domain.ext%2Cfemkeennienkeinghana.domain.ext%2Cfennahijlkjealbertine.domain.ext%2Cfernanenlininazie.domain.ext%2Cfleurhinzuidafrika.domain.ext%2Cfop.domain.ext%2Cfranksjak.domain.ext%2Cfudith.domain.ext%2Cganze77venezuela2005.domain.ext%2Cgeerte07.domain.ext%2
Cgeng.domain.ext%2Cgerbenendorien.domain.ext%2Cgertencisca.domain.ext%2Cgewoonmarin.domain.ext%2Cgezien.domain.ext%2Cgieneneric.domain.ext%2Cgrita.domain.ext%2Cguapa.domain.ext%2Cguill.domain.ext%2Channyenhans.domain.ext%2Chanszweers.domain.ext%2Chasaka.domain.ext%2Chegoestochina.domain.ext%2Chenrikepeters.domain.ext%2Chettyvanewijk.domain.ext%2Cils.domain.ext%2Cinganiet.domain.ext%2Cingekortekaas.domain.ext%2Cjacinthameilink.domain.ext%2Cjasperwildenborg.domain.ext%2Cjennieelam.domain.ext%2Cjossmits.domain.ext%2Ckarindekeijzer.domain.ext
toRefine=[Ljava.util.List;@35a76700
response={nl.wikipedia.org=1,corenwilvandee.domain.ext=0,annevandervelde.domain.ext=0,bramvantriest.domain.ext=0,dewilepelaar.domain.ext=0,dirkenjasper.domain.ext=0,verslootinqatar.domain.ext=0,www.kieskeurig.nl=0,yvonnetromp.domain.ext=0,2goglobal.domain.ext=0,888888.domain.ext=0,aafkezuidervliet.domain.ext=0,aandeanderekant.domain.ext^aanoukk.domain.ext=0,agdoun.domain.ext=0,aishakuipers.domain.ext=0,akkelyn.domain.ext=0,albertinev.domain.ext=0,alexandra1987.domain.ext=0,alva.domain.ext=0,anamika.domain.ext=0,angelicawibmer.domain.ext
angeliquetijs.domain.ext=0,aniekduyverman.domain.ext=0,ankeroeffen.domain.ext=0,annekegouw.domain.ext=0,anneliesvanheek.domain.ext=0,anneman.domain.ext=0,annemiekenlisette.domain.ext=0,annevelthorst.domain.ext=0,annewieke.domain.ext=0,anoek82.domain.ext=0,anoukenjulie.domain.ext=0,anoukensjoerd.domain.ext=0,arendenbarbara.domain.ext=0,arieenruubje.domain.ext=0,arindafamilie.domain.ext=0,arjankreeft.domain.ext=0,arletontour.domain.ext=0,asiatraveller.domain.ext=0,assistanatlille.domain.ext=0,aukeenannemieke.domain.ext=0,avontuurlijkediana.domain.ext=0,babeth.domain.ext=0,bartschillings.domain.ext=0,bartvanhoorn.domain.ext=0,basbronsing.domain.ext=0,basenalberto.domain.ext=0,basinzuidafrika.domain.ext=0,belanda.domain.ext=0,benjealintokyo.domain.ext=0,bertenbeasoest.domain.ext=0,bianca2007.domain.ext=0,biancaenleeinmalta.domain.ext=0,biancaf.domain.ext=0,bienekeenrene.domain.ext=0,bobhendriks.domain.ext=0,bon
ke.domain.ext=0,boontje.domain.ext=0,bramvanworkum.domain.ext=0,caatjevanrijswijk.domain.ext=0,celinevd.domain.ext=0,charlyn.domain.ext=0,chelsey.domain.ext=0,china2006.domain.ext=0,chrisenmarleen.domain.ext=0,chrisvanhaften.domain.ext=0,cindyverheij.domain.ext=0,connyengeorge.domain.ext=0,constantijnbrouwer.domain.ext=0,costapiet.domain.ext=0,daaninzuidafrika.domain.ext=0,daniellewillems.domain.ext=0,dappel.domain.ext=0,debbyatbulamu.domain.ext=0,diannehofenk.domain.ext=0,dickenellenoppad.domain.ext=0,dickenirene.domain.ext=0,didikortekaas.domain.ext=0,dielange1982.domain.ext=0,digigee.domain.ext=0,dikhuizing.domain.ext=0,dirkopdefiets.domain.ext=0,docent.domain.ext=0,dorienstraathof.domain.ext=0,dorijnsuriname.domain.ext=0,dutchpebble.domain.ext=0,eddienicole.domain.ext=0,edenheleen.domain.ext=0,eelcokarin.domain.ext=0,eelkeeva.domain.ext=0,elianneansems.domain.ext=0,elkeheurkens.domain.ext=0,ellenenhe
nkjan.domain.ext=0,elleninnz.domain.ext=0,ellenvandenberg.domain.ext=0,ellisscheinhardt.domain.ext=0,ellyenhans.domain.ext=0,ericenyvonchina.domain.ext=0,ericschuit.domain.ext=0,estherinoeganda.domain.ext=0,evamagre.domain.ext=0,eveliendelijster.domain.ext=0,eveliensombekke.domain.ext=0,fajalobi.domain.ext=0,feemeel.domain.ext=0,femkeennienkeinghana.domain.ext=0,fennahijlkjealbertine.domain.ext=0,fernanenlininazie.domain.ext=0,fleurhinzuidafrika.domain.ext=0,fop.domain.ext=0,franksjak.domain.ext=0,fudith.domain.ext=0,ganze77venezuela2005.domain.ext=0,geerte07.domain.ext=0,geng.domain.ext=0,gerbenendorien.domain.ext=0,gertencisca.domain.ext=0,gewoonmarin.domain.ext=0,gezien.domain.ext=0,gieneneric.domain.ext=0,grita.domain.ext=0,guapa.domain.ext=0,guill.domain.ext=0,hannyenhans.domain.ext=0,hanszweers.domain.ext=0,hasaka.domain.ext=0,hegoestochina.domain.ext=0,henrikepeters.domain.ext=0,hettyvanewijk.doma
in.ext=0,ils.domain.ext=0,inganiet.domain.ext=0,ingekortekaas.domain.ext=0,jacinthameilink.domain.ext=0,jasperwildenborg.domain.ext=0,jennieelam.domain.ext=0,jossmits.domain.ext=0,karindekeijzer.domain.ext=0}
Before opening an issue i'd like to know in which direction to look.
Thanks,
Markus
RE: SolrCloud breaks distributed query strings
Posted by Markus Jelsma <ma...@openindex.io>.
This problem persists, i've filed an issue to track it:
https://issues.apache.org/jira/browse/SOLR-4285
-----Original message-----
> From:Markus Jelsma <ma...@openindex.io>
> Sent: Mon 17-Dec-2012 10:49
> To: solr-user@lucene.apache.org
> Subject: RE: SolrCloud breaks distributed query strings
>
> Anyone else noticed a similar issue where Solr mangles distributed query parameters? Any hints on how to track this issue? Where to look?
>
> Thanks
>
> -----Original message-----
> > From:Markus Jelsma <ma...@openindex.io>
> > Sent: Wed 12-Dec-2012 15:11
> > To: solr-user@lucene.apache.org
> > Subject: RE: SolrCloud breaks distributed query strings
> >
> > Hi Per,
> >
> > We're running Tomcat6 with the today's checkout from trunk. I cannot remember i've seen it before and i cannot reproduce it manually in my browser, only in concurrent stress tests firing queries.
> >
> > Thanks
> > Markus
> >
> > -----Original message-----
> > > From:Per Steffensen <st...@designware.dk>
> > > Sent: Wed 12-Dec-2012 15:04
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: SolrCloud breaks distributed query strings
> > >
> > > It doesnt sound exactly like a problem we experienced some time ago,
> > > where long request where mixed put during transport. Jetty was to blame.
> > > I might be Jetty that f.... up you request too? SOLR-4031. Are you still
> > > running 8.1.2?
> > >
> > > Regards, Per Steffensen
> > >
> > > Markus Jelsma skrev:
> > > > Hi,
> > > >
> > > > We're starting to see issues on a test cluster where Solr breaks up query string parameters that are either defined in the request handler or are passed in the URL in the initial request.
> > > >
> > > > In our request handler we have an SF parameter for edismax (SOLR-3925):
> > > >
> > > > <str name="sf">
> > > > title_general~2^4
> > > > title_nl~2^4
> > > > title_en~2^4
> > > > title_de~2^4
> > > > </str>
> > > >
> > > > Almost all queries pass without issue but some fail because the parameter arrives in an incorrect format, i've logged several occurences:
> > > >
> > > > 2012-12-12 12:01:12,159 ERROR [solr.core.SolrCore] - [http-8080-exec-23] - : org
> > > > .apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid a
> > > > rguments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > > > title_general~2^4
> > > > title_nl~2^4
> > > > title_en~2^4
> > > > title_de~2
> > > > 4
> > > >
> > > >
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > > nt.java:154)
> > > > ....
> > > >
> > > > 2012-12-12 12:00:57,164 ERROR [solr.core.SolrCore] - [http-8080-exec-1] - : org.
> > > > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > > > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > > > title_general~2^4
> > > > title_nl~2
> > > > 4
> > > > title_en~2^4
> > > > title_de~2^4
> > > >
> > > >
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > > nt.java:154)
> > > > ....
> > > >
> > > > 2012-12-12 12:01:11,223 ERROR [solr.core.SolrCore] - [http-8080-exec-8] - : org.
> > > > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > > > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got ^ title_general~2^4
> > > > title_nl~2^4
> > > > title_en~2^4
> > > > title_de~2^4
> > > >
> > > >
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > > nt.java:154)
> > > > ....
> > > >
> > > > This seems crazy! For some reason, some times, the parameter get corrupted in some manner! We've also seen this with a function query in the edismax boost parameter where for some reasons a comma is replaced by a newline:
> > > >
> > > > 2012-12-12 11:11:45,527 ERROR [solr.core.SolrCore] - [http-8080-exec-16] - : org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > > > .9),.8),.7)'
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:154)
> > > > ...
> > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > at java.lang.Thread.run(Thread.java:662)
> > > > Caused by: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > > > .9),.8),.7)'
> > > >
> > > > Accompanying these errors is a number of AIOOBexceptions without stack trace and Spellchecker NPE's (SOLR-4049). I'm completely puzzled here because it queries get randomly mangled in some manner. The SF parameter seems to get mangled only by replacing ^ with a newline. The boost query seems to be mangled in the same way if it fails. Only about 6% of all queries fired to the cluster end in such an error.
> > > >
> > > > We're also seeing strange facets returned where two constraints seem to appear in a single returned value for a field, completely messed up :)
> > > >
> > > > 2012-12-12 12:00:56,341 ERROR [handler.component.FacetComponent] - [http-8080-exec-11] - : Unexpected term returned for facet refining. key=host term='aandeanderekant.domain.ext^aanoukk.domain.ext'
> > > > request params=spellcheck=false&facet=true&sort=score+desc&tie=0.35&spellcheck.maxCollationTries=2&ps3=5&facet.limit=8&hl.simple.pre=%3Cem%3E&q.alt=*%3A*&distrib=true&facet.method=enum&hl=false&shards.tolerant=true&omitHeader=true&echoParams=none&fl=md_*+title_*+id+type+subcollection+host+cat+date+size+lang&ps2=10&hl.simple.post=%3C%2Fem%3E&spellcheck.count=1&qs=9&spellcheck.alternativeTermCount=1&hl.fragsize=192&mm=80%25&spellcheck.maxResultsForSuggest=12&facet.mincount=1&spellcheck.extendedResults=true&uf=-*&f.host.facet.method=fc&qf=%0A++++++++domain_grams%5E3.7%0A++++++++domain_idx%5E15.9%0A++++++++host_idx%5E2.8%0A++++++++url%5E3.64%0A++++++++content_general%5E1.6+title_general%5E6.4+h1_general%5E5.4+h2_general%5E2.3%0A++++++++content_nl%5E1.6+title_nl%5E6.4+h1_nl%5E5.4+h2_nl%5E2.3%0A++++++++content_en%5E1.6+title_en%5E6.4+h1_en%5E5.4+h2_en%5E2.3%0A++++++++content_de%5E1.6+title_de%5E6.4+h1_de%5E
5.
> 4+
> > h2_d
> > > > e%5E2.3%0A%0A++++++&sf=%0A++++++++title_general%7E2%5E4%0A++++++++title_nl%7E2%5E4%0A++++++++title_en%7E2%5E4%0A++++++++title_de%7E2%5E4%0A%0A++++++&hl.fl=content_*&json.nl=map&spellcheck.collate=true&wt=json&rows=0&defType=edismax&pf=%0A++++++++content_general%0A++++++++content_nl%0A++++++++content_en%0A++++++++content_de%0A%0A++++++&hl.useFastVectorHighlighter=true&hl.snippets=1&hl.maxAlternateFieldLength=192&facet.sort=count&spellcheck.dictionary=default&hl.alternateField=content_*&ps=25&spellcheck.q=aanvraagformulier+sterabonnement&spellcheck.q=aanvraagformulier+sterabonnement&bqhomepage=homepage%3Atrue&ssi=5b4fc43933c03c4f&ssi=5b4fc43933c03c4f&boost=def%28query%28%24bqhomepage%29%2C.68%29&boost=if%28exists%28date%29%2Cmax%28recip%28ms%28NOW%2FDAY%2Cdate%29%2C3.17e-8%2C143%2C.9%29%2C.8%29%2C.7%29&q=aanvraagformulier+sterabonnement&q=aanvraagformulier+sterabonnement&facet.field=%7B%21terms%3D%24host__terms+ex%3Dhos
t%
> 7D
> > host
> > > > &host__terms=nl.wikipedia.org%2Ccorenwilvandee.domain.ext%2Cannevandervelde.domain.ext%2Cbramvantriest.domain.ext%2Cdewilepelaar.domain.ext%2Cdirkenjasper.domain.ext%2Cverslootinqatar.domain.ext%2Cwww.kieskeurig.nl%2Cyvonnetromp.domain.ext%2C2goglobal.domain.ext%2C888888.domain.ext%2Caafkezuidervliet.domain.ext%2Caandeanderekant.domain.ext%2Caanoukk.domain.ext%2Cagdoun.domain.ext%2Caishakuipers.domain.ext%2Cakkelyn.domain.ext%2Calbertinev.domain.ext%2Calexandra1987.domain.ext%2Calva.domain.ext%2Canamika.domain.ext%2Cangelicawibmer.domain.ext%2Cangeliquetijs.domain.ext%2Caniekduyverman.domain.ext%2Cankeroeffen.domain.ext%2Cannekegouw.domain.ext%2Canneliesvanheek.domain.ext%2Canneman.domain.ext%2Cannemiekenlisette.domain.ext%2Cannevelthorst.domain.ext%2Cannewieke.domain.ext%2Canoek82.domain.ext%2Canoukenjulie.domain.ext%2Canoukensjoerd.domain.ext%2Carendenbarbara.domain.ext%2Carieenruubje.domain.ext%2Carindafamilie.domain.ext%2Carjankreeft.domain.ext%2Carletontour.do
ma
> in
> > .ext
> > > > %2Casiatraveller.domain.ext%2Cassistanatlille.domain.ext%2Caukeenannemieke.domain.ext%2Cavontuurlijkediana.domain.ext%2Cbabeth.domain.ext%2Cbartschillings.domain.ext%2Cbartvanhoorn.domain.ext%2Cbasbronsing.domain.ext%2Cbasenalberto.domain.ext%2Cbasinzuidafrika.domain.ext%2Cbelanda.domain.ext%2Cbenjealintokyo.domain.ext%2Cbertenbeasoest.domain.ext%2Cbianca2007.domain.ext%2Cbiancaenleeinmalta.domain.ext%2Cbiancaf.domain.ext%2Cbienekeenrene.domain.ext%2Cbobhendriks.domain.ext%2Cbonke.domain.ext%2Cboontje.domain.ext%2Cbramvanworkum.domain.ext%2Ccaatjevanrijswijk.domain.ext%2Ccelinevd.domain.ext%2Ccharlyn.domain.ext%2Cchelsey.domain.ext%2Cchina2006.domain.ext%2Cchrisenmarleen.domain.ext%2Cchrisvanhaften.domain.ext%2Ccindyverheij.domain.ext%2Cconnyengeorge.domain.ext%2Cconstantijnbrouwer.domain.ext%2Ccostapiet.domain.ext%2Cdaaninzuidafrika.domain.ext%2Cdaniellewillems.domain.ext%2Cdappel.domain.ext%2Cdebbyatbulamu.domain.ext%2Cdiannehofenk.domain.ext%2Cdickenellenoppad.dom
ai
> n.
> > ext%
> > > > 2Cdickenirene.domain.ext%2Cdidikortekaas.domain.ext%2Cdielange1982.domain.ext%2Cdigigee.domain.ext%2Cdikhuizing.domain.ext%2Cdirkopdefiets.domain.ext%2Cdocent.domain.ext%2Cdorienstraathof.domain.ext%2Cdorijnsuriname.domain.ext%2Cdutchpebble.domain.ext%2Ceddienicole.domain.ext%2Cedenheleen.domain.ext%2Ceelcokarin.domain.ext%2Ceelkeeva.domain.ext%2Celianneansems.domain.ext%2Celkeheurkens.domain.ext%2Cellenenhenkjan.domain.ext%2Celleninnz.domain.ext%2Cellenvandenberg.domain.ext%2Cellisscheinhardt.domain.ext%2Cellyenhans.domain.ext%2Cericenyvonchina.domain.ext%2Cericschuit.domain.ext%2Cestherinoeganda.domain.ext%2Cevamagre.domain.ext%2Ceveliendelijster.domain.ext%2Ceveliensombekke.domain.ext%2Cfajalobi.domain.ext%2Cfeemeel.domain.ext%2Cfemkeennienkeinghana.domain.ext%2Cfennahijlkjealbertine.domain.ext%2Cfernanenlininazie.domain.ext%2Cfleurhinzuidafrika.domain.ext%2Cfop.domain.ext%2Cfranksjak.domain.ext%2Cfudith.domain.ext%2Cganze77venezuela2005.domain.ext%2Cgeerte07.doma
in
> .e
> > xt%2
> > > > Cgeng.domain.ext%2Cgerbenendorien.domain.ext%2Cgertencisca.domain.ext%2Cgewoonmarin.domain.ext%2Cgezien.domain.ext%2Cgieneneric.domain.ext%2Cgrita.domain.ext%2Cguapa.domain.ext%2Cguill.domain.ext%2Channyenhans.domain.ext%2Chanszweers.domain.ext%2Chasaka.domain.ext%2Chegoestochina.domain.ext%2Chenrikepeters.domain.ext%2Chettyvanewijk.domain.ext%2Cils.domain.ext%2Cinganiet.domain.ext%2Cingekortekaas.domain.ext%2Cjacinthameilink.domain.ext%2Cjasperwildenborg.domain.ext%2Cjennieelam.domain.ext%2Cjossmits.domain.ext%2Ckarindekeijzer.domain.ext
> > > > toRefine=[Ljava.util.List;@35a76700
> > > > response={nl.wikipedia.org=1,corenwilvandee.domain.ext=0,annevandervelde.domain.ext=0,bramvantriest.domain.ext=0,dewilepelaar.domain.ext=0,dirkenjasper.domain.ext=0,verslootinqatar.domain.ext=0,www.kieskeurig.nl=0,yvonnetromp.domain.ext=0,2goglobal.domain.ext=0,888888.domain.ext=0,aafkezuidervliet.domain.ext=0,aandeanderekant.domain.ext^aanoukk.domain.ext=0,agdoun.domain.ext=0,aishakuipers.domain.ext=0,akkelyn.domain.ext=0,albertinev.domain.ext=0,alexandra1987.domain.ext=0,alva.domain.ext=0,anamika.domain.ext=0,angelicawibmer.domain.ext
> > > > angeliquetijs.domain.ext=0,aniekduyverman.domain.ext=0,ankeroeffen.domain.ext=0,annekegouw.domain.ext=0,anneliesvanheek.domain.ext=0,anneman.domain.ext=0,annemiekenlisette.domain.ext=0,annevelthorst.domain.ext=0,annewieke.domain.ext=0,anoek82.domain.ext=0,anoukenjulie.domain.ext=0,anoukensjoerd.domain.ext=0,arendenbarbara.domain.ext=0,arieenruubje.domain.ext=0,arindafamilie.domain.ext=0,arjankreeft.domain.ext=0,arletontour.domain.ext=0,asiatraveller.domain.ext=0,assistanatlille.domain.ext=0,aukeenannemieke.domain.ext=0,avontuurlijkediana.domain.ext=0,babeth.domain.ext=0,bartschillings.domain.ext=0,bartvanhoorn.domain.ext=0,basbronsing.domain.ext=0,basenalberto.domain.ext=0,basinzuidafrika.domain.ext=0,belanda.domain.ext=0,benjealintokyo.domain.ext=0,bertenbeasoest.domain.ext=0,bianca2007.domain.ext=0,biancaenleeinmalta.domain.ext=0,biancaf.domain.ext=0,bienekeenrene.domain.ext=0,bobhendriks.domain.ext
0
> D0
> > ,bon
> > > > ke.domain.ext=0,boontje.domain.ext=0,bramvanworkum.domain.ext=0,caatjevanrijswijk.domain.ext=0,celinevd.domain.ext=0,charlyn.domain.ext=0,chelsey.domain.ext=0,china2006.domain.ext=0,chrisenmarleen.domain.ext=0,chrisvanhaften.domain.ext=0,cindyverheij.domain.ext=0,connyengeorge.domain.ext=0,constantijnbrouwer.domain.ext=0,costapiet.domain.ext=0,daaninzuidafrika.domain.ext=0,daniellewillems.domain.ext=0,dappel.domain.ext=0,debbyatbulamu.domain.ext=0,diannehofenk.domain.ext=0,dickenellenoppad.domain.ext=0,dickenirene.domain.ext=0,didikortekaas.domain.ext=0,dielange1982.domain.ext=0,digigee.domain.ext=0,dikhuizing.domain.ext=0,dirkopdefiets.domain.ext=0,docent.domain.ext=0,dorienstraathof.domain.ext=0,dorijnsuriname.domain.ext=0,dutchpebble.domain.ext=0,eddienicole.domain.ext=0,edenheleen.domain.ext=0,eelcokarin.domain.ext=0,eelkeeva.domain.ext=0,elianneansems.domain.ext=0,elkeheurkens.domain.ext=0,e
ll
> en
> > enhe
> > > > nkjan.domain.ext=0,elleninnz.domain.ext=0,ellenvandenberg.domain.ext=0,ellisscheinhardt.domain.ext=0,ellyenhans.domain.ext=0,ericenyvonchina.domain.ext=0,ericschuit.domain.ext=0,estherinoeganda.domain.ext=0,evamagre.domain.ext=0,eveliendelijster.domain.ext=0,eveliensombekke.domain.ext=0,fajalobi.domain.ext=0,feemeel.domain.ext=0,femkeennienkeinghana.domain.ext=0,fennahijlkjealbertine.domain.ext=0,fernanenlininazie.domain.ext=0,fleurhinzuidafrika.domain.ext=0,fop.domain.ext=0,franksjak.domain.ext=0,fudith.domain.ext=0,ganze77venezuela2005.domain.ext=0,geerte07.domain.ext=0,geng.domain.ext=0,gerbenendorien.domain.ext=0,gertencisca.domain.ext=0,gewoonmarin.domain.ext=0,gezien.domain.ext=0,gieneneric.domain.ext=0,grita.domain.ext=0,guapa.domain.ext=0,guill.domain.ext=0,hannyenhans.domain.ext=0,hanszweers.domain.ext=0,hasaka.domain.ext=0,hegoestochina.domain.ext=0,henrikepeters.domain.ext=0,hettyvanew
ij
> k.
> > doma
> > > > in.ext=0,ils.domain.ext=0,inganiet.domain.ext=0,ingekortekaas.domain.ext=0,jacinthameilink.domain.ext=0,jasperwildenborg.domain.ext=0,jennieelam.domain.ext=0,jossmits.domain.ext=0,karindekeijzer.domain.ext=0}
> > > >
> > > > Before opening an issue i'd like to know in which direction to look.
> > > >
> > > > Thanks,
> > > > Markus
> > > >
> > > >
> > >
> > >
> > >
> >
>
RE: SolrCloud breaks distributed query strings
Posted by Markus Jelsma <ma...@openindex.io>.
FYI: also noticed the same incorrect facet behaviour in a much older trunk check out, around august 2012.
-----Original message-----
> From:Markus Jelsma <ma...@openindex.io>
> Sent: Mon 17-Dec-2012 10:49
> To: solr-user@lucene.apache.org
> Subject: RE: SolrCloud breaks distributed query strings
>
> Anyone else noticed a similar issue where Solr mangles distributed query parameters? Any hints on how to track this issue? Where to look?
>
> Thanks
>
> -----Original message-----
> > From:Markus Jelsma <ma...@openindex.io>
> > Sent: Wed 12-Dec-2012 15:11
> > To: solr-user@lucene.apache.org
> > Subject: RE: SolrCloud breaks distributed query strings
> >
> > Hi Per,
> >
> > We're running Tomcat6 with the today's checkout from trunk. I cannot remember i've seen it before and i cannot reproduce it manually in my browser, only in concurrent stress tests firing queries.
> >
> > Thanks
> > Markus
> >
> > -----Original message-----
> > > From:Per Steffensen <st...@designware.dk>
> > > Sent: Wed 12-Dec-2012 15:04
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: SolrCloud breaks distributed query strings
> > >
> > > It doesnt sound exactly like a problem we experienced some time ago,
> > > where long request where mixed put during transport. Jetty was to blame.
> > > I might be Jetty that f.... up you request too? SOLR-4031. Are you still
> > > running 8.1.2?
> > >
> > > Regards, Per Steffensen
> > >
> > > Markus Jelsma skrev:
> > > > Hi,
> > > >
> > > > We're starting to see issues on a test cluster where Solr breaks up query string parameters that are either defined in the request handler or are passed in the URL in the initial request.
> > > >
> > > > In our request handler we have an SF parameter for edismax (SOLR-3925):
> > > >
> > > > <str name="sf">
> > > > title_general~2^4
> > > > title_nl~2^4
> > > > title_en~2^4
> > > > title_de~2^4
> > > > </str>
> > > >
> > > > Almost all queries pass without issue but some fail because the parameter arrives in an incorrect format, i've logged several occurences:
> > > >
> > > > 2012-12-12 12:01:12,159 ERROR [solr.core.SolrCore] - [http-8080-exec-23] - : org
> > > > .apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid a
> > > > rguments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > > > title_general~2^4
> > > > title_nl~2^4
> > > > title_en~2^4
> > > > title_de~2
> > > > 4
> > > >
> > > >
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > > nt.java:154)
> > > > ....
> > > >
> > > > 2012-12-12 12:00:57,164 ERROR [solr.core.SolrCore] - [http-8080-exec-1] - : org.
> > > > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > > > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > > > title_general~2^4
> > > > title_nl~2
> > > > 4
> > > > title_en~2^4
> > > > title_de~2^4
> > > >
> > > >
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > > nt.java:154)
> > > > ....
> > > >
> > > > 2012-12-12 12:01:11,223 ERROR [solr.core.SolrCore] - [http-8080-exec-8] - : org.
> > > > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > > > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got ^ title_general~2^4
> > > > title_nl~2^4
> > > > title_en~2^4
> > > > title_de~2^4
> > > >
> > > >
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > > nt.java:154)
> > > > ....
> > > >
> > > > This seems crazy! For some reason, some times, the parameter get corrupted in some manner! We've also seen this with a function query in the edismax boost parameter where for some reasons a comma is replaced by a newline:
> > > >
> > > > 2012-12-12 11:11:45,527 ERROR [solr.core.SolrCore] - [http-8080-exec-16] - : org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > > > .9),.8),.7)'
> > > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:154)
> > > > ...
> > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > at java.lang.Thread.run(Thread.java:662)
> > > > Caused by: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > > > .9),.8),.7)'
> > > >
> > > > Accompanying these errors is a number of AIOOBexceptions without stack trace and Spellchecker NPE's (SOLR-4049). I'm completely puzzled here because it queries get randomly mangled in some manner. The SF parameter seems to get mangled only by replacing ^ with a newline. The boost query seems to be mangled in the same way if it fails. Only about 6% of all queries fired to the cluster end in such an error.
> > > >
> > > > We're also seeing strange facets returned where two constraints seem to appear in a single returned value for a field, completely messed up :)
> > > >
> > > > 2012-12-12 12:00:56,341 ERROR [handler.component.FacetComponent] - [http-8080-exec-11] - : Unexpected term returned for facet refining. key=host term='aandeanderekant.domain.ext^aanoukk.domain.ext'
> > > > request params=spellcheck=false&facet=true&sort=score+desc&tie=0.35&spellcheck.maxCollationTries=2&ps3=5&facet.limit=8&hl.simple.pre=%3Cem%3E&q.alt=*%3A*&distrib=true&facet.method=enum&hl=false&shards.tolerant=true&omitHeader=true&echoParams=none&fl=md_*+title_*+id+type+subcollection+host+cat+date+size+lang&ps2=10&hl.simple.post=%3C%2Fem%3E&spellcheck.count=1&qs=9&spellcheck.alternativeTermCount=1&hl.fragsize=192&mm=80%25&spellcheck.maxResultsForSuggest=12&facet.mincount=1&spellcheck.extendedResults=true&uf=-*&f.host.facet.method=fc&qf=%0A++++++++domain_grams%5E3.7%0A++++++++domain_idx%5E15.9%0A++++++++host_idx%5E2.8%0A++++++++url%5E3.64%0A++++++++content_general%5E1.6+title_general%5E6.4+h1_general%5E5.4+h2_general%5E2.3%0A++++++++content_nl%5E1.6+title_nl%5E6.4+h1_nl%5E5.4+h2_nl%5E2.3%0A++++++++content_en%5E1.6+title_en%5E6.4+h1_en%5E5.4+h2_en%5E2.3%0A++++++++content_de%5E1.6+title_de%5E6.4+h1_de%5E
5.
> 4+
> > h2_d
> > > > e%5E2.3%0A%0A++++++&sf=%0A++++++++title_general%7E2%5E4%0A++++++++title_nl%7E2%5E4%0A++++++++title_en%7E2%5E4%0A++++++++title_de%7E2%5E4%0A%0A++++++&hl.fl=content_*&json.nl=map&spellcheck.collate=true&wt=json&rows=0&defType=edismax&pf=%0A++++++++content_general%0A++++++++content_nl%0A++++++++content_en%0A++++++++content_de%0A%0A++++++&hl.useFastVectorHighlighter=true&hl.snippets=1&hl.maxAlternateFieldLength=192&facet.sort=count&spellcheck.dictionary=default&hl.alternateField=content_*&ps=25&spellcheck.q=aanvraagformulier+sterabonnement&spellcheck.q=aanvraagformulier+sterabonnement&bqhomepage=homepage%3Atrue&ssi=5b4fc43933c03c4f&ssi=5b4fc43933c03c4f&boost=def%28query%28%24bqhomepage%29%2C.68%29&boost=if%28exists%28date%29%2Cmax%28recip%28ms%28NOW%2FDAY%2Cdate%29%2C3.17e-8%2C143%2C.9%29%2C.8%29%2C.7%29&q=aanvraagformulier+sterabonnement&q=aanvraagformulier+sterabonnement&facet.field=%7B%21terms%3D%24host__terms+ex%3Dhos
t%
> 7D
> > host
> > > > &host__terms=nl.wikipedia.org%2Ccorenwilvandee.domain.ext%2Cannevandervelde.domain.ext%2Cbramvantriest.domain.ext%2Cdewilepelaar.domain.ext%2Cdirkenjasper.domain.ext%2Cverslootinqatar.domain.ext%2Cwww.kieskeurig.nl%2Cyvonnetromp.domain.ext%2C2goglobal.domain.ext%2C888888.domain.ext%2Caafkezuidervliet.domain.ext%2Caandeanderekant.domain.ext%2Caanoukk.domain.ext%2Cagdoun.domain.ext%2Caishakuipers.domain.ext%2Cakkelyn.domain.ext%2Calbertinev.domain.ext%2Calexandra1987.domain.ext%2Calva.domain.ext%2Canamika.domain.ext%2Cangelicawibmer.domain.ext%2Cangeliquetijs.domain.ext%2Caniekduyverman.domain.ext%2Cankeroeffen.domain.ext%2Cannekegouw.domain.ext%2Canneliesvanheek.domain.ext%2Canneman.domain.ext%2Cannemiekenlisette.domain.ext%2Cannevelthorst.domain.ext%2Cannewieke.domain.ext%2Canoek82.domain.ext%2Canoukenjulie.domain.ext%2Canoukensjoerd.domain.ext%2Carendenbarbara.domain.ext%2Carieenruubje.domain.ext%2Carindafamilie.domain.ext%2Carjankreeft.domain.ext%2Carletontour.do
ma
> in
> > .ext
> > > > %2Casiatraveller.domain.ext%2Cassistanatlille.domain.ext%2Caukeenannemieke.domain.ext%2Cavontuurlijkediana.domain.ext%2Cbabeth.domain.ext%2Cbartschillings.domain.ext%2Cbartvanhoorn.domain.ext%2Cbasbronsing.domain.ext%2Cbasenalberto.domain.ext%2Cbasinzuidafrika.domain.ext%2Cbelanda.domain.ext%2Cbenjealintokyo.domain.ext%2Cbertenbeasoest.domain.ext%2Cbianca2007.domain.ext%2Cbiancaenleeinmalta.domain.ext%2Cbiancaf.domain.ext%2Cbienekeenrene.domain.ext%2Cbobhendriks.domain.ext%2Cbonke.domain.ext%2Cboontje.domain.ext%2Cbramvanworkum.domain.ext%2Ccaatjevanrijswijk.domain.ext%2Ccelinevd.domain.ext%2Ccharlyn.domain.ext%2Cchelsey.domain.ext%2Cchina2006.domain.ext%2Cchrisenmarleen.domain.ext%2Cchrisvanhaften.domain.ext%2Ccindyverheij.domain.ext%2Cconnyengeorge.domain.ext%2Cconstantijnbrouwer.domain.ext%2Ccostapiet.domain.ext%2Cdaaninzuidafrika.domain.ext%2Cdaniellewillems.domain.ext%2Cdappel.domain.ext%2Cdebbyatbulamu.domain.ext%2Cdiannehofenk.domain.ext%2Cdickenellenoppad.dom
ai
> n.
> > ext%
> > > > 2Cdickenirene.domain.ext%2Cdidikortekaas.domain.ext%2Cdielange1982.domain.ext%2Cdigigee.domain.ext%2Cdikhuizing.domain.ext%2Cdirkopdefiets.domain.ext%2Cdocent.domain.ext%2Cdorienstraathof.domain.ext%2Cdorijnsuriname.domain.ext%2Cdutchpebble.domain.ext%2Ceddienicole.domain.ext%2Cedenheleen.domain.ext%2Ceelcokarin.domain.ext%2Ceelkeeva.domain.ext%2Celianneansems.domain.ext%2Celkeheurkens.domain.ext%2Cellenenhenkjan.domain.ext%2Celleninnz.domain.ext%2Cellenvandenberg.domain.ext%2Cellisscheinhardt.domain.ext%2Cellyenhans.domain.ext%2Cericenyvonchina.domain.ext%2Cericschuit.domain.ext%2Cestherinoeganda.domain.ext%2Cevamagre.domain.ext%2Ceveliendelijster.domain.ext%2Ceveliensombekke.domain.ext%2Cfajalobi.domain.ext%2Cfeemeel.domain.ext%2Cfemkeennienkeinghana.domain.ext%2Cfennahijlkjealbertine.domain.ext%2Cfernanenlininazie.domain.ext%2Cfleurhinzuidafrika.domain.ext%2Cfop.domain.ext%2Cfranksjak.domain.ext%2Cfudith.domain.ext%2Cganze77venezuela2005.domain.ext%2Cgeerte07.doma
in
> .e
> > xt%2
> > > > Cgeng.domain.ext%2Cgerbenendorien.domain.ext%2Cgertencisca.domain.ext%2Cgewoonmarin.domain.ext%2Cgezien.domain.ext%2Cgieneneric.domain.ext%2Cgrita.domain.ext%2Cguapa.domain.ext%2Cguill.domain.ext%2Channyenhans.domain.ext%2Chanszweers.domain.ext%2Chasaka.domain.ext%2Chegoestochina.domain.ext%2Chenrikepeters.domain.ext%2Chettyvanewijk.domain.ext%2Cils.domain.ext%2Cinganiet.domain.ext%2Cingekortekaas.domain.ext%2Cjacinthameilink.domain.ext%2Cjasperwildenborg.domain.ext%2Cjennieelam.domain.ext%2Cjossmits.domain.ext%2Ckarindekeijzer.domain.ext
> > > > toRefine=[Ljava.util.List;@35a76700
> > > > response={nl.wikipedia.org=1,corenwilvandee.domain.ext=0,annevandervelde.domain.ext=0,bramvantriest.domain.ext=0,dewilepelaar.domain.ext=0,dirkenjasper.domain.ext=0,verslootinqatar.domain.ext=0,www.kieskeurig.nl=0,yvonnetromp.domain.ext=0,2goglobal.domain.ext=0,888888.domain.ext=0,aafkezuidervliet.domain.ext=0,aandeanderekant.domain.ext^aanoukk.domain.ext=0,agdoun.domain.ext=0,aishakuipers.domain.ext=0,akkelyn.domain.ext=0,albertinev.domain.ext=0,alexandra1987.domain.ext=0,alva.domain.ext=0,anamika.domain.ext=0,angelicawibmer.domain.ext
> > > > angeliquetijs.domain.ext=0,aniekduyverman.domain.ext=0,ankeroeffen.domain.ext=0,annekegouw.domain.ext=0,anneliesvanheek.domain.ext=0,anneman.domain.ext=0,annemiekenlisette.domain.ext=0,annevelthorst.domain.ext=0,annewieke.domain.ext=0,anoek82.domain.ext=0,anoukenjulie.domain.ext=0,anoukensjoerd.domain.ext=0,arendenbarbara.domain.ext=0,arieenruubje.domain.ext=0,arindafamilie.domain.ext=0,arjankreeft.domain.ext=0,arletontour.domain.ext=0,asiatraveller.domain.ext=0,assistanatlille.domain.ext=0,aukeenannemieke.domain.ext=0,avontuurlijkediana.domain.ext=0,babeth.domain.ext=0,bartschillings.domain.ext=0,bartvanhoorn.domain.ext=0,basbronsing.domain.ext=0,basenalberto.domain.ext=0,basinzuidafrika.domain.ext=0,belanda.domain.ext=0,benjealintokyo.domain.ext=0,bertenbeasoest.domain.ext=0,bianca2007.domain.ext=0,biancaenleeinmalta.domain.ext=0,biancaf.domain.ext=0,bienekeenrene.domain.ext=0,bobhendriks.domain.ext
0
> D0
> > ,bon
> > > > ke.domain.ext=0,boontje.domain.ext=0,bramvanworkum.domain.ext=0,caatjevanrijswijk.domain.ext=0,celinevd.domain.ext=0,charlyn.domain.ext=0,chelsey.domain.ext=0,china2006.domain.ext=0,chrisenmarleen.domain.ext=0,chrisvanhaften.domain.ext=0,cindyverheij.domain.ext=0,connyengeorge.domain.ext=0,constantijnbrouwer.domain.ext=0,costapiet.domain.ext=0,daaninzuidafrika.domain.ext=0,daniellewillems.domain.ext=0,dappel.domain.ext=0,debbyatbulamu.domain.ext=0,diannehofenk.domain.ext=0,dickenellenoppad.domain.ext=0,dickenirene.domain.ext=0,didikortekaas.domain.ext=0,dielange1982.domain.ext=0,digigee.domain.ext=0,dikhuizing.domain.ext=0,dirkopdefiets.domain.ext=0,docent.domain.ext=0,dorienstraathof.domain.ext=0,dorijnsuriname.domain.ext=0,dutchpebble.domain.ext=0,eddienicole.domain.ext=0,edenheleen.domain.ext=0,eelcokarin.domain.ext=0,eelkeeva.domain.ext=0,elianneansems.domain.ext=0,elkeheurkens.domain.ext=0,e
ll
> en
> > enhe
> > > > nkjan.domain.ext=0,elleninnz.domain.ext=0,ellenvandenberg.domain.ext=0,ellisscheinhardt.domain.ext=0,ellyenhans.domain.ext=0,ericenyvonchina.domain.ext=0,ericschuit.domain.ext=0,estherinoeganda.domain.ext=0,evamagre.domain.ext=0,eveliendelijster.domain.ext=0,eveliensombekke.domain.ext=0,fajalobi.domain.ext=0,feemeel.domain.ext=0,femkeennienkeinghana.domain.ext=0,fennahijlkjealbertine.domain.ext=0,fernanenlininazie.domain.ext=0,fleurhinzuidafrika.domain.ext=0,fop.domain.ext=0,franksjak.domain.ext=0,fudith.domain.ext=0,ganze77venezuela2005.domain.ext=0,geerte07.domain.ext=0,geng.domain.ext=0,gerbenendorien.domain.ext=0,gertencisca.domain.ext=0,gewoonmarin.domain.ext=0,gezien.domain.ext=0,gieneneric.domain.ext=0,grita.domain.ext=0,guapa.domain.ext=0,guill.domain.ext=0,hannyenhans.domain.ext=0,hanszweers.domain.ext=0,hasaka.domain.ext=0,hegoestochina.domain.ext=0,henrikepeters.domain.ext=0,hettyvanew
ij
> k.
> > doma
> > > > in.ext=0,ils.domain.ext=0,inganiet.domain.ext=0,ingekortekaas.domain.ext=0,jacinthameilink.domain.ext=0,jasperwildenborg.domain.ext=0,jennieelam.domain.ext=0,jossmits.domain.ext=0,karindekeijzer.domain.ext=0}
> > > >
> > > > Before opening an issue i'd like to know in which direction to look.
> > > >
> > > > Thanks,
> > > > Markus
> > > >
> > > >
> > >
> > >
> > >
> >
>
RE: SolrCloud breaks distributed query strings
Posted by Markus Jelsma <ma...@openindex.io>.
Anyone else noticed a similar issue where Solr mangles distributed query parameters? Any hints on how to track this issue? Where to look?
Thanks
-----Original message-----
> From:Markus Jelsma <ma...@openindex.io>
> Sent: Wed 12-Dec-2012 15:11
> To: solr-user@lucene.apache.org
> Subject: RE: SolrCloud breaks distributed query strings
>
> Hi Per,
>
> We're running Tomcat6 with the today's checkout from trunk. I cannot remember i've seen it before and i cannot reproduce it manually in my browser, only in concurrent stress tests firing queries.
>
> Thanks
> Markus
>
> -----Original message-----
> > From:Per Steffensen <st...@designware.dk>
> > Sent: Wed 12-Dec-2012 15:04
> > To: solr-user@lucene.apache.org
> > Subject: Re: SolrCloud breaks distributed query strings
> >
> > It doesnt sound exactly like a problem we experienced some time ago,
> > where long request where mixed put during transport. Jetty was to blame.
> > I might be Jetty that f.... up you request too? SOLR-4031. Are you still
> > running 8.1.2?
> >
> > Regards, Per Steffensen
> >
> > Markus Jelsma skrev:
> > > Hi,
> > >
> > > We're starting to see issues on a test cluster where Solr breaks up query string parameters that are either defined in the request handler or are passed in the URL in the initial request.
> > >
> > > In our request handler we have an SF parameter for edismax (SOLR-3925):
> > >
> > > <str name="sf">
> > > title_general~2^4
> > > title_nl~2^4
> > > title_en~2^4
> > > title_de~2^4
> > > </str>
> > >
> > > Almost all queries pass without issue but some fail because the parameter arrives in an incorrect format, i've logged several occurences:
> > >
> > > 2012-12-12 12:01:12,159 ERROR [solr.core.SolrCore] - [http-8080-exec-23] - : org
> > > .apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid a
> > > rguments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > > title_general~2^4
> > > title_nl~2^4
> > > title_en~2^4
> > > title_de~2
> > > 4
> > >
> > >
> > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > nt.java:154)
> > > ....
> > >
> > > 2012-12-12 12:00:57,164 ERROR [solr.core.SolrCore] - [http-8080-exec-1] - : org.
> > > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > > title_general~2^4
> > > title_nl~2
> > > 4
> > > title_en~2^4
> > > title_de~2^4
> > >
> > >
> > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > nt.java:154)
> > > ....
> > >
> > > 2012-12-12 12:01:11,223 ERROR [solr.core.SolrCore] - [http-8080-exec-8] - : org.
> > > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got ^ title_general~2^4
> > > title_nl~2^4
> > > title_en~2^4
> > > title_de~2^4
> > >
> > >
> > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > > nt.java:154)
> > > ....
> > >
> > > This seems crazy! For some reason, some times, the parameter get corrupted in some manner! We've also seen this with a function query in the edismax boost parameter where for some reasons a comma is replaced by a newline:
> > >
> > > 2012-12-12 11:11:45,527 ERROR [solr.core.SolrCore] - [http-8080-exec-16] - : org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > > .9),.8),.7)'
> > > at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:154)
> > > ...
> > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > at java.lang.Thread.run(Thread.java:662)
> > > Caused by: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > > .9),.8),.7)'
> > >
> > > Accompanying these errors is a number of AIOOBexceptions without stack trace and Spellchecker NPE's (SOLR-4049). I'm completely puzzled here because it queries get randomly mangled in some manner. The SF parameter seems to get mangled only by replacing ^ with a newline. The boost query seems to be mangled in the same way if it fails. Only about 6% of all queries fired to the cluster end in such an error.
> > >
> > > We're also seeing strange facets returned where two constraints seem to appear in a single returned value for a field, completely messed up :)
> > >
> > > 2012-12-12 12:00:56,341 ERROR [handler.component.FacetComponent] - [http-8080-exec-11] - : Unexpected term returned for facet refining. key=host term='aandeanderekant.domain.ext^aanoukk.domain.ext'
> > > request params=spellcheck=false&facet=true&sort=score+desc&tie=0.35&spellcheck.maxCollationTries=2&ps3=5&facet.limit=8&hl.simple.pre=%3Cem%3E&q.alt=*%3A*&distrib=true&facet.method=enum&hl=false&shards.tolerant=true&omitHeader=true&echoParams=none&fl=md_*+title_*+id+type+subcollection+host+cat+date+size+lang&ps2=10&hl.simple.post=%3C%2Fem%3E&spellcheck.count=1&qs=9&spellcheck.alternativeTermCount=1&hl.fragsize=192&mm=80%25&spellcheck.maxResultsForSuggest=12&facet.mincount=1&spellcheck.extendedResults=true&uf=-*&f.host.facet.method=fc&qf=%0A++++++++domain_grams%5E3.7%0A++++++++domain_idx%5E15.9%0A++++++++host_idx%5E2.8%0A++++++++url%5E3.64%0A++++++++content_general%5E1.6+title_general%5E6.4+h1_general%5E5.4+h2_general%5E2.3%0A++++++++content_nl%5E1.6+title_nl%5E6.4+h1_nl%5E5.4+h2_nl%5E2.3%0A++++++++content_en%5E1.6+title_en%5E6.4+h1_en%5E5.4+h2_en%5E2.3%0A++++++++content_de%5E1.6+title_de%5E6.4+h1_de%5E5.
4+
> h2_d
> > > e%5E2.3%0A%0A++++++&sf=%0A++++++++title_general%7E2%5E4%0A++++++++title_nl%7E2%5E4%0A++++++++title_en%7E2%5E4%0A++++++++title_de%7E2%5E4%0A%0A++++++&hl.fl=content_*&json.nl=map&spellcheck.collate=true&wt=json&rows=0&defType=edismax&pf=%0A++++++++content_general%0A++++++++content_nl%0A++++++++content_en%0A++++++++content_de%0A%0A++++++&hl.useFastVectorHighlighter=true&hl.snippets=1&hl.maxAlternateFieldLength=192&facet.sort=count&spellcheck.dictionary=default&hl.alternateField=content_*&ps=25&spellcheck.q=aanvraagformulier+sterabonnement&spellcheck.q=aanvraagformulier+sterabonnement&bqhomepage=homepage%3Atrue&ssi=5b4fc43933c03c4f&ssi=5b4fc43933c03c4f&boost=def%28query%28%24bqhomepage%29%2C.68%29&boost=if%28exists%28date%29%2Cmax%28recip%28ms%28NOW%2FDAY%2Cdate%29%2C3.17e-8%2C143%2C.9%29%2C.8%29%2C.7%29&q=aanvraagformulier+sterabonnement&q=aanvraagformulier+sterabonnement&facet.field=%7B%21terms%3D%24host__terms+ex%3Dhost%
7D
> host
> > > &host__terms=nl.wikipedia.org%2Ccorenwilvandee.domain.ext%2Cannevandervelde.domain.ext%2Cbramvantriest.domain.ext%2Cdewilepelaar.domain.ext%2Cdirkenjasper.domain.ext%2Cverslootinqatar.domain.ext%2Cwww.kieskeurig.nl%2Cyvonnetromp.domain.ext%2C2goglobal.domain.ext%2C888888.domain.ext%2Caafkezuidervliet.domain.ext%2Caandeanderekant.domain.ext%2Caanoukk.domain.ext%2Cagdoun.domain.ext%2Caishakuipers.domain.ext%2Cakkelyn.domain.ext%2Calbertinev.domain.ext%2Calexandra1987.domain.ext%2Calva.domain.ext%2Canamika.domain.ext%2Cangelicawibmer.domain.ext%2Cangeliquetijs.domain.ext%2Caniekduyverman.domain.ext%2Cankeroeffen.domain.ext%2Cannekegouw.domain.ext%2Canneliesvanheek.domain.ext%2Canneman.domain.ext%2Cannemiekenlisette.domain.ext%2Cannevelthorst.domain.ext%2Cannewieke.domain.ext%2Canoek82.domain.ext%2Canoukenjulie.domain.ext%2Canoukensjoerd.domain.ext%2Carendenbarbara.domain.ext%2Carieenruubje.domain.ext%2Carindafamilie.domain.ext%2Carjankreeft.domain.ext%2Carletontour.doma
in
> .ext
> > > %2Casiatraveller.domain.ext%2Cassistanatlille.domain.ext%2Caukeenannemieke.domain.ext%2Cavontuurlijkediana.domain.ext%2Cbabeth.domain.ext%2Cbartschillings.domain.ext%2Cbartvanhoorn.domain.ext%2Cbasbronsing.domain.ext%2Cbasenalberto.domain.ext%2Cbasinzuidafrika.domain.ext%2Cbelanda.domain.ext%2Cbenjealintokyo.domain.ext%2Cbertenbeasoest.domain.ext%2Cbianca2007.domain.ext%2Cbiancaenleeinmalta.domain.ext%2Cbiancaf.domain.ext%2Cbienekeenrene.domain.ext%2Cbobhendriks.domain.ext%2Cbonke.domain.ext%2Cboontje.domain.ext%2Cbramvanworkum.domain.ext%2Ccaatjevanrijswijk.domain.ext%2Ccelinevd.domain.ext%2Ccharlyn.domain.ext%2Cchelsey.domain.ext%2Cchina2006.domain.ext%2Cchrisenmarleen.domain.ext%2Cchrisvanhaften.domain.ext%2Ccindyverheij.domain.ext%2Cconnyengeorge.domain.ext%2Cconstantijnbrouwer.domain.ext%2Ccostapiet.domain.ext%2Cdaaninzuidafrika.domain.ext%2Cdaniellewillems.domain.ext%2Cdappel.domain.ext%2Cdebbyatbulamu.domain.ext%2Cdiannehofenk.domain.ext%2Cdickenellenoppad.domai
n.
> ext%
> > > 2Cdickenirene.domain.ext%2Cdidikortekaas.domain.ext%2Cdielange1982.domain.ext%2Cdigigee.domain.ext%2Cdikhuizing.domain.ext%2Cdirkopdefiets.domain.ext%2Cdocent.domain.ext%2Cdorienstraathof.domain.ext%2Cdorijnsuriname.domain.ext%2Cdutchpebble.domain.ext%2Ceddienicole.domain.ext%2Cedenheleen.domain.ext%2Ceelcokarin.domain.ext%2Ceelkeeva.domain.ext%2Celianneansems.domain.ext%2Celkeheurkens.domain.ext%2Cellenenhenkjan.domain.ext%2Celleninnz.domain.ext%2Cellenvandenberg.domain.ext%2Cellisscheinhardt.domain.ext%2Cellyenhans.domain.ext%2Cericenyvonchina.domain.ext%2Cericschuit.domain.ext%2Cestherinoeganda.domain.ext%2Cevamagre.domain.ext%2Ceveliendelijster.domain.ext%2Ceveliensombekke.domain.ext%2Cfajalobi.domain.ext%2Cfeemeel.domain.ext%2Cfemkeennienkeinghana.domain.ext%2Cfennahijlkjealbertine.domain.ext%2Cfernanenlininazie.domain.ext%2Cfleurhinzuidafrika.domain.ext%2Cfop.domain.ext%2Cfranksjak.domain.ext%2Cfudith.domain.ext%2Cganze77venezuela2005.domain.ext%2Cgeerte07.domain
.e
> xt%2
> > > Cgeng.domain.ext%2Cgerbenendorien.domain.ext%2Cgertencisca.domain.ext%2Cgewoonmarin.domain.ext%2Cgezien.domain.ext%2Cgieneneric.domain.ext%2Cgrita.domain.ext%2Cguapa.domain.ext%2Cguill.domain.ext%2Channyenhans.domain.ext%2Chanszweers.domain.ext%2Chasaka.domain.ext%2Chegoestochina.domain.ext%2Chenrikepeters.domain.ext%2Chettyvanewijk.domain.ext%2Cils.domain.ext%2Cinganiet.domain.ext%2Cingekortekaas.domain.ext%2Cjacinthameilink.domain.ext%2Cjasperwildenborg.domain.ext%2Cjennieelam.domain.ext%2Cjossmits.domain.ext%2Ckarindekeijzer.domain.ext
> > > toRefine=[Ljava.util.List;@35a76700
> > > response={nl.wikipedia.org=1,corenwilvandee.domain.ext=0,annevandervelde.domain.ext=0,bramvantriest.domain.ext=0,dewilepelaar.domain.ext=0,dirkenjasper.domain.ext=0,verslootinqatar.domain.ext=0,www.kieskeurig.nl=0,yvonnetromp.domain.ext=0,2goglobal.domain.ext=0,888888.domain.ext=0,aafkezuidervliet.domain.ext=0,aandeanderekant.domain.ext^aanoukk.domain.ext=0,agdoun.domain.ext=0,aishakuipers.domain.ext=0,akkelyn.domain.ext=0,albertinev.domain.ext=0,alexandra1987.domain.ext=0,alva.domain.ext=0,anamika.domain.ext=0,angelicawibmer.domain.ext
> > > angeliquetijs.domain.ext=0,aniekduyverman.domain.ext=0,ankeroeffen.domain.ext=0,annekegouw.domain.ext=0,anneliesvanheek.domain.ext=0,anneman.domain.ext=0,annemiekenlisette.domain.ext=0,annevelthorst.domain.ext=0,annewieke.domain.ext=0,anoek82.domain.ext=0,anoukenjulie.domain.ext=0,anoukensjoerd.domain.ext=0,arendenbarbara.domain.ext=0,arieenruubje.domain.ext=0,arindafamilie.domain.ext=0,arjankreeft.domain.ext=0,arletontour.domain.ext=0,asiatraveller.domain.ext=0,assistanatlille.domain.ext=0,aukeenannemieke.domain.ext=0,avontuurlijkediana.domain.ext=0,babeth.domain.ext=0,bartschillings.domain.ext=0,bartvanhoorn.domain.ext=0,basbronsing.domain.ext=0,basenalberto.domain.ext=0,basinzuidafrika.domain.ext=0,belanda.domain.ext=0,benjealintokyo.domain.ext=0,bertenbeasoest.domain.ext=0,bianca2007.domain.ext=0,biancaenleeinmalta.domain.ext=0,biancaf.domain.ext=0,bienekeenrene.domain.ext=0,bobhendriks.domain.ext=3
D0
> ,bon
> > > ke.domain.ext=0,boontje.domain.ext=0,bramvanworkum.domain.ext=0,caatjevanrijswijk.domain.ext=0,celinevd.domain.ext=0,charlyn.domain.ext=0,chelsey.domain.ext=0,china2006.domain.ext=0,chrisenmarleen.domain.ext=0,chrisvanhaften.domain.ext=0,cindyverheij.domain.ext=0,connyengeorge.domain.ext=0,constantijnbrouwer.domain.ext=0,costapiet.domain.ext=0,daaninzuidafrika.domain.ext=0,daniellewillems.domain.ext=0,dappel.domain.ext=0,debbyatbulamu.domain.ext=0,diannehofenk.domain.ext=0,dickenellenoppad.domain.ext=0,dickenirene.domain.ext=0,didikortekaas.domain.ext=0,dielange1982.domain.ext=0,digigee.domain.ext=0,dikhuizing.domain.ext=0,dirkopdefiets.domain.ext=0,docent.domain.ext=0,dorienstraathof.domain.ext=0,dorijnsuriname.domain.ext=0,dutchpebble.domain.ext=0,eddienicole.domain.ext=0,edenheleen.domain.ext=0,eelcokarin.domain.ext=0,eelkeeva.domain.ext=0,elianneansems.domain.ext=0,elkeheurkens.domain.ext=0,ell
en
> enhe
> > > nkjan.domain.ext=0,elleninnz.domain.ext=0,ellenvandenberg.domain.ext=0,ellisscheinhardt.domain.ext=0,ellyenhans.domain.ext=0,ericenyvonchina.domain.ext=0,ericschuit.domain.ext=0,estherinoeganda.domain.ext=0,evamagre.domain.ext=0,eveliendelijster.domain.ext=0,eveliensombekke.domain.ext=0,fajalobi.domain.ext=0,feemeel.domain.ext=0,femkeennienkeinghana.domain.ext=0,fennahijlkjealbertine.domain.ext=0,fernanenlininazie.domain.ext=0,fleurhinzuidafrika.domain.ext=0,fop.domain.ext=0,franksjak.domain.ext=0,fudith.domain.ext=0,ganze77venezuela2005.domain.ext=0,geerte07.domain.ext=0,geng.domain.ext=0,gerbenendorien.domain.ext=0,gertencisca.domain.ext=0,gewoonmarin.domain.ext=0,gezien.domain.ext=0,gieneneric.domain.ext=0,grita.domain.ext=0,guapa.domain.ext=0,guill.domain.ext=0,hannyenhans.domain.ext=0,hanszweers.domain.ext=0,hasaka.domain.ext=0,hegoestochina.domain.ext=0,henrikepeters.domain.ext=0,hettyvanewij
k.
> doma
> > > in.ext=0,ils.domain.ext=0,inganiet.domain.ext=0,ingekortekaas.domain.ext=0,jacinthameilink.domain.ext=0,jasperwildenborg.domain.ext=0,jennieelam.domain.ext=0,jossmits.domain.ext=0,karindekeijzer.domain.ext=0}
> > >
> > > Before opening an issue i'd like to know in which direction to look.
> > >
> > > Thanks,
> > > Markus
> > >
> > >
> >
> >
> >
>
RE: SolrCloud breaks distributed query strings
Posted by Markus Jelsma <ma...@openindex.io>.
Hi Per,
We're running Tomcat6 with the today's checkout from trunk. I cannot remember i've seen it before and i cannot reproduce it manually in my browser, only in concurrent stress tests firing queries.
Thanks
Markus
-----Original message-----
> From:Per Steffensen <st...@designware.dk>
> Sent: Wed 12-Dec-2012 15:04
> To: solr-user@lucene.apache.org
> Subject: Re: SolrCloud breaks distributed query strings
>
> It doesnt sound exactly like a problem we experienced some time ago,
> where long request where mixed put during transport. Jetty was to blame.
> I might be Jetty that f.... up you request too? SOLR-4031. Are you still
> running 8.1.2?
>
> Regards, Per Steffensen
>
> Markus Jelsma skrev:
> > Hi,
> >
> > We're starting to see issues on a test cluster where Solr breaks up query string parameters that are either defined in the request handler or are passed in the URL in the initial request.
> >
> > In our request handler we have an SF parameter for edismax (SOLR-3925):
> >
> > <str name="sf">
> > title_general~2^4
> > title_nl~2^4
> > title_en~2^4
> > title_de~2^4
> > </str>
> >
> > Almost all queries pass without issue but some fail because the parameter arrives in an incorrect format, i've logged several occurences:
> >
> > 2012-12-12 12:01:12,159 ERROR [solr.core.SolrCore] - [http-8080-exec-23] - : org
> > .apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid a
> > rguments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > title_general~2^4
> > title_nl~2^4
> > title_en~2^4
> > title_de~2
> > 4
> >
> >
> > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > nt.java:154)
> > ....
> >
> > 2012-12-12 12:00:57,164 ERROR [solr.core.SolrCore] - [http-8080-exec-1] - : org.
> > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> > title_general~2^4
> > title_nl~2
> > 4
> > title_en~2^4
> > title_de~2^4
> >
> >
> > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > nt.java:154)
> > ....
> >
> > 2012-12-12 12:01:11,223 ERROR [solr.core.SolrCore] - [http-8080-exec-8] - : org.
> > apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> > guments for sf, must be sf=FIELD~DISTANCE^BOOST, got ^ title_general~2^4
> > title_nl~2^4
> > title_en~2^4
> > title_de~2^4
> >
> >
> > at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> > nt.java:154)
> > ....
> >
> > This seems crazy! For some reason, some times, the parameter get corrupted in some manner! We've also seen this with a function query in the edismax boost parameter where for some reasons a comma is replaced by a newline:
> >
> > 2012-12-12 11:11:45,527 ERROR [solr.core.SolrCore] - [http-8080-exec-16] - : org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > .9),.8),.7)'
> > at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:154)
> > ...
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > at java.lang.Thread.run(Thread.java:662)
> > Caused by: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> > .9),.8),.7)'
> >
> > Accompanying these errors is a number of AIOOBexceptions without stack trace and Spellchecker NPE's (SOLR-4049). I'm completely puzzled here because it queries get randomly mangled in some manner. The SF parameter seems to get mangled only by replacing ^ with a newline. The boost query seems to be mangled in the same way if it fails. Only about 6% of all queries fired to the cluster end in such an error.
> >
> > We're also seeing strange facets returned where two constraints seem to appear in a single returned value for a field, completely messed up :)
> >
> > 2012-12-12 12:00:56,341 ERROR [handler.component.FacetComponent] - [http-8080-exec-11] - : Unexpected term returned for facet refining. key=host term='aandeanderekant.domain.ext^aanoukk.domain.ext'
> > request params=spellcheck=false&facet=true&sort=score+desc&tie=0.35&spellcheck.maxCollationTries=2&ps3=5&facet.limit=8&hl.simple.pre=%3Cem%3E&q.alt=*%3A*&distrib=true&facet.method=enum&hl=false&shards.tolerant=true&omitHeader=true&echoParams=none&fl=md_*+title_*+id+type+subcollection+host+cat+date+size+lang&ps2=10&hl.simple.post=%3C%2Fem%3E&spellcheck.count=1&qs=9&spellcheck.alternativeTermCount=1&hl.fragsize=192&mm=80%25&spellcheck.maxResultsForSuggest=12&facet.mincount=1&spellcheck.extendedResults=true&uf=-*&f.host.facet.method=fc&qf=%0A++++++++domain_grams%5E3.7%0A++++++++domain_idx%5E15.9%0A++++++++host_idx%5E2.8%0A++++++++url%5E3.64%0A++++++++content_general%5E1.6+title_general%5E6.4+h1_general%5E5.4+h2_general%5E2.3%0A++++++++content_nl%5E1.6+title_nl%5E6.4+h1_nl%5E5.4+h2_nl%5E2.3%0A++++++++content_en%5E1.6+title_en%5E6.4+h1_en%5E5.4+h2_en%5E2.3%0A++++++++content_de%5E1.6+title_de%5E6.4+h1_de%5E5.4+
h2_d
> > e%5E2.3%0A%0A++++++&sf=%0A++++++++title_general%7E2%5E4%0A++++++++title_nl%7E2%5E4%0A++++++++title_en%7E2%5E4%0A++++++++title_de%7E2%5E4%0A%0A++++++&hl.fl=content_*&json.nl=map&spellcheck.collate=true&wt=json&rows=0&defType=edismax&pf=%0A++++++++content_general%0A++++++++content_nl%0A++++++++content_en%0A++++++++content_de%0A%0A++++++&hl.useFastVectorHighlighter=true&hl.snippets=1&hl.maxAlternateFieldLength=192&facet.sort=count&spellcheck.dictionary=default&hl.alternateField=content_*&ps=25&spellcheck.q=aanvraagformulier+sterabonnement&spellcheck.q=aanvraagformulier+sterabonnement&bqhomepage=homepage%3Atrue&ssi=5b4fc43933c03c4f&ssi=5b4fc43933c03c4f&boost=def%28query%28%24bqhomepage%29%2C.68%29&boost=if%28exists%28date%29%2Cmax%28recip%28ms%28NOW%2FDAY%2Cdate%29%2C3.17e-8%2C143%2C.9%29%2C.8%29%2C.7%29&q=aanvraagformulier+sterabonnement&q=aanvraagformulier+sterabonnement&facet.field=%7B%21terms%3D%24host__terms+ex%3Dhost%7D
host
> > &host__terms=nl.wikipedia.org%2Ccorenwilvandee.domain.ext%2Cannevandervelde.domain.ext%2Cbramvantriest.domain.ext%2Cdewilepelaar.domain.ext%2Cdirkenjasper.domain.ext%2Cverslootinqatar.domain.ext%2Cwww.kieskeurig.nl%2Cyvonnetromp.domain.ext%2C2goglobal.domain.ext%2C888888.domain.ext%2Caafkezuidervliet.domain.ext%2Caandeanderekant.domain.ext%2Caanoukk.domain.ext%2Cagdoun.domain.ext%2Caishakuipers.domain.ext%2Cakkelyn.domain.ext%2Calbertinev.domain.ext%2Calexandra1987.domain.ext%2Calva.domain.ext%2Canamika.domain.ext%2Cangelicawibmer.domain.ext%2Cangeliquetijs.domain.ext%2Caniekduyverman.domain.ext%2Cankeroeffen.domain.ext%2Cannekegouw.domain.ext%2Canneliesvanheek.domain.ext%2Canneman.domain.ext%2Cannemiekenlisette.domain.ext%2Cannevelthorst.domain.ext%2Cannewieke.domain.ext%2Canoek82.domain.ext%2Canoukenjulie.domain.ext%2Canoukensjoerd.domain.ext%2Carendenbarbara.domain.ext%2Carieenruubje.domain.ext%2Carindafamilie.domain.ext%2Carjankreeft.domain.ext%2Carletontour.domain
.ext
> > %2Casiatraveller.domain.ext%2Cassistanatlille.domain.ext%2Caukeenannemieke.domain.ext%2Cavontuurlijkediana.domain.ext%2Cbabeth.domain.ext%2Cbartschillings.domain.ext%2Cbartvanhoorn.domain.ext%2Cbasbronsing.domain.ext%2Cbasenalberto.domain.ext%2Cbasinzuidafrika.domain.ext%2Cbelanda.domain.ext%2Cbenjealintokyo.domain.ext%2Cbertenbeasoest.domain.ext%2Cbianca2007.domain.ext%2Cbiancaenleeinmalta.domain.ext%2Cbiancaf.domain.ext%2Cbienekeenrene.domain.ext%2Cbobhendriks.domain.ext%2Cbonke.domain.ext%2Cboontje.domain.ext%2Cbramvanworkum.domain.ext%2Ccaatjevanrijswijk.domain.ext%2Ccelinevd.domain.ext%2Ccharlyn.domain.ext%2Cchelsey.domain.ext%2Cchina2006.domain.ext%2Cchrisenmarleen.domain.ext%2Cchrisvanhaften.domain.ext%2Ccindyverheij.domain.ext%2Cconnyengeorge.domain.ext%2Cconstantijnbrouwer.domain.ext%2Ccostapiet.domain.ext%2Cdaaninzuidafrika.domain.ext%2Cdaniellewillems.domain.ext%2Cdappel.domain.ext%2Cdebbyatbulamu.domain.ext%2Cdiannehofenk.domain.ext%2Cdickenellenoppad.domain.
ext%
> > 2Cdickenirene.domain.ext%2Cdidikortekaas.domain.ext%2Cdielange1982.domain.ext%2Cdigigee.domain.ext%2Cdikhuizing.domain.ext%2Cdirkopdefiets.domain.ext%2Cdocent.domain.ext%2Cdorienstraathof.domain.ext%2Cdorijnsuriname.domain.ext%2Cdutchpebble.domain.ext%2Ceddienicole.domain.ext%2Cedenheleen.domain.ext%2Ceelcokarin.domain.ext%2Ceelkeeva.domain.ext%2Celianneansems.domain.ext%2Celkeheurkens.domain.ext%2Cellenenhenkjan.domain.ext%2Celleninnz.domain.ext%2Cellenvandenberg.domain.ext%2Cellisscheinhardt.domain.ext%2Cellyenhans.domain.ext%2Cericenyvonchina.domain.ext%2Cericschuit.domain.ext%2Cestherinoeganda.domain.ext%2Cevamagre.domain.ext%2Ceveliendelijster.domain.ext%2Ceveliensombekke.domain.ext%2Cfajalobi.domain.ext%2Cfeemeel.domain.ext%2Cfemkeennienkeinghana.domain.ext%2Cfennahijlkjealbertine.domain.ext%2Cfernanenlininazie.domain.ext%2Cfleurhinzuidafrika.domain.ext%2Cfop.domain.ext%2Cfranksjak.domain.ext%2Cfudith.domain.ext%2Cganze77venezuela2005.domain.ext%2Cgeerte07.domain.e
xt%2
> > Cgeng.domain.ext%2Cgerbenendorien.domain.ext%2Cgertencisca.domain.ext%2Cgewoonmarin.domain.ext%2Cgezien.domain.ext%2Cgieneneric.domain.ext%2Cgrita.domain.ext%2Cguapa.domain.ext%2Cguill.domain.ext%2Channyenhans.domain.ext%2Chanszweers.domain.ext%2Chasaka.domain.ext%2Chegoestochina.domain.ext%2Chenrikepeters.domain.ext%2Chettyvanewijk.domain.ext%2Cils.domain.ext%2Cinganiet.domain.ext%2Cingekortekaas.domain.ext%2Cjacinthameilink.domain.ext%2Cjasperwildenborg.domain.ext%2Cjennieelam.domain.ext%2Cjossmits.domain.ext%2Ckarindekeijzer.domain.ext
> > toRefine=[Ljava.util.List;@35a76700
> > response={nl.wikipedia.org=1,corenwilvandee.domain.ext=0,annevandervelde.domain.ext=0,bramvantriest.domain.ext=0,dewilepelaar.domain.ext=0,dirkenjasper.domain.ext=0,verslootinqatar.domain.ext=0,www.kieskeurig.nl=0,yvonnetromp.domain.ext=0,2goglobal.domain.ext=0,888888.domain.ext=0,aafkezuidervliet.domain.ext=0,aandeanderekant.domain.ext^aanoukk.domain.ext=0,agdoun.domain.ext=0,aishakuipers.domain.ext=0,akkelyn.domain.ext=0,albertinev.domain.ext=0,alexandra1987.domain.ext=0,alva.domain.ext=0,anamika.domain.ext=0,angelicawibmer.domain.ext
> > angeliquetijs.domain.ext=0,aniekduyverman.domain.ext=0,ankeroeffen.domain.ext=0,annekegouw.domain.ext=0,anneliesvanheek.domain.ext=0,anneman.domain.ext=0,annemiekenlisette.domain.ext=0,annevelthorst.domain.ext=0,annewieke.domain.ext=0,anoek82.domain.ext=0,anoukenjulie.domain.ext=0,anoukensjoerd.domain.ext=0,arendenbarbara.domain.ext=0,arieenruubje.domain.ext=0,arindafamilie.domain.ext=0,arjankreeft.domain.ext=0,arletontour.domain.ext=0,asiatraveller.domain.ext=0,assistanatlille.domain.ext=0,aukeenannemieke.domain.ext=0,avontuurlijkediana.domain.ext=0,babeth.domain.ext=0,bartschillings.domain.ext=0,bartvanhoorn.domain.ext=0,basbronsing.domain.ext=0,basenalberto.domain.ext=0,basinzuidafrika.domain.ext=0,belanda.domain.ext=0,benjealintokyo.domain.ext=0,bertenbeasoest.domain.ext=0,bianca2007.domain.ext=0,biancaenleeinmalta.domain.ext=0,biancaf.domain.ext=0,bienekeenrene.domain.ext=0,bobhendriks.domain.ext=0
,bon
> > ke.domain.ext=0,boontje.domain.ext=0,bramvanworkum.domain.ext=0,caatjevanrijswijk.domain.ext=0,celinevd.domain.ext=0,charlyn.domain.ext=0,chelsey.domain.ext=0,china2006.domain.ext=0,chrisenmarleen.domain.ext=0,chrisvanhaften.domain.ext=0,cindyverheij.domain.ext=0,connyengeorge.domain.ext=0,constantijnbrouwer.domain.ext=0,costapiet.domain.ext=0,daaninzuidafrika.domain.ext=0,daniellewillems.domain.ext=0,dappel.domain.ext=0,debbyatbulamu.domain.ext=0,diannehofenk.domain.ext=0,dickenellenoppad.domain.ext=0,dickenirene.domain.ext=0,didikortekaas.domain.ext=0,dielange1982.domain.ext=0,digigee.domain.ext=0,dikhuizing.domain.ext=0,dirkopdefiets.domain.ext=0,docent.domain.ext=0,dorienstraathof.domain.ext=0,dorijnsuriname.domain.ext=0,dutchpebble.domain.ext=0,eddienicole.domain.ext=0,edenheleen.domain.ext=0,eelcokarin.domain.ext=0,eelkeeva.domain.ext=0,elianneansems.domain.ext=0,elkeheurkens.domain.ext=0,ellen
enhe
> > nkjan.domain.ext=0,elleninnz.domain.ext=0,ellenvandenberg.domain.ext=0,ellisscheinhardt.domain.ext=0,ellyenhans.domain.ext=0,ericenyvonchina.domain.ext=0,ericschuit.domain.ext=0,estherinoeganda.domain.ext=0,evamagre.domain.ext=0,eveliendelijster.domain.ext=0,eveliensombekke.domain.ext=0,fajalobi.domain.ext=0,feemeel.domain.ext=0,femkeennienkeinghana.domain.ext=0,fennahijlkjealbertine.domain.ext=0,fernanenlininazie.domain.ext=0,fleurhinzuidafrika.domain.ext=0,fop.domain.ext=0,franksjak.domain.ext=0,fudith.domain.ext=0,ganze77venezuela2005.domain.ext=0,geerte07.domain.ext=0,geng.domain.ext=0,gerbenendorien.domain.ext=0,gertencisca.domain.ext=0,gewoonmarin.domain.ext=0,gezien.domain.ext=0,gieneneric.domain.ext=0,grita.domain.ext=0,guapa.domain.ext=0,guill.domain.ext=0,hannyenhans.domain.ext=0,hanszweers.domain.ext=0,hasaka.domain.ext=0,hegoestochina.domain.ext=0,henrikepeters.domain.ext=0,hettyvanewijk.
doma
> > in.ext=0,ils.domain.ext=0,inganiet.domain.ext=0,ingekortekaas.domain.ext=0,jacinthameilink.domain.ext=0,jasperwildenborg.domain.ext=0,jennieelam.domain.ext=0,jossmits.domain.ext=0,karindekeijzer.domain.ext=0}
> >
> > Before opening an issue i'd like to know in which direction to look.
> >
> > Thanks,
> > Markus
> >
> >
>
>
>
Re: SolrCloud breaks distributed query strings
Posted by Per Steffensen <st...@designware.dk>.
It doesnt sound exactly like a problem we experienced some time ago,
where long request where mixed put during transport. Jetty was to blame.
I might be Jetty that f.... up you request too? SOLR-4031. Are you still
running 8.1.2?
Regards, Per Steffensen
Markus Jelsma skrev:
> Hi,
>
> We're starting to see issues on a test cluster where Solr breaks up query string parameters that are either defined in the request handler or are passed in the URL in the initial request.
>
> In our request handler we have an SF parameter for edismax (SOLR-3925):
>
> <str name="sf">
> title_general~2^4
> title_nl~2^4
> title_en~2^4
> title_de~2^4
> </str>
>
> Almost all queries pass without issue but some fail because the parameter arrives in an incorrect format, i've logged several occurences:
>
> 2012-12-12 12:01:12,159 ERROR [solr.core.SolrCore] - [http-8080-exec-23] - : org
> .apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid a
> rguments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> title_general~2^4
> title_nl~2^4
> title_en~2^4
> title_de~2
> 4
>
>
> at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> nt.java:154)
> ....
>
> 2012-12-12 12:00:57,164 ERROR [solr.core.SolrCore] - [http-8080-exec-1] - : org.
> apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> guments for sf, must be sf=FIELD~DISTANCE^BOOST, got
> title_general~2^4
> title_nl~2
> 4
> title_en~2^4
> title_de~2^4
>
>
> at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> nt.java:154)
> ....
>
> 2012-12-12 12:01:11,223 ERROR [solr.core.SolrCore] - [http-8080-exec-8] - : org.
> apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Invalid ar
> guments for sf, must be sf=FIELD~DISTANCE^BOOST, got ^ title_general~2^4
> title_nl~2^4
> title_en~2^4
> title_de~2^4
>
>
> at org.apache.solr.handler.component.QueryComponent.prepare(QueryCompone
> nt.java:154)
> ....
>
> This seems crazy! For some reason, some times, the parameter get corrupted in some manner! We've also seen this with a function query in the edismax boost parameter where for some reasons a comma is replaced by a newline:
>
> 2012-12-12 11:11:45,527 ERROR [solr.core.SolrCore] - [http-8080-exec-16] - : org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> .9),.8),.7)'
> at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:154)
> ...
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.solr.search.SyntaxError: Expected ',' at position 55 in 'if(exists(date),max(recip(ms(NOW/DAY,date),3.17e-8,143
> .9),.8),.7)'
>
> Accompanying these errors is a number of AIOOBexceptions without stack trace and Spellchecker NPE's (SOLR-4049). I'm completely puzzled here because it queries get randomly mangled in some manner. The SF parameter seems to get mangled only by replacing ^ with a newline. The boost query seems to be mangled in the same way if it fails. Only about 6% of all queries fired to the cluster end in such an error.
>
> We're also seeing strange facets returned where two constraints seem to appear in a single returned value for a field, completely messed up :)
>
> 2012-12-12 12:00:56,341 ERROR [handler.component.FacetComponent] - [http-8080-exec-11] - : Unexpected term returned for facet refining. key=host term='aandeanderekant.domain.ext^aanoukk.domain.ext'
> request params=spellcheck=false&facet=true&sort=score+desc&tie=0.35&spellcheck.maxCollationTries=2&ps3=5&facet.limit=8&hl.simple.pre=%3Cem%3E&q.alt=*%3A*&distrib=true&facet.method=enum&hl=false&shards.tolerant=true&omitHeader=true&echoParams=none&fl=md_*+title_*+id+type+subcollection+host+cat+date+size+lang&ps2=10&hl.simple.post=%3C%2Fem%3E&spellcheck.count=1&qs=9&spellcheck.alternativeTermCount=1&hl.fragsize=192&mm=80%25&spellcheck.maxResultsForSuggest=12&facet.mincount=1&spellcheck.extendedResults=true&uf=-*&f.host.facet.method=fc&qf=%0A++++++++domain_grams%5E3.7%0A++++++++domain_idx%5E15.9%0A++++++++host_idx%5E2.8%0A++++++++url%5E3.64%0A++++++++content_general%5E1.6+title_general%5E6.4+h1_general%5E5.4+h2_general%5E2.3%0A++++++++content_nl%5E1.6+title_nl%5E6.4+h1_nl%5E5.4+h2_nl%5E2.3%0A++++++++content_en%5E1.6+title_en%5E6.4+h1_en%5E5.4+h2_en%5E2.3%0A++++++++content_de%5E1.6+title_de%5E6.4+h1_de%5E5.4+h2_d
> e%5E2.3%0A%0A++++++&sf=%0A++++++++title_general%7E2%5E4%0A++++++++title_nl%7E2%5E4%0A++++++++title_en%7E2%5E4%0A++++++++title_de%7E2%5E4%0A%0A++++++&hl.fl=content_*&json.nl=map&spellcheck.collate=true&wt=json&rows=0&defType=edismax&pf=%0A++++++++content_general%0A++++++++content_nl%0A++++++++content_en%0A++++++++content_de%0A%0A++++++&hl.useFastVectorHighlighter=true&hl.snippets=1&hl.maxAlternateFieldLength=192&facet.sort=count&spellcheck.dictionary=default&hl.alternateField=content_*&ps=25&spellcheck.q=aanvraagformulier+sterabonnement&spellcheck.q=aanvraagformulier+sterabonnement&bqhomepage=homepage%3Atrue&ssi=5b4fc43933c03c4f&ssi=5b4fc43933c03c4f&boost=def%28query%28%24bqhomepage%29%2C.68%29&boost=if%28exists%28date%29%2Cmax%28recip%28ms%28NOW%2FDAY%2Cdate%29%2C3.17e-8%2C143%2C.9%29%2C.8%29%2C.7%29&q=aanvraagformulier+sterabonnement&q=aanvraagformulier+sterabonnement&facet.field=%7B%21terms%3D%24host__terms+ex%3Dhost%7Dhost
> &host__terms=nl.wikipedia.org%2Ccorenwilvandee.domain.ext%2Cannevandervelde.domain.ext%2Cbramvantriest.domain.ext%2Cdewilepelaar.domain.ext%2Cdirkenjasper.domain.ext%2Cverslootinqatar.domain.ext%2Cwww.kieskeurig.nl%2Cyvonnetromp.domain.ext%2C2goglobal.domain.ext%2C888888.domain.ext%2Caafkezuidervliet.domain.ext%2Caandeanderekant.domain.ext%2Caanoukk.domain.ext%2Cagdoun.domain.ext%2Caishakuipers.domain.ext%2Cakkelyn.domain.ext%2Calbertinev.domain.ext%2Calexandra1987.domain.ext%2Calva.domain.ext%2Canamika.domain.ext%2Cangelicawibmer.domain.ext%2Cangeliquetijs.domain.ext%2Caniekduyverman.domain.ext%2Cankeroeffen.domain.ext%2Cannekegouw.domain.ext%2Canneliesvanheek.domain.ext%2Canneman.domain.ext%2Cannemiekenlisette.domain.ext%2Cannevelthorst.domain.ext%2Cannewieke.domain.ext%2Canoek82.domain.ext%2Canoukenjulie.domain.ext%2Canoukensjoerd.domain.ext%2Carendenbarbara.domain.ext%2Carieenruubje.domain.ext%2Carindafamilie.domain.ext%2Carjankreeft.domain.ext%2Carletontour.domain.ext
> %2Casiatraveller.domain.ext%2Cassistanatlille.domain.ext%2Caukeenannemieke.domain.ext%2Cavontuurlijkediana.domain.ext%2Cbabeth.domain.ext%2Cbartschillings.domain.ext%2Cbartvanhoorn.domain.ext%2Cbasbronsing.domain.ext%2Cbasenalberto.domain.ext%2Cbasinzuidafrika.domain.ext%2Cbelanda.domain.ext%2Cbenjealintokyo.domain.ext%2Cbertenbeasoest.domain.ext%2Cbianca2007.domain.ext%2Cbiancaenleeinmalta.domain.ext%2Cbiancaf.domain.ext%2Cbienekeenrene.domain.ext%2Cbobhendriks.domain.ext%2Cbonke.domain.ext%2Cboontje.domain.ext%2Cbramvanworkum.domain.ext%2Ccaatjevanrijswijk.domain.ext%2Ccelinevd.domain.ext%2Ccharlyn.domain.ext%2Cchelsey.domain.ext%2Cchina2006.domain.ext%2Cchrisenmarleen.domain.ext%2Cchrisvanhaften.domain.ext%2Ccindyverheij.domain.ext%2Cconnyengeorge.domain.ext%2Cconstantijnbrouwer.domain.ext%2Ccostapiet.domain.ext%2Cdaaninzuidafrika.domain.ext%2Cdaniellewillems.domain.ext%2Cdappel.domain.ext%2Cdebbyatbulamu.domain.ext%2Cdiannehofenk.domain.ext%2Cdickenellenoppad.domain.ext%
> 2Cdickenirene.domain.ext%2Cdidikortekaas.domain.ext%2Cdielange1982.domain.ext%2Cdigigee.domain.ext%2Cdikhuizing.domain.ext%2Cdirkopdefiets.domain.ext%2Cdocent.domain.ext%2Cdorienstraathof.domain.ext%2Cdorijnsuriname.domain.ext%2Cdutchpebble.domain.ext%2Ceddienicole.domain.ext%2Cedenheleen.domain.ext%2Ceelcokarin.domain.ext%2Ceelkeeva.domain.ext%2Celianneansems.domain.ext%2Celkeheurkens.domain.ext%2Cellenenhenkjan.domain.ext%2Celleninnz.domain.ext%2Cellenvandenberg.domain.ext%2Cellisscheinhardt.domain.ext%2Cellyenhans.domain.ext%2Cericenyvonchina.domain.ext%2Cericschuit.domain.ext%2Cestherinoeganda.domain.ext%2Cevamagre.domain.ext%2Ceveliendelijster.domain.ext%2Ceveliensombekke.domain.ext%2Cfajalobi.domain.ext%2Cfeemeel.domain.ext%2Cfemkeennienkeinghana.domain.ext%2Cfennahijlkjealbertine.domain.ext%2Cfernanenlininazie.domain.ext%2Cfleurhinzuidafrika.domain.ext%2Cfop.domain.ext%2Cfranksjak.domain.ext%2Cfudith.domain.ext%2Cganze77venezuela2005.domain.ext%2Cgeerte07.domain.ext%2
> Cgeng.domain.ext%2Cgerbenendorien.domain.ext%2Cgertencisca.domain.ext%2Cgewoonmarin.domain.ext%2Cgezien.domain.ext%2Cgieneneric.domain.ext%2Cgrita.domain.ext%2Cguapa.domain.ext%2Cguill.domain.ext%2Channyenhans.domain.ext%2Chanszweers.domain.ext%2Chasaka.domain.ext%2Chegoestochina.domain.ext%2Chenrikepeters.domain.ext%2Chettyvanewijk.domain.ext%2Cils.domain.ext%2Cinganiet.domain.ext%2Cingekortekaas.domain.ext%2Cjacinthameilink.domain.ext%2Cjasperwildenborg.domain.ext%2Cjennieelam.domain.ext%2Cjossmits.domain.ext%2Ckarindekeijzer.domain.ext
> toRefine=[Ljava.util.List;@35a76700
> response={nl.wikipedia.org=1,corenwilvandee.domain.ext=0,annevandervelde.domain.ext=0,bramvantriest.domain.ext=0,dewilepelaar.domain.ext=0,dirkenjasper.domain.ext=0,verslootinqatar.domain.ext=0,www.kieskeurig.nl=0,yvonnetromp.domain.ext=0,2goglobal.domain.ext=0,888888.domain.ext=0,aafkezuidervliet.domain.ext=0,aandeanderekant.domain.ext^aanoukk.domain.ext=0,agdoun.domain.ext=0,aishakuipers.domain.ext=0,akkelyn.domain.ext=0,albertinev.domain.ext=0,alexandra1987.domain.ext=0,alva.domain.ext=0,anamika.domain.ext=0,angelicawibmer.domain.ext
> angeliquetijs.domain.ext=0,aniekduyverman.domain.ext=0,ankeroeffen.domain.ext=0,annekegouw.domain.ext=0,anneliesvanheek.domain.ext=0,anneman.domain.ext=0,annemiekenlisette.domain.ext=0,annevelthorst.domain.ext=0,annewieke.domain.ext=0,anoek82.domain.ext=0,anoukenjulie.domain.ext=0,anoukensjoerd.domain.ext=0,arendenbarbara.domain.ext=0,arieenruubje.domain.ext=0,arindafamilie.domain.ext=0,arjankreeft.domain.ext=0,arletontour.domain.ext=0,asiatraveller.domain.ext=0,assistanatlille.domain.ext=0,aukeenannemieke.domain.ext=0,avontuurlijkediana.domain.ext=0,babeth.domain.ext=0,bartschillings.domain.ext=0,bartvanhoorn.domain.ext=0,basbronsing.domain.ext=0,basenalberto.domain.ext=0,basinzuidafrika.domain.ext=0,belanda.domain.ext=0,benjealintokyo.domain.ext=0,bertenbeasoest.domain.ext=0,bianca2007.domain.ext=0,biancaenleeinmalta.domain.ext=0,biancaf.domain.ext=0,bienekeenrene.domain.ext=0,bobhendriks.domain.ext=0,bon
> ke.domain.ext=0,boontje.domain.ext=0,bramvanworkum.domain.ext=0,caatjevanrijswijk.domain.ext=0,celinevd.domain.ext=0,charlyn.domain.ext=0,chelsey.domain.ext=0,china2006.domain.ext=0,chrisenmarleen.domain.ext=0,chrisvanhaften.domain.ext=0,cindyverheij.domain.ext=0,connyengeorge.domain.ext=0,constantijnbrouwer.domain.ext=0,costapiet.domain.ext=0,daaninzuidafrika.domain.ext=0,daniellewillems.domain.ext=0,dappel.domain.ext=0,debbyatbulamu.domain.ext=0,diannehofenk.domain.ext=0,dickenellenoppad.domain.ext=0,dickenirene.domain.ext=0,didikortekaas.domain.ext=0,dielange1982.domain.ext=0,digigee.domain.ext=0,dikhuizing.domain.ext=0,dirkopdefiets.domain.ext=0,docent.domain.ext=0,dorienstraathof.domain.ext=0,dorijnsuriname.domain.ext=0,dutchpebble.domain.ext=0,eddienicole.domain.ext=0,edenheleen.domain.ext=0,eelcokarin.domain.ext=0,eelkeeva.domain.ext=0,elianneansems.domain.ext=0,elkeheurkens.domain.ext=0,ellenenhe
> nkjan.domain.ext=0,elleninnz.domain.ext=0,ellenvandenberg.domain.ext=0,ellisscheinhardt.domain.ext=0,ellyenhans.domain.ext=0,ericenyvonchina.domain.ext=0,ericschuit.domain.ext=0,estherinoeganda.domain.ext=0,evamagre.domain.ext=0,eveliendelijster.domain.ext=0,eveliensombekke.domain.ext=0,fajalobi.domain.ext=0,feemeel.domain.ext=0,femkeennienkeinghana.domain.ext=0,fennahijlkjealbertine.domain.ext=0,fernanenlininazie.domain.ext=0,fleurhinzuidafrika.domain.ext=0,fop.domain.ext=0,franksjak.domain.ext=0,fudith.domain.ext=0,ganze77venezuela2005.domain.ext=0,geerte07.domain.ext=0,geng.domain.ext=0,gerbenendorien.domain.ext=0,gertencisca.domain.ext=0,gewoonmarin.domain.ext=0,gezien.domain.ext=0,gieneneric.domain.ext=0,grita.domain.ext=0,guapa.domain.ext=0,guill.domain.ext=0,hannyenhans.domain.ext=0,hanszweers.domain.ext=0,hasaka.domain.ext=0,hegoestochina.domain.ext=0,henrikepeters.domain.ext=0,hettyvanewijk.doma
> in.ext=0,ils.domain.ext=0,inganiet.domain.ext=0,ingekortekaas.domain.ext=0,jacinthameilink.domain.ext=0,jasperwildenborg.domain.ext=0,jennieelam.domain.ext=0,jossmits.domain.ext=0,karindekeijzer.domain.ext=0}
>
> Before opening an issue i'd like to know in which direction to look.
>
> Thanks,
> Markus
>
>