You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by rajini maski <ra...@gmail.com> on 2011/05/27 07:08:04 UTC
Query regarding Solr-2242 patch for getting distinct facet counts.
The patch solr 2242 for getting count of distinct facet terms doesn't
work for distributedProcess
(https://issues.apache.org/jira/browse/SOLR-2242)
The error log says
HTTP ERROR 500
Problem accessing /solr/select. Reason:
For input string: "numFacetTerms"
java.lang.NumberFormatException: For input string: "numFacetTerms"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:403)
at java.lang.Long.parseLong(Long.java:461)
at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331)
at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344)
at
org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(FacetComponent.java:619)
at
org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponent.java:265)
at
org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:235)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:290)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
The query I passed :
http://localhost:8983/solr/select?q=*:*&facet=true&facet.field=2&facet.field=648&facet.mincount=1&facet.limit=-1&f.2.facet.numFacetTerms=1&rows=0&shards=localhost:8983/solr,localhost:8985/solrtwo
Anyone can suggest me the changes i need to make to enable the same
funcionality for shards?
When i do it across single core.. I get the correct results. I have applied
the solr 2242 patch in solr1.4.1
Awaiting for reply
Regards,
Rajani
Re: Query regarding Solr-2242 patch for getting distinct facet counts.
Posted by rajini maski <ra...@gmail.com>.
In solr 1.4.1, for getting "distinct facet terms count" across shards,
The piece of code added for getting count of distinct facet terms across
distributed process is as followed:
Class: facetcomponent.java
Function: -- finishStage(ResponseBuilder rb)
for (DistribFieldFacet dff : fi.facets.values()) {
//just after this line of code
else { // TODO: log error or throw exception?
counts = dff.getLexSorted();
int namedistint = 0;
namedistint=rb.req.getParams().getFieldInt(dff.getKey().toString(),FacetParams.FACET_NAMEDISTINCT,0);
if (namedistint == 0)
facet_fields.add(dff.getKey(), fieldCounts);
if (namedistint == 1)
facet_fields.add("numfacetTerms", counts.length);
if (namedistint == 2) {
NamedList resCount = new NamedList();
resCount.add("numfacetTerms", counts.length);
resCount.add("counts", fieldCounts);
facet_fields.add(dff.getKey(), resCount);
}
Is this flow correct ? I have worked with few test cases and it has worked
fine. but i want to know if there are any bugs that can creep in here? (My
concern is this piece of code should not effect the rest of logic)
*Code flow with comments for reference:*
Function : -- finishStage(ResponseBuilder rb)
//in this for loop ,
for (DistribFieldFacet dff : fi.facets.values()) {
//just after this line of code
else { // TODO: log error or throw exception?
counts = dff.getLexSorted();
int namedistint = 0; //default
//get the value of facet.numterms from the input query
namedistint=rb.req.getParams().getFieldInt(dff.getKey().toString(),FacetParams.FACET_NAMEDISTINCT,0);
// based on the value for facet.numterms==0 or 1 or 2 , if conditions
//Get only facet field counts
if (namedistint == 0)
{
facet_fields.add(dff.getKey(), fieldCounts);
}
//get only distinct facet term count
if (namedistint == 1)
{
facet_fields.add("numfacetTerms", counts.length);
}
//get facet field count and distinct term count.
if (namedistint == 2) {
NamedList resCount = new NamedList();
resCount.add("numfacetTerms", counts.length);
resCount.add("counts", fieldCounts);
facet_fields.add(dff.getKey(), resCount);
}
Regards,
Rajani
On Fri, May 27, 2011 at 1:14 PM, rajini maski <ra...@gmail.com> wrote:
> No such issues . Successfully integrated with 1.4.1 and it works across
> single index.
>
> for f.2.facet.numFacetTerms=1 parameter it will give the distinct count
> result
>
> for f.2.facet.numFacetTerms=2 parameter it will give counts as well as
> results for facets.
>
> But this is working only across single index not distributed process. The
> conditions you have added in simple facet.java- "if namedistinct count ==int
> " ( 0, 1 and 2 condtions).. Should it be added in distributed process
> function to enable it work across shards?
>
> Rajani
>
>
>
> On Fri, May 27, 2011 at 12:33 PM, Bill Bell <bi...@gmail.com> wrote:
>
>> I am pretty sure it does not yet support distributed shards..
>>
>> But the patch was written for 4.0... So there might be issues with running
>> it on 1.4.1.
>>
>> On 5/26/11 11:08 PM, "rajini maski" <ra...@gmail.com> wrote:
>>
>> > The patch solr 2242 for getting count of distinct facet terms
>> doesn't
>> >work for distributedProcess
>> >
>> >(https://issues.apache.org/jira/browse/SOLR-2242)
>> >
>> >The error log says
>> >
>> > HTTP ERROR 500
>> >Problem accessing /solr/select. Reason:
>> >
>> > For input string: "numFacetTerms"
>> >
>> >java.lang.NumberFormatException: For input string: "numFacetTerms"
>> >at
>>
>> >java.lang.NumberFormatException.forInputString(NumberFormatException.java:
>> >48)
>> >at java.lang.Long.parseLong(Long.java:403)
>> >at java.lang.Long.parseLong(Long.java:461)
>> >at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331)
>> >at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344)
>> >at
>>
>> >org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(Fac
>> >etComponent.java:619)
>> >at
>>
>> >org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponen
>> >t.java:265)
>> >at
>>
>> >org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComp
>> >onent.java:235)
>> >at
>>
>> >org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHa
>> >ndler.java:290)
>> >at
>>
>> >org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas
>> >e.java:131)
>> >at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>> >at
>>
>> >org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java
>> >:338)
>> >at
>>
>> >org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav
>> >a:241)
>> >at
>>
>> >org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl
>> >er.java:1212)
>> >at
>> >org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>> >at
>>
>> >org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216
>> >)
>> >at
>> >org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>> >at
>> >org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>> >at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>> >at
>> >org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> >at org.mortbay.jetty.Server.handle(Server.java:326)
>> >at
>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> >at
>>
>> >org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnect
>> >ion.java:928)
>> >at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>> >at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>> >at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> >at
>>
>> >org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:41
>> >0)
>> >at
>>
>> >org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:5
>> >82)
>> >
>> >
>> >The query I passed :
>> >
>> http://localhost:8983/solr/select?q=*:*&facet=true&facet.field=2&facet.fie
>>
>> >ld=648&facet.mincount=1&facet.limit=-1&f.2.facet.numFacetTerms=1&rows=0&sh
>> >ards=localhost:8983/solr,localhost:8985/solrtwo
>> >
>> >Anyone can suggest me the changes i need to make to enable the same
>> >funcionality for shards?
>> >
>> >When i do it across single core.. I get the correct results. I have
>> >applied
>> >the solr 2242 patch in solr1.4.1
>> >
>> >Awaiting for reply
>> >
>> >Regards,
>> >Rajani
>>
>>
>>
>
Re: Query regarding Solr-2242 patch for getting distinct facet counts.
Posted by rajini maski <ra...@gmail.com>.
No such issues . Successfully integrated with 1.4.1 and it works across
single index.
for f.2.facet.numFacetTerms=1 parameter it will give the distinct count
result
for f.2.facet.numFacetTerms=2 parameter it will give counts as well as
results for facets.
But this is working only across single index not distributed process. The
conditions you have added in simple facet.java- "if namedistinct count ==int
" ( 0, 1 and 2 condtions).. Should it be added in distributed process
function to enable it work across shards?
Rajani
On Fri, May 27, 2011 at 12:33 PM, Bill Bell <bi...@gmail.com> wrote:
> I am pretty sure it does not yet support distributed shards..
>
> But the patch was written for 4.0... So there might be issues with running
> it on 1.4.1.
>
> On 5/26/11 11:08 PM, "rajini maski" <ra...@gmail.com> wrote:
>
> > The patch solr 2242 for getting count of distinct facet terms doesn't
> >work for distributedProcess
> >
> >(https://issues.apache.org/jira/browse/SOLR-2242)
> >
> >The error log says
> >
> > HTTP ERROR 500
> >Problem accessing /solr/select. Reason:
> >
> > For input string: "numFacetTerms"
> >
> >java.lang.NumberFormatException: For input string: "numFacetTerms"
> >at
> >java.lang.NumberFormatException.forInputString(NumberFormatException.java:
> >48)
> >at java.lang.Long.parseLong(Long.java:403)
> >at java.lang.Long.parseLong(Long.java:461)
> >at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331)
> >at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344)
> >at
> >org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(Fac
> >etComponent.java:619)
> >at
> >org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponen
> >t.java:265)
> >at
> >org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComp
> >onent.java:235)
> >at
> >org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHa
> >ndler.java:290)
> >at
> >org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas
> >e.java:131)
> >at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> >at
> >org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java
> >:338)
> >at
> >org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav
> >a:241)
> >at
> >org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl
> >er.java:1212)
> >at
> >org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> >at
> >org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216
> >)
> >at
> >org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> >at
> >org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> >at
> >org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >at org.mortbay.jetty.Server.handle(Server.java:326)
> >at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> >at
> >org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnect
> >ion.java:928)
> >at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> >at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> >at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> >at
> >org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:41
> >0)
> >at
> >org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:5
> >82)
> >
> >
> >The query I passed :
> >
> http://localhost:8983/solr/select?q=*:*&facet=true&facet.field=2&facet.fie
> >ld=648&facet.mincount=1&facet.limit=-1&f.2.facet.numFacetTerms=1&rows=0&sh
> >ards=localhost:8983/solr,localhost:8985/solrtwo
> >
> >Anyone can suggest me the changes i need to make to enable the same
> >funcionality for shards?
> >
> >When i do it across single core.. I get the correct results. I have
> >applied
> >the solr 2242 patch in solr1.4.1
> >
> >Awaiting for reply
> >
> >Regards,
> >Rajani
>
>
>
Re: Query regarding Solr-2242 patch for getting distinct facet
counts.
Posted by Bill Bell <bi...@gmail.com>.
I am pretty sure it does not yet support distributed shards..
But the patch was written for 4.0... So there might be issues with running
it on 1.4.1.
On 5/26/11 11:08 PM, "rajini maski" <ra...@gmail.com> wrote:
> The patch solr 2242 for getting count of distinct facet terms doesn't
>work for distributedProcess
>
>(https://issues.apache.org/jira/browse/SOLR-2242)
>
>The error log says
>
> HTTP ERROR 500
>Problem accessing /solr/select. Reason:
>
> For input string: "numFacetTerms"
>
>java.lang.NumberFormatException: For input string: "numFacetTerms"
>at
>java.lang.NumberFormatException.forInputString(NumberFormatException.java:
>48)
>at java.lang.Long.parseLong(Long.java:403)
>at java.lang.Long.parseLong(Long.java:461)
>at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331)
>at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344)
>at
>org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(Fac
>etComponent.java:619)
>at
>org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponen
>t.java:265)
>at
>org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComp
>onent.java:235)
>at
>org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHa
>ndler.java:290)
>at
>org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas
>e.java:131)
>at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>at
>org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java
>:338)
>at
>org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav
>a:241)
>at
>org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl
>er.java:1212)
>at
>org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>at
>org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216
>)
>at
>org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>at
>org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>at
>org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>at org.mortbay.jetty.Server.handle(Server.java:326)
>at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>at
>org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnect
>ion.java:928)
>at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>at
>org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:41
>0)
>at
>org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:5
>82)
>
>
>The query I passed :
>http://localhost:8983/solr/select?q=*:*&facet=true&facet.field=2&facet.fie
>ld=648&facet.mincount=1&facet.limit=-1&f.2.facet.numFacetTerms=1&rows=0&sh
>ards=localhost:8983/solr,localhost:8985/solrtwo
>
>Anyone can suggest me the changes i need to make to enable the same
>funcionality for shards?
>
>When i do it across single core.. I get the correct results. I have
>applied
>the solr 2242 patch in solr1.4.1
>
>Awaiting for reply
>
>Regards,
>Rajani