Field collapsing (SOLR-236) and Solr 1.3.0 release version

[Stephen Weiss <sw...@stylesight.com>]

Hi,

A requirement has come up in a project where we're going to need to  
group by a field in the result set.  I looked into the SOLR-236 patch  
and it seems there are a couple versions out now that are supposed to  
work against the Solr 1.3.0 release.

This is a production site, it really can't be running anything that's  
going to crash or take up too many resources.  I wanted to check with  
the list and see if anyone is using this patch with the Solr 1.3.0  
release and if it is stable enough / performs well enough for serious  
usage.  We have an index of 3M+ documents and a grouped result set  
would be about 50-75% the total size of the ungrouped results.

Thanks for any information or pointers.

--
Steve Weiss
Stylesight

Re: Field collapsing (SOLR-236) and Solr 1.3.0 release version

[Ryan McKinley <ry...@gmail.com>]

> if only i could magic all these damn pdfs I have into some code :)

+1

I want some of that magic too!

Re: Field collapsing (SOLR-236) and Solr 1.3.0 release version

[Mark Miller <ma...@gmail.com>]

gurudev wrote:
>
> One thing that you can go with is using "adjacent" field collapsing rather
> than simple collapsing. As internally SOLR would first sort on the collapse
> field to use simple collapsing, which is not the case with "adjacent"
> collapsing.
>   
This something that I think could be improved in that patch (non 
adjacent collapsing). There is a lot of work done on this as its 
essentially a groupby in sql and shares a lot of the issues of an sql 
join. The current strategy is to use two sorts - thats certainly a valid 
approach, but I think there are much better possibilities. Again, I have 
like half a dozen PDFs on possible algorithms or whatnot if anyone is 
interested - if only i could magic all these damn pdfs I have into some 
code :)

Re: Field collapsing (SOLR-236) and Solr 1.3.0 release version

[gurudev <su...@yahoo.com>]

We are about to release Field collapsing in our production site, but the
index size is not as big as yours.
Definitely collapsing is an added overhead. You can do some load testing and
bench mark on some dataset as you would expect on your production project as
SOLR-236 is currently available only as patch.

Secondly, provide mechanism so you can switch it off/on depending on the
condition of the production servers.

One thing that you can go with is using "adjacent" field collapsing rather
than simple collapsing. As internally SOLR would first sort on the collapse
field to use simple collapsing, which is not the case with "adjacent"
collapsing.



Stephen Weiss-2 wrote:
> 
> Hi,
> 
> A requirement has come up in a project where we're going to need to  
> group by a field in the result set.  I looked into the SOLR-236 patch  
> and it seems there are a couple versions out now that are supposed to  
> work against the Solr 1.3.0 release.
> 
> This is a production site, it really can't be running anything that's  
> going to crash or take up too many resources.  I wanted to check with  
> the list and see if anyone is using this patch with the Solr 1.3.0  
> release and if it is stable enough / performs well enough for serious  
> usage.  We have an index of 3M+ documents and a grouped result set  
> would be about 50-75% the total size of the ungrouped results.
> 
> Thanks for any information or pointers.
> 
> --
> Steve Weiss
> Stylesight
> 
> 

-- 
View this message in context: http://www.nabble.com/Field-collapsing-%28SOLR-236%29-and-Solr-1.3.0-release-version-tp20595266p20600959.html
Sent from the Solr - User mailing list archive at Nabble.com.