You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Erik Hatcher (Created) (JIRA)" <ji...@apache.org> on 2011/11/13 12:21:53 UTC
[jira] [Created] (SOLR-2894) Implement distributed pivot faceting
Implement distributed pivot faceting
------------------------------------
Key: SOLR-2894
URL: https://issues.apache.org/jira/browse/SOLR-2894
Project: Solr
Issue Type: Improvement
Affects Versions: 4.0
Reporter: Erik Hatcher
Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Antoine Le Floc'h (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190915#comment-13190915 ]
Antoine Le Floc'h commented on SOLR-2894:
-----------------------------------------
Do you think that this will be available for Solr 4.0 ? I would think that this is very similar to distributing regular facets ?
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Ben Roubicek (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154368#comment-13154368 ]
Ben Roubicek commented on SOLR-2894:
------------------------------------
Based on SOLR-792, it looked like there was some traction in getting distributed pivoting in the trunk codebase beyond the functional prototype. This feature has a lot of value within my company where we perform 50 separate queries where one would suffice if we had distributed pivot support.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Dan Cooper (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222450#comment-13222450 ]
Dan Cooper commented on SOLR-2894:
----------------------------------
Added a patch to provide distributed pivot faceting. We've been running this code for a while now and it seems to work OK, also created a unit test to test distributed pivot faceting on a small set of data.
The patch was created against Solr trunk revision 1297102.
It should perform in much the same way as single shard pivot faceting. It only sorts by count if you specify that option otherwise it returns results in the order they were generated (may be useful is performance is important but ordering is not). Most will want to specify facet.sort=count. This patch also supports limiting results using facet.limit.
To do the merge I'm converting the NamedList objects that get returned by each shard in a giant map (should be more efficient for merging the results) and then converting back into a NamedList when the merge is complete. This merge should support N depth pivots but I've only properly tested a depth of 2.
I've added some new parameters to support the features we require from pivot faceting and thought they may as well go in the patch in case others find them useful.
* facet.pivot.limit.method
** set to 'combined' if you want only the N number of top results to be returned across all pivots, where N is set by facet.limit. e.g. if you pivoted by country,manufacturer and limited by 5, obviously the top 5 countries would be returned, but only the top 5 manufacturers by combined total would be returned too. e.g. Each country would return the same 5 manufacturers (or less if no results).
* facet.pivot.limit.ignore
** Ignores the specified field from the limiting operations. e.g. if you pivoted by country,manufacturer and limited by 5 and set facet.pivot.limit.ignore=country then you would get all available countries returned (not limited) but only 5 manufacturers for each country.
Can someone test the patch and give some feedback?
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
> Attachments: distribPatch-05-03-12.txt
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260994#comment-13260994 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
I figured out the unit test. It's because facet_pivot is different from the other facet_blah in that it only comes back when you request pivots, whereas the others always come back. In Dan's patch he had facet_pivot coming back even when it was empty or not requested, and this did not match the behavior of pivots in a non-distributed setting. I am working on an update to my patch.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Russell updated SOLR-2894:
--------------------------------
Attachment: (was: distributed_pivot.patch)
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Russell updated SOLR-2894:
--------------------------------
Attachment: SOLR-2894.patch
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: distributed_pivot.patch, distributed_pivot.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492619#comment-13492619 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
In regards to my above comment, I have determined that it is because if you specify a limit for a field that you are not requesting facet counts for, solr will not automatically over-request on that field.
i.e.
facet.pivot=somefield
f.somefield.facet.limit=10
This will make your pivots weird because the limit of 10 will not be over requested unless you add this line:
facet.field=somefield
Since solr does not do distributed pivoting yet, this has not been an issue yet.
I am working on an update to the patch that will correct this issue.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: distributed_pivot.patch, distributed_pivot.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Comment Edited] (SOLR-2894) Implement distributed pivot
faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495296#comment-13495296 ]
Chris Russell edited comment on SOLR-2894 at 11/12/12 3:10 PM:
---------------------------------------------------------------
Updated to apply to trunk 1404975. (Based on Dzmitry's update)
Added ability to limit on individual fields. (f.fieldname.facet.limit)
Added pivot fields to fields being over-requested during distributed queries.
Made it so you can pivot on a single field.
was (Author: selah):
Updated to apply to trunk 1404975.
Added ability to limit on individual fields. (f.fieldname.facet.limit)
Added pivot fields to fields being over-requested during distributed queries.
Made it so you can pivot on a single field.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295426#comment-13295426 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
Erik, I can't get your patch to apply cleanly to solr 1350445
$ patch -p0 -i SOLR-2894.patch
patching file solr/core/src/test/org/apache/solr/handler/component/DistributedFacetPivotTest.java
patching file solr/core/src/java/org/apache/solr/handler/component/EntryCountComparator.java
patching file solr/core/src/java/org/apache/solr/handler/component/PivotNamedListCountComparator.java
patching file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java
Hunk #2 FAILED at 103.
1 out of 2 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java
Hunk #11 FAILED at 799.
1 out of 17 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java.rej
patching file solr/core/src/java/org/apache/solr/util/PivotListEntry.java
patching file solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java
patching file solr/test-framework/src/java/org/apache/solr/BaseDistributedSearchTestCase.java
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Dan Cooper (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Cooper updated SOLR-2894:
-----------------------------
Attachment: distribPatch-05-03-12.txt
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
> Attachments: distribPatch-05-03-12.txt
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482734#comment-13482734 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
In my experience using this patch, it seems that it does not over-request when enforcing a limit?
This is problematic because, for example, in a situation where you have many slaves and you are pivoting on a fairly evenly distributed field and setting your facet limit to X, the Xth distinct value for that field by document count on each slave is likely to be different. The result is that some facet values close to your limit boundary will not get reported for aggregation, which will make your ultimate results somewhat inaccurate.
It was my impression that other facet-based features of solr over-request when there is a limit to combat this situation? For example if you specify limit 10, the distributed query might have limit 100 or 1000, and then during aggregation it would be limited to the top 10.
I am working on similar functionality for this patch.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: distributed_pivot.patch, distributed_pivot.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295026#comment-13295026 ]
Erik Hatcher commented on SOLR-2894:
------------------------------------
Trey - thanks for the positive feedback. I'll apply the patch, run the tests, review the code, and so on. Might be a couple of weeks, unless I can get to this today.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Russell updated SOLR-2894:
--------------------------------
Attachment: (was: distributed_pivot.patch)
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-2894) Implement distributed
pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259947#comment-13259947 ]
Chris Russell edited comment on SOLR-2894 at 4/23/12 9:05 PM:
--------------------------------------------------------------
Hi Dan.
I have been working with your patch, 2894, to Solr and I am having some issues with unit testing.
First of all the patch doesn't seem to apply cleanly:
crussell@WAT-CRUSSELL /cygdrive/d/matrixdev/solr_1297102/CBSolr/SolrLucene
$ patch -p0 -i SOLR-2894.patch
patching file solr/core/src/java/org/apache/solr/handler/component/EntryCountComparator.java
patching file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java
Hunk #10 FAILED at 797.
1 out of 16 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java
Hunk #2 FAILED at 106.
1 out of 2 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotNamedListCountComparator.java
patching file solr/core/src/java/org/apache/solr/util/NamedListHelper.java
patching file solr/core/src/java/org/apache/solr/util/PivotListEntry.java
patching file solr/core/src/test/org/apache/solr/handler/component/DistributedFacetPivotTest.java
patching file solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java
$ patch --version
patch 2.5.8
Copyright (C) 1988 Larry Wall
Copyright (C) 2002 Free Software Foundation, Inc.
A lot of the contents of your original patch seemed to be formatting changes like where line breaks should go and how spacing should be handled.
I was able to examine the patch and manually incorporate the changes from the failed chunks.
One question, on this line (1119) of your patch, why did you choose not to initialize the map as the ones above it are? Couldn't that cause an NRE?
+ public SimpleOrderedMap<List<NamedList<Object>>> pivotFacets;
While looking at DistributedFacetPivotTest.java I noticed an error on line 42. Your "q" should probably be "*:*" instead of "*". Edit: asterisk colon asterisk instead of just asterisk.
I've attached the patch file I came up with. I added the initialization and test correction I mentioned.
When I ran the solr/lucene unit tests after I patched, there are some unit test failures like this one:
[junit] Testsuite: org.apache.solr.TestDistributedGrouping
[junit] Testcase: testDistribSearch(org.apache.solr.TestDistributedGrouping): FAILED
[junit] .facet_counts.size()==5,4skipped=0,0
[junit] junit.framework.AssertionFailedError: .facet_counts.size()==5,4skipped=0,0
[junit] at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:656)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:383)
[junit] at org.apache.solr.TestDistributedGrouping.doTest(TestDistributedGrouping.java:51)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:671)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:736)
[junit] at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:632)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit] at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:531)
[junit] at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(LuceneTestCase.java:593)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:165)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:57)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit]
[junit]
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 2.685 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] 2520 T1 oas.BaseDistributedSearchTestCase.compareResponses SEVERE Mismatched responses:
[junit] {responseHeader={status=0,QTime=19},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={},facet_pivot={}}}
[junit] {responseHeader={status=0,QTime=18},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={}}}
[junit] NOTE: reproduce with: ant test -Dtestcase=TestDistributedGrouping -Dtestmethod=testDistribSearch -Dtests.seed=-c7cfa73dbca93c9:-4751af558bf6f59:3e523b50870b3b1b -Dargs="-Dfile.encoding=Cp1252"
It looks like the facet_pivot is being included in the results of one query, and not the other. I'm trying to figure out why this is occurring.
Any insight would be appreciated.
was (Author: selah):
Hi Dan.
I have been working with your patch, 2894, to Solr and I am having some issues with unit testing.
First of all the patch doesn't seem to apply cleanly:
crussell@WAT-CRUSSELL /cygdrive/d/matrixdev/solr_1297102/CBSolr/SolrLucene
$ patch -p0 -i SOLR-2894.patch
patching file solr/core/src/java/org/apache/solr/handler/component/EntryCountComparator.java
patching file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java
Hunk #10 FAILED at 797.
1 out of 16 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java
Hunk #2 FAILED at 106.
1 out of 2 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotNamedListCountComparator.java
patching file solr/core/src/java/org/apache/solr/util/NamedListHelper.java
patching file solr/core/src/java/org/apache/solr/util/PivotListEntry.java
patching file solr/core/src/test/org/apache/solr/handler/component/DistributedFacetPivotTest.java
patching file solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java
$ patch --version
patch 2.5.8
Copyright (C) 1988 Larry Wall
Copyright (C) 2002 Free Software Foundation, Inc.
A lot of the contents of your original patch seemed to be formatting changes like where line breaks should go and how spacing should be handled.
I was able to examine the patch and manually incorporate the changes from the failed chunks.
One question, on this line (1119) of your patch, why did you choose not to initialize the map as the ones above it are? Couldn't that cause an NRE?
+ public SimpleOrderedMap<List<NamedList<Object>>> pivotFacets;
While looking at DistributedFacetPivotTest.java I noticed an error on line 42. Your "q" should probably be *:* instead of "*".
I've attached the patch file I came up with. I added the initialization and test correction I mentioned.
When I ran the solr/lucene unit tests after I patched, there are some unit test failures like this one:
[junit] Testsuite: org.apache.solr.TestDistributedGrouping
[junit] Testcase: testDistribSearch(org.apache.solr.TestDistributedGrouping): FAILED
[junit] .facet_counts.size()==5,4skipped=0,0
[junit] junit.framework.AssertionFailedError: .facet_counts.size()==5,4skipped=0,0
[junit] at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:656)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:383)
[junit] at org.apache.solr.TestDistributedGrouping.doTest(TestDistributedGrouping.java:51)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:671)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:736)
[junit] at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:632)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit] at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:531)
[junit] at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(LuceneTestCase.java:593)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:165)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:57)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit]
[junit]
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 2.685 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] 2520 T1 oas.BaseDistributedSearchTestCase.compareResponses SEVERE Mismatched responses:
[junit] {responseHeader={status=0,QTime=19},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={},facet_pivot={}}}
[junit] {responseHeader={status=0,QTime=18},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={}}}
[junit] NOTE: reproduce with: ant test -Dtestcase=TestDistributedGrouping -Dtestmethod=testDistribSearch -Dtests.seed=-c7cfa73dbca93c9:-4751af558bf6f59:3e523b50870b3b1b -Dargs="-Dfile.encoding=Cp1252"
It looks like the facet_pivot is being included in the results of one query, and not the other. I'm trying to figure out why this is occurring.
Any insight would be appreciated.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Russell updated SOLR-2894:
--------------------------------
Attachment: SOLR-2894.patch
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Russell updated SOLR-2894:
--------------------------------
Attachment: distributed_pivot.patch
facet_pivot will not show up in distrib search if no contents, reversed behavior of sorting to comply with solr standard for facet.sort
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Hatcher updated SOLR-2894:
-------------------------------
Attachment: SOLR-2894.patch
Patch updated to 4x branch.
Simon, just for you, I removed NamedListHelper as well :) (folded its one method into PivotFacetHelper)
Tests pass.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Mark Miller (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-2894:
------------------------------
Affects Version/s: (was: 4.0)
Fix Version/s: 4.0
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Comment Edited] (SOLR-2894) Implement distributed pivot
faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455182#comment-13455182 ]
Chris Russell edited comment on SOLR-2894 at 10/23/12 9:45 PM:
---------------------------------------------------------------
Regarding facet.pivot.limit.method and facet.limit, it looks like these are not checked on a per-field basis?
So, if a user sets different limits for different fields and wants 'combined' limiting, that is not possible?
For example a user might set:
f.field1.facet.limit=10
f.field1.facet.pivot.limit.method=combined
f.field2.facet.limit=20
And the combined method will not be used...
If the user sets facet.pivot.limit.method=combined it looks like the same limit will be used for all fields? Whatever the global facet.limit is set to?
was (Author: selah):
Regarding facet.pivot.limit.method and facet.limit, it looks like these are not checked on a per-field basis?
So, if a user sets different limits for different fields and wants 'combined' limiting, that is not possible?
For example a user might set:
f.field1.facet.limit=10
f.field1.facet.pivot.limit.method=combined
f.field2.facet.limit=20
And the combined method will not be used...
If the user sets facet.pivot.limit.method=combined it looks like the same limit will be used for all fields? Whatever the global facet.limit is set to?
Unfortunate.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: distributed_pivot.patch, distributed_pivot.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295608#comment-13295608 ]
Erik Hatcher commented on SOLR-2894:
------------------------------------
Chris - I generated the patch from an r1350348 checkout (which is not branch_4x as I mentioned above). It might be a couple of weeks before I can get back to this and sort it out though, unfortunately. Sorry.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Dan Cooper (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Cooper updated SOLR-2894:
-----------------------------
Attachment: SOLR-2894.patch
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Dan Cooper (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Cooper updated SOLR-2894:
-----------------------------
Attachment: (was: distribPatch-05-03-12.txt)
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Dan Cooper (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Cooper updated SOLR-2894:
-----------------------------
Attachment: SOLR-2894.patch
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
> Attachments: SOLR-2894.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-2894) Implement distributed
pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259947#comment-13259947 ]
Chris Russell edited comment on SOLR-2894 at 4/23/12 9:04 PM:
--------------------------------------------------------------
Hi Dan.
I have been working with your patch, 2894, to Solr and I am having some issues with unit testing.
First of all the patch doesn't seem to apply cleanly:
crussell@WAT-CRUSSELL /cygdrive/d/matrixdev/solr_1297102/CBSolr/SolrLucene
$ patch -p0 -i SOLR-2894.patch
patching file solr/core/src/java/org/apache/solr/handler/component/EntryCountComparator.java
patching file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java
Hunk #10 FAILED at 797.
1 out of 16 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java
Hunk #2 FAILED at 106.
1 out of 2 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotNamedListCountComparator.java
patching file solr/core/src/java/org/apache/solr/util/NamedListHelper.java
patching file solr/core/src/java/org/apache/solr/util/PivotListEntry.java
patching file solr/core/src/test/org/apache/solr/handler/component/DistributedFacetPivotTest.java
patching file solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java
$ patch --version
patch 2.5.8
Copyright (C) 1988 Larry Wall
Copyright (C) 2002 Free Software Foundation, Inc.
A lot of the contents of your original patch seemed to be formatting changes like where line breaks should go and how spacing should be handled.
I was able to examine the patch and manually incorporate the changes from the failed chunks.
One question, on this line (1119) of your patch, why did you choose not to initialize the map as the ones above it are? Couldn't that cause an NRE?
+ public SimpleOrderedMap<List<NamedList<Object>>> pivotFacets;
While looking at DistributedFacetPivotTest.java I noticed an error on line 42. Your "q" should probably be *:* instead of "*".
I've attached the patch file I came up with. I added the initialization and test correction I mentioned.
When I ran the solr/lucene unit tests after I patched, there are some unit test failures like this one:
[junit] Testsuite: org.apache.solr.TestDistributedGrouping
[junit] Testcase: testDistribSearch(org.apache.solr.TestDistributedGrouping): FAILED
[junit] .facet_counts.size()==5,4skipped=0,0
[junit] junit.framework.AssertionFailedError: .facet_counts.size()==5,4skipped=0,0
[junit] at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:656)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:383)
[junit] at org.apache.solr.TestDistributedGrouping.doTest(TestDistributedGrouping.java:51)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:671)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:736)
[junit] at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:632)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit] at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:531)
[junit] at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(LuceneTestCase.java:593)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:165)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:57)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit]
[junit]
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 2.685 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] 2520 T1 oas.BaseDistributedSearchTestCase.compareResponses SEVERE Mismatched responses:
[junit] {responseHeader={status=0,QTime=19},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={},facet_pivot={}}}
[junit] {responseHeader={status=0,QTime=18},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={}}}
[junit] NOTE: reproduce with: ant test -Dtestcase=TestDistributedGrouping -Dtestmethod=testDistribSearch -Dtests.seed=-c7cfa73dbca93c9:-4751af558bf6f59:3e523b50870b3b1b -Dargs="-Dfile.encoding=Cp1252"
It looks like the facet_pivot is being included in the results of one query, and not the other. I'm trying to figure out why this is occurring.
Any insight would be appreciated.
was (Author: selah):
Hi Dan.
I have been working with your patch, 2894, to Solr and I am having some issues with unit testing.
First of all the patch doesn't seem to apply cleanly:
crussell@WAT-CRUSSELL /cygdrive/d/matrixdev/solr_1297102/CBSolr/SolrLucene
$ patch -p0 -i SOLR-2894.patch
patching file solr/core/src/java/org/apache/solr/handler/component/EntryCountComparator.java
patching file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java
Hunk #10 FAILED at 797.
1 out of 16 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java
Hunk #2 FAILED at 106.
1 out of 2 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotNamedListCountComparator.java
patching file solr/core/src/java/org/apache/solr/util/NamedListHelper.java
patching file solr/core/src/java/org/apache/solr/util/PivotListEntry.java
patching file solr/core/src/test/org/apache/solr/handler/component/DistributedFacetPivotTest.java
patching file solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java
$ patch --version
patch 2.5.8
Copyright (C) 1988 Larry Wall
Copyright (C) 2002 Free Software Foundation, Inc.
A lot of the contents of your original patch seemed to be formatting changes like where line breaks should go and how spacing should be handled.
I was able to examine the patch and manually incorporate the changes from the failed chunks.
One question, on this line (1119) of your patch, why did you choose not to initialize the map as the ones above it are? Couldn't that cause an NRE?
+ public SimpleOrderedMap<List<NamedList<Object>>> pivotFacets;
While looking at DistributedFacetPivotTest.java I noticed an error on line 42. Your "q" should probably be "*:*" instead of "*".
I've attached the patch file I came up with. I added the initialization and test correction I mentioned.
When I ran the solr/lucene unit tests after I patched, there are some unit test failures like this one:
[junit] Testsuite: org.apache.solr.TestDistributedGrouping
[junit] Testcase: testDistribSearch(org.apache.solr.TestDistributedGrouping): FAILED
[junit] .facet_counts.size()==5,4skipped=0,0
[junit] junit.framework.AssertionFailedError: .facet_counts.size()==5,4skipped=0,0
[junit] at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:656)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:383)
[junit] at org.apache.solr.TestDistributedGrouping.doTest(TestDistributedGrouping.java:51)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:671)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:736)
[junit] at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:632)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit] at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:531)
[junit] at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(LuceneTestCase.java:593)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:165)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:57)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit]
[junit]
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 2.685 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] 2520 T1 oas.BaseDistributedSearchTestCase.compareResponses SEVERE Mismatched responses:
[junit] {responseHeader={status=0,QTime=19},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={},facet_pivot={}}}
[junit] {responseHeader={status=0,QTime=18},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={}}}
[junit] NOTE: reproduce with: ant test -Dtestcase=TestDistributedGrouping -Dtestmethod=testDistribSearch -Dtests.seed=-c7cfa73dbca93c9:-4751af558bf6f59:3e523b50870b3b1b -Dargs="-Dfile.encoding=Cp1252"
It looks like the facet_pivot is being included in the results of one query, and not the other. I'm trying to figure out why this is occurring.
Any insight would be appreciated.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495296#comment-13495296 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
Updated to apply to trunk 1404975.
Added ability to limit on individual fields. (f.fieldname.facet.limit)
Added pivot fields to fields being over-requested during distributed queries.
Made it so you can pivot on a single field.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: distributed_pivot.patch, distributed_pivot.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Russell updated SOLR-2894:
--------------------------------
Attachment: distributed_pivot.patch
Some modifications to SOLR-2894.patch that I made while trying to get it to patch on rev 1297102.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295261#comment-13295261 ]
Erik Hatcher commented on SOLR-2894:
------------------------------------
Trey - would you be in a position to test out the latest patch? I built my latest one by starting with the March 5, 2012 SOLR-2894.patch file.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (SOLR-2894) Implement distributed pivot faceting
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man reassigned SOLR-2894:
------------------------------
Assignee: Erik Hatcher
Erik: Can you triage this for 4.0? commit if you think it's ready, otherwise remove the fix version?
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Dan Cooper (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Cooper updated SOLR-2894:
-----------------------------
Attachment: (was: SOLR-2894.patch)
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 4.0
> Reporter: Erik Hatcher
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Russell updated SOLR-2894:
--------------------------------
Attachment: (was: SOLR-2894.patch)
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Trey Grainger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294795#comment-13294795 ]
Trey Grainger commented on SOLR-2894:
-------------------------------------
For what it's worth, we're actively using the April 25th version of this patch in production at CareerBuilder (with an older version of trunk) with no issues.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455182#comment-13455182 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
Regarding facet.pivot.limit.method and facet.limit, it looks like these are not checked on a per-field basis?
So, if a user sets different limits for different fields and wants 'combined' limiting, that is not possible?
For example a user might set:
f.field1.facet.limit=10
f.field1.facet.pivot.limit.method=combined
f.field2.facet.limit=20
And the combined method will not be used...
If the user sets facet.pivot.limit.method=combined it looks like the same limit will be used for all fields? Whatever the global facet.limit is set to?
Unfortunate.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.1
>
> Attachments: distributed_pivot.patch, distributed_pivot.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894-reworked.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295389#comment-13295389 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
Erik, what revision of solr did you apply the patch to? Did you not encounter the issues I encountered?
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403341#comment-13403341 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
I have posted an enhancement to this patch as SOLR-3583. It is based on the Apr 25th version.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting
Posted by "Chris Russell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259947#comment-13259947 ]
Chris Russell commented on SOLR-2894:
-------------------------------------
Hi Dan.
I have been working with your patch, 2894, to Solr and I am having some issues with unit testing.
First of all the patch doesn't seem to apply cleanly:
crussell@WAT-CRUSSELL /cygdrive/d/matrixdev/solr_1297102/CBSolr/SolrLucene
$ patch -p0 -i SOLR-2894.patch
patching file solr/core/src/java/org/apache/solr/handler/component/EntryCountComparator.java
patching file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java
Hunk #10 FAILED at 797.
1 out of 16 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java
Hunk #2 FAILED at 106.
1 out of 2 hunks FAILED -- saving rejects to file solr/core/src/java/org/apache/solr/handler/component/PivotFacetHelper.java.rej
patching file solr/core/src/java/org/apache/solr/handler/component/PivotNamedListCountComparator.java
patching file solr/core/src/java/org/apache/solr/util/NamedListHelper.java
patching file solr/core/src/java/org/apache/solr/util/PivotListEntry.java
patching file solr/core/src/test/org/apache/solr/handler/component/DistributedFacetPivotTest.java
patching file solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java
$ patch --version
patch 2.5.8
Copyright (C) 1988 Larry Wall
Copyright (C) 2002 Free Software Foundation, Inc.
A lot of the contents of your original patch seemed to be formatting changes like where line breaks should go and how spacing should be handled.
I was able to examine the patch and manually incorporate the changes from the failed chunks.
One question, on this line (1119) of your patch, why did you choose not to initialize the map as the ones above it are? Couldn't that cause an NRE?
+ public SimpleOrderedMap<List<NamedList<Object>>> pivotFacets;
While looking at DistributedFacetPivotTest.java I noticed an error on line 42. Your "q" should probably be "*:*" instead of "*".
I've attached the patch file I came up with. I added the initialization and test correction I mentioned.
When I ran the solr/lucene unit tests after I patched, there are some unit test failures like this one:
[junit] Testsuite: org.apache.solr.TestDistributedGrouping
[junit] Testcase: testDistribSearch(org.apache.solr.TestDistributedGrouping): FAILED
[junit] .facet_counts.size()==5,4skipped=0,0
[junit] junit.framework.AssertionFailedError: .facet_counts.size()==5,4skipped=0,0
[junit] at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:656)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:383)
[junit] at org.apache.solr.TestDistributedGrouping.doTest(TestDistributedGrouping.java:51)
[junit] at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:671)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:736)
[junit] at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:632)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit] at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:531)
[junit] at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(LuceneTestCase.java:593)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:165)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:57)
[junit] at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:20)
[junit] at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:22)
[junit]
[junit]
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 2.685 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] 2520 T1 oas.BaseDistributedSearchTestCase.compareResponses SEVERE Mismatched responses:
[junit] {responseHeader={status=0,QTime=19},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={},facet_pivot={}}}
[junit] {responseHeader={status=0,QTime=18},grouped={a_si={matches=0,groups=[]}},facet_counts={facet_queries={},facet_fields={a_t={}},facet_dates={},facet_ranges={}}}
[junit] NOTE: reproduce with: ant test -Dtestcase=TestDistributedGrouping -Dtestmethod=testDistribSearch -Dtests.seed=-c7cfa73dbca93c9:-4751af558bf6f59:3e523b50870b3b1b -Dargs="-Dfile.encoding=Cp1252"
It looks like the facet_pivot is being included in the results of one query, and not the other. I'm trying to figure out why this is occurring.
Any insight would be appreciated.
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org