[jira] [Commented] (LUCENE-3639) Add test case support for shard searching

["Robert Muir (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169662#comment-13169662 ] 

Robert Muir commented on LUCENE-3639:
-------------------------------------

starting to look good, i like testing of the distributed stats!

I think, looking at this test, that unfortunately that distributed scoring is still too difficult with lucene.
Its nice how you dont do any extra rewrites or anything like that, but I don't like this:
{noformat}
+    // TODO: nothing evicts from here!!!  Somehow, on searcher
+    // expiration on remote nodes we must evict from our
+    // local cache...?
{noformat}

There are two problems here we should separate:
1. the searcher should be able to get the stats, and ensure that they are available for scorers. this is separate from:
2. the searcher doing some caching of stats to prevent network traffic.

Currently your cache handles both 1 and 2, but i think a cache should be a cache.

Maybe we can improve the scoring api, here is where the problem is:

{noformat}
  public Weight createNormalizedWeight(Query query) throws IOException {
    query = rewrite(query);
    // right here is where you want to extractTerms and get stats (from cache or remotely)
    Weight weight = query.createWeight(this); // right here, in the weights ctor, is where its going to callback to your IS to ask for those stats
{noformat}

I don't like the fact that you need to handle this crazy state here... there has to be somethign we can do to simplify this.
                
> Add test case support for shard searching
> -----------------------------------------
>
>                 Key: LUCENE-3639
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3639
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 3.5, 4.0
>
>         Attachments: LUCENE-3639.patch, LUCENE-3639.patch
>
>
> New test case that helps stress test the APIs to support sharding....

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org