[jira] [Resolved] (SOLR-8297) Allow join query over 2 sharded collections: enhance functionality and exception handling

["Mikhail Khludnev (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mikhail Khludnev resolved SOLR-8297.
------------------------------------
    Resolution: Won't Fix

stuck in negotiation. I want to start something small from scratch.

> Allow join query over 2 sharded collections: enhance functionality and exception handling
> -----------------------------------------------------------------------------------------
>
>                 Key: SOLR-8297
>                 URL: https://issues.apache.org/jira/browse/SOLR-8297
>             Project: Solr
>          Issue Type: Improvement
>          Components: SolrCloud
>    Affects Versions: 5.3
>            Reporter: Paul Blanchaert
>            Priority: Major
>         Attachments: SOLR-8297.patch, SOLR-8297_Latest.patch
>
>
> h2. Proposal
> h3. General Idea
> Approach [~shikhasomani]'s range check algorithm to the most cases
> h3. Join behavior depending on router types of joined collections
> || to\\from ||CompositeId||Implicit||
> ||CompositeId| shard range check, see table below | allow |
> ||Implicit| allow | shard to shard |
> h3. CompositeId to CompositeId join behaviour for certain number of shards
>  
> || to\\from ||single||>1||
> ||single| allow (as is) | allow (range check) |
> ||>1| allow (as is) | per shard range check |
> h3. Rules from the tables above
> * joining from/to CompositeId and Implicit is blindly allowed, it pick ups any collocated replica, because users who do that probably understand what they do.
> * when both sides are Implicit let's join shards by name. ie if request hits collectionTO_shardY_replica2 at a node, the collocated collectionFROM_shardY_replica* is expected.
> * when both sides are CompositeId
> ** from single shard to single shard - nobrainer, just needs collocated replica;
> ** from multiple shards to single shard - all "from" shards (any it's replicas) are picked for joining 
> ** from single shard to multiple shards - existing SOLR-4905 functionality
> ** from multiple to multiple - generic range check algorithm
> ### check that fromField and toField are router.keys in these collections
> ### take shard range for the current "to" collection replica (keep in mind that request is distributed across "to" collection shards)   
> ### enumerate "from" collection shrads, find their subset which covers "to" shard range (this allows to handle any number of shards at both sides)
> ### pickup collocated replicas of these "from" shard subset 
> h3. Caveat 
> this is quite sensitive to shard allocation (and/or replica placement) ie failed "from" replica cannot be collocated with the required "to" shard.  
> h2. Initial Description
> Enhancement based on SOLR-4905. New Jira issue raised as suggested by Mikhail Khludnev.
> A) exception handling:
> The exception "SolrCloud join: multiple shards not yet supported" thrown in the function findLocalReplicaForFromIndex of JoinQParserPlugin is not triggered correctly: In my use-case, I've a join on a facet.query and when my results are only found in 1 shard and the facet.query with the join is querying the last replica of the last slice, then the exception is not thrown.
> I believe it's better to verify the nr of slices when we want to verify the  "multiple shards not yet supported" exception (so exception is thrown when zkController.getClusterState().getSlices(fromIndex).size()>1).
> B) functional enhancement:
> I would expect that there is no problem to perform a cross-core join over sharded collections when the following conditions are met:
> 1) both collections are sharded with the same replicationFactor and numShards
> 2) router.field of the collections is set to the same "key-field" (collection of "fromindex" has router.field = "from" field and collection joined to has router.field = "to" field)
> The router.field setup ensures that documents with the same "key-field" are routed to the same node. 
> So the combination based on the "key-field" should always be available within the same node.
> From a user perspective, I believe these assumptions seem to be a "normal" use-case in the cross-core join in SolrCloud.
> Hope this helps



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org