You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Sergio García Maroto <ma...@gmail.com> on 2023/04/27 13:52:59 UTC
crossCollection on multivalue fields
Hi,
I am trying to use crossCollection join to be able to use sharding.
I am joining to collections Person and documents. Finding people with
documetns of type pdf. Just realised no people comes back
and noticed PersonID in document core is multivalue field.
<field name="PersonID" type="string" indexed="true" stored="true" docValues=
"true" multiValued="true" />
My query looks like
{!join method=crossCollection from=PersonID to=PersonID fromIndex=document
v='type:(pdf)'}
Is this feature not working on multivalue fields? Is there any other option?
Regards
Sergio Maroto
Re: crossCollection on multivalue fields
Posted by Mikhail Khludnev <mk...@apache.org>.
Hi,
FWIW I'm working on https://issues.apache.org/jira/browse/SOLR-16717 which
allows to join equally sharded collections.
The point is to distribute collocate join operation.
On Tue, May 2, 2023 at 11:27 AM Sergio García Maroto <ma...@gmail.com>
wrote:
> Ok. I am using a different join to work on sharding. This one seems like
> doesn´t allow multivalue field join.
> {!join method=crossCollection from=PersonID to=PersonID fromIndex=document
> v='type:(pdf)'}
>
> Thanks
> Sergio Maroto
>
> On Fri, 28 Apr 2023 at 17:52, Ron Haines <mi...@gmail.com> wrote:
>
> > using Solr 8.11, and 9.1.
> >
> > ok, looks like I'm doing a: {!join from=group_id_mv
> > fromIndex=primary_rollup to=group_member_id score=none}
> >
> > where 'group_id_mv' is multiValued='true'
> > and 'group_member_id' is multiValued='true'
> >
> > Are you certain the docs you need to join are on the same shard? I
> believe
> > that is a requirement.
> >
> > Ron
> >
> > On Fri, Apr 28, 2023 at 6:48 AM Sergio García Maroto <marotosg@gmail.com
> >
> > wrote:
> >
> > > Thanks both for your responses.
> > >
> > > I am on Solr 8.10.1 and I found that as far as I define any field
> > > multivalued. It doesn´t work.
> > > I tried using both multivalue with no luck.
> > >
> > > Hey Ron. You mentioned it works with multivalued fields. What Solr
> > version
> > > are you using?
> > > Can you confirm that please and maybe give a bit more info?
> > >
> > > Thanks a lot
> > > Sergio
> > >
> > >
> > >
> > > On Thu, 27 Apr 2023 at 20:29, Mikhail Khludnev <mk...@apache.org>
> wrote:
> > >
> > > > Hi. With regards to having multi-valued fromField
> > > >
> > > >
> > > >
> > >
> >
> org.apache.solr.search.join.CrossCollectionJoinQuery.CrossCollectionJoinQueryWeight#createCloudSolrStream
> > > > requests export handler with sort by fromField
> > > >
> > > > params.set(CommonParams.FL, fromField);
> > > > params.set(CommonParams.SORT, fromField + " asc");
> > > > params.set(CommonParams.QT, "/export");
> > > >
> > > > However, the export handler doesn't sort by mv field
> > > >
> > >
> >
> https://solr.apache.org/guide/8_9/exporting-result-sets.html#specifying-the-sort-criteria
> > > > So, there should be an error somewhere.
> > > >
> > > >
> > > > On Thu, Apr 27, 2023 at 8:48 PM Ron Haines <mi...@gmail.com>
> wrote:
> > > >
> > > >> For what it's worth, I have recently used the cross collection join.
> > I
> > > >> did
> > > >> make sure that the 2 fields were defined the same, as far as both
> > being
> > > >> 'multi-value'. Although, I am not certain that is a requirement.
> > > >>
> > > >> Now, the other thing I ran into is that my 'documents' collection
> was
> > > >> sharded, say across 5 shards. But, my 'person' collection was on a
> > > single
> > > >> shard. I came to realize that the only time the join was working
> was
> > > for
> > > >> the 'documents' collection documents, that were on the same shard as
> > the
> > > >> 'person' collection. My 'person' collection was quite small. So,
> to
> > > >> address this, I simply replicated my 'person' collection across all
> 5
> > > >> shards.
> > > >>
> > > >> I'm still in the process of 'certifying' if what I described is
> > > >> actually behaving the way I think it should. So far, so good.
> > > >>
> > > >> Ron Haines
> > > >>
> > > >> On Thu, Apr 27, 2023 at 9:53 AM Sergio García Maroto <
> > > marotosg@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > I am trying to use crossCollection join to be able to use
> sharding.
> > > >> > I am joining to collections Person and documents. Finding people
> > with
> > > >> > documetns of type pdf. Just realised no people comes back
> > > >> > and noticed PersonID in document core is multivalue field.
> > > >> >
> > > >> > <field name="PersonID" type="string" indexed="true" stored="true"
> > > >> > docValues=
> > > >> > "true" multiValued="true" />
> > > >> >
> > > >> > My query looks like
> > > >> > {!join method=crossCollection from=PersonID to=PersonID
> > > >> fromIndex=document
> > > >> > v='type:(pdf)'}
> > > >> >
> > > >> > Is this feature not working on multivalue fields? Is there any
> other
> > > >> > option?
> > > >> >
> > > >> > Regards
> > > >> > Sergio Maroto
> > > >> >
> > > >>
> > > >
> > > >
> > > > --
> > > > Sincerely yours
> > > > Mikhail Khludnev
> > > > https://t.me/MUST_SEARCH
> > > > A caveat: Cyrillic!
> > > >
> > >
> >
>
--
Sincerely yours
Mikhail Khludnev
https://t.me/MUST_SEARCH
A caveat: Cyrillic!
Re: crossCollection on multivalue fields
Posted by Sergio García Maroto <ma...@gmail.com>.
Ok. I am using a different join to work on sharding. This one seems like
doesn´t allow multivalue field join.
{!join method=crossCollection from=PersonID to=PersonID fromIndex=document
v='type:(pdf)'}
Thanks
Sergio Maroto
On Fri, 28 Apr 2023 at 17:52, Ron Haines <mi...@gmail.com> wrote:
> using Solr 8.11, and 9.1.
>
> ok, looks like I'm doing a: {!join from=group_id_mv
> fromIndex=primary_rollup to=group_member_id score=none}
>
> where 'group_id_mv' is multiValued='true'
> and 'group_member_id' is multiValued='true'
>
> Are you certain the docs you need to join are on the same shard? I believe
> that is a requirement.
>
> Ron
>
> On Fri, Apr 28, 2023 at 6:48 AM Sergio García Maroto <ma...@gmail.com>
> wrote:
>
> > Thanks both for your responses.
> >
> > I am on Solr 8.10.1 and I found that as far as I define any field
> > multivalued. It doesn´t work.
> > I tried using both multivalue with no luck.
> >
> > Hey Ron. You mentioned it works with multivalued fields. What Solr
> version
> > are you using?
> > Can you confirm that please and maybe give a bit more info?
> >
> > Thanks a lot
> > Sergio
> >
> >
> >
> > On Thu, 27 Apr 2023 at 20:29, Mikhail Khludnev <mk...@apache.org> wrote:
> >
> > > Hi. With regards to having multi-valued fromField
> > >
> > >
> > >
> >
> org.apache.solr.search.join.CrossCollectionJoinQuery.CrossCollectionJoinQueryWeight#createCloudSolrStream
> > > requests export handler with sort by fromField
> > >
> > > params.set(CommonParams.FL, fromField);
> > > params.set(CommonParams.SORT, fromField + " asc");
> > > params.set(CommonParams.QT, "/export");
> > >
> > > However, the export handler doesn't sort by mv field
> > >
> >
> https://solr.apache.org/guide/8_9/exporting-result-sets.html#specifying-the-sort-criteria
> > > So, there should be an error somewhere.
> > >
> > >
> > > On Thu, Apr 27, 2023 at 8:48 PM Ron Haines <mi...@gmail.com> wrote:
> > >
> > >> For what it's worth, I have recently used the cross collection join.
> I
> > >> did
> > >> make sure that the 2 fields were defined the same, as far as both
> being
> > >> 'multi-value'. Although, I am not certain that is a requirement.
> > >>
> > >> Now, the other thing I ran into is that my 'documents' collection was
> > >> sharded, say across 5 shards. But, my 'person' collection was on a
> > single
> > >> shard. I came to realize that the only time the join was working was
> > for
> > >> the 'documents' collection documents, that were on the same shard as
> the
> > >> 'person' collection. My 'person' collection was quite small. So, to
> > >> address this, I simply replicated my 'person' collection across all 5
> > >> shards.
> > >>
> > >> I'm still in the process of 'certifying' if what I described is
> > >> actually behaving the way I think it should. So far, so good.
> > >>
> > >> Ron Haines
> > >>
> > >> On Thu, Apr 27, 2023 at 9:53 AM Sergio García Maroto <
> > marotosg@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I am trying to use crossCollection join to be able to use sharding.
> > >> > I am joining to collections Person and documents. Finding people
> with
> > >> > documetns of type pdf. Just realised no people comes back
> > >> > and noticed PersonID in document core is multivalue field.
> > >> >
> > >> > <field name="PersonID" type="string" indexed="true" stored="true"
> > >> > docValues=
> > >> > "true" multiValued="true" />
> > >> >
> > >> > My query looks like
> > >> > {!join method=crossCollection from=PersonID to=PersonID
> > >> fromIndex=document
> > >> > v='type:(pdf)'}
> > >> >
> > >> > Is this feature not working on multivalue fields? Is there any other
> > >> > option?
> > >> >
> > >> > Regards
> > >> > Sergio Maroto
> > >> >
> > >>
> > >
> > >
> > > --
> > > Sincerely yours
> > > Mikhail Khludnev
> > > https://t.me/MUST_SEARCH
> > > A caveat: Cyrillic!
> > >
> >
>
Re: crossCollection on multivalue fields
Posted by Ron Haines <mi...@gmail.com>.
using Solr 8.11, and 9.1.
ok, looks like I'm doing a: {!join from=group_id_mv
fromIndex=primary_rollup to=group_member_id score=none}
where 'group_id_mv' is multiValued='true'
and 'group_member_id' is multiValued='true'
Are you certain the docs you need to join are on the same shard? I believe
that is a requirement.
Ron
On Fri, Apr 28, 2023 at 6:48 AM Sergio García Maroto <ma...@gmail.com>
wrote:
> Thanks both for your responses.
>
> I am on Solr 8.10.1 and I found that as far as I define any field
> multivalued. It doesn´t work.
> I tried using both multivalue with no luck.
>
> Hey Ron. You mentioned it works with multivalued fields. What Solr version
> are you using?
> Can you confirm that please and maybe give a bit more info?
>
> Thanks a lot
> Sergio
>
>
>
> On Thu, 27 Apr 2023 at 20:29, Mikhail Khludnev <mk...@apache.org> wrote:
>
> > Hi. With regards to having multi-valued fromField
> >
> >
> >
> org.apache.solr.search.join.CrossCollectionJoinQuery.CrossCollectionJoinQueryWeight#createCloudSolrStream
> > requests export handler with sort by fromField
> >
> > params.set(CommonParams.FL, fromField);
> > params.set(CommonParams.SORT, fromField + " asc");
> > params.set(CommonParams.QT, "/export");
> >
> > However, the export handler doesn't sort by mv field
> >
> https://solr.apache.org/guide/8_9/exporting-result-sets.html#specifying-the-sort-criteria
> > So, there should be an error somewhere.
> >
> >
> > On Thu, Apr 27, 2023 at 8:48 PM Ron Haines <mi...@gmail.com> wrote:
> >
> >> For what it's worth, I have recently used the cross collection join. I
> >> did
> >> make sure that the 2 fields were defined the same, as far as both being
> >> 'multi-value'. Although, I am not certain that is a requirement.
> >>
> >> Now, the other thing I ran into is that my 'documents' collection was
> >> sharded, say across 5 shards. But, my 'person' collection was on a
> single
> >> shard. I came to realize that the only time the join was working was
> for
> >> the 'documents' collection documents, that were on the same shard as the
> >> 'person' collection. My 'person' collection was quite small. So, to
> >> address this, I simply replicated my 'person' collection across all 5
> >> shards.
> >>
> >> I'm still in the process of 'certifying' if what I described is
> >> actually behaving the way I think it should. So far, so good.
> >>
> >> Ron Haines
> >>
> >> On Thu, Apr 27, 2023 at 9:53 AM Sergio García Maroto <
> marotosg@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I am trying to use crossCollection join to be able to use sharding.
> >> > I am joining to collections Person and documents. Finding people with
> >> > documetns of type pdf. Just realised no people comes back
> >> > and noticed PersonID in document core is multivalue field.
> >> >
> >> > <field name="PersonID" type="string" indexed="true" stored="true"
> >> > docValues=
> >> > "true" multiValued="true" />
> >> >
> >> > My query looks like
> >> > {!join method=crossCollection from=PersonID to=PersonID
> >> fromIndex=document
> >> > v='type:(pdf)'}
> >> >
> >> > Is this feature not working on multivalue fields? Is there any other
> >> > option?
> >> >
> >> > Regards
> >> > Sergio Maroto
> >> >
> >>
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> > https://t.me/MUST_SEARCH
> > A caveat: Cyrillic!
> >
>
Re: crossCollection on multivalue fields
Posted by Sergio García Maroto <ma...@gmail.com>.
Thanks both for your responses.
I am on Solr 8.10.1 and I found that as far as I define any field
multivalued. It doesn´t work.
I tried using both multivalue with no luck.
Hey Ron. You mentioned it works with multivalued fields. What Solr version
are you using?
Can you confirm that please and maybe give a bit more info?
Thanks a lot
Sergio
On Thu, 27 Apr 2023 at 20:29, Mikhail Khludnev <mk...@apache.org> wrote:
> Hi. With regards to having multi-valued fromField
>
>
> org.apache.solr.search.join.CrossCollectionJoinQuery.CrossCollectionJoinQueryWeight#createCloudSolrStream
> requests export handler with sort by fromField
>
> params.set(CommonParams.FL, fromField);
> params.set(CommonParams.SORT, fromField + " asc");
> params.set(CommonParams.QT, "/export");
>
> However, the export handler doesn't sort by mv field
> https://solr.apache.org/guide/8_9/exporting-result-sets.html#specifying-the-sort-criteria
> So, there should be an error somewhere.
>
>
> On Thu, Apr 27, 2023 at 8:48 PM Ron Haines <mi...@gmail.com> wrote:
>
>> For what it's worth, I have recently used the cross collection join. I
>> did
>> make sure that the 2 fields were defined the same, as far as both being
>> 'multi-value'. Although, I am not certain that is a requirement.
>>
>> Now, the other thing I ran into is that my 'documents' collection was
>> sharded, say across 5 shards. But, my 'person' collection was on a single
>> shard. I came to realize that the only time the join was working was for
>> the 'documents' collection documents, that were on the same shard as the
>> 'person' collection. My 'person' collection was quite small. So, to
>> address this, I simply replicated my 'person' collection across all 5
>> shards.
>>
>> I'm still in the process of 'certifying' if what I described is
>> actually behaving the way I think it should. So far, so good.
>>
>> Ron Haines
>>
>> On Thu, Apr 27, 2023 at 9:53 AM Sergio García Maroto <ma...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I am trying to use crossCollection join to be able to use sharding.
>> > I am joining to collections Person and documents. Finding people with
>> > documetns of type pdf. Just realised no people comes back
>> > and noticed PersonID in document core is multivalue field.
>> >
>> > <field name="PersonID" type="string" indexed="true" stored="true"
>> > docValues=
>> > "true" multiValued="true" />
>> >
>> > My query looks like
>> > {!join method=crossCollection from=PersonID to=PersonID
>> fromIndex=document
>> > v='type:(pdf)'}
>> >
>> > Is this feature not working on multivalue fields? Is there any other
>> > option?
>> >
>> > Regards
>> > Sergio Maroto
>> >
>>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> https://t.me/MUST_SEARCH
> A caveat: Cyrillic!
>
Re: crossCollection on multivalue fields
Posted by Mikhail Khludnev <mk...@apache.org>.
Hi. With regards to having multi-valued fromField
org.apache.solr.search.join.CrossCollectionJoinQuery.CrossCollectionJoinQueryWeight#createCloudSolrStream
requests export handler with sort by fromField
params.set(CommonParams.FL, fromField);
params.set(CommonParams.SORT, fromField + " asc");
params.set(CommonParams.QT, "/export");
However, the export handler doesn't sort by mv field
https://solr.apache.org/guide/8_9/exporting-result-sets.html#specifying-the-sort-criteria
So, there should be an error somewhere.
On Thu, Apr 27, 2023 at 8:48 PM Ron Haines <mi...@gmail.com> wrote:
> For what it's worth, I have recently used the cross collection join. I did
> make sure that the 2 fields were defined the same, as far as both being
> 'multi-value'. Although, I am not certain that is a requirement.
>
> Now, the other thing I ran into is that my 'documents' collection was
> sharded, say across 5 shards. But, my 'person' collection was on a single
> shard. I came to realize that the only time the join was working was for
> the 'documents' collection documents, that were on the same shard as the
> 'person' collection. My 'person' collection was quite small. So, to
> address this, I simply replicated my 'person' collection across all 5
> shards.
>
> I'm still in the process of 'certifying' if what I described is
> actually behaving the way I think it should. So far, so good.
>
> Ron Haines
>
> On Thu, Apr 27, 2023 at 9:53 AM Sergio García Maroto <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I am trying to use crossCollection join to be able to use sharding.
> > I am joining to collections Person and documents. Finding people with
> > documetns of type pdf. Just realised no people comes back
> > and noticed PersonID in document core is multivalue field.
> >
> > <field name="PersonID" type="string" indexed="true" stored="true"
> > docValues=
> > "true" multiValued="true" />
> >
> > My query looks like
> > {!join method=crossCollection from=PersonID to=PersonID
> fromIndex=document
> > v='type:(pdf)'}
> >
> > Is this feature not working on multivalue fields? Is there any other
> > option?
> >
> > Regards
> > Sergio Maroto
> >
>
--
Sincerely yours
Mikhail Khludnev
https://t.me/MUST_SEARCH
A caveat: Cyrillic!
Re: crossCollection on multivalue fields
Posted by Ron Haines <mi...@gmail.com>.
For what it's worth, I have recently used the cross collection join. I did
make sure that the 2 fields were defined the same, as far as both being
'multi-value'. Although, I am not certain that is a requirement.
Now, the other thing I ran into is that my 'documents' collection was
sharded, say across 5 shards. But, my 'person' collection was on a single
shard. I came to realize that the only time the join was working was for
the 'documents' collection documents, that were on the same shard as the
'person' collection. My 'person' collection was quite small. So, to
address this, I simply replicated my 'person' collection across all 5
shards.
I'm still in the process of 'certifying' if what I described is
actually behaving the way I think it should. So far, so good.
Ron Haines
On Thu, Apr 27, 2023 at 9:53 AM Sergio García Maroto <ma...@gmail.com>
wrote:
> Hi,
>
> I am trying to use crossCollection join to be able to use sharding.
> I am joining to collections Person and documents. Finding people with
> documetns of type pdf. Just realised no people comes back
> and noticed PersonID in document core is multivalue field.
>
> <field name="PersonID" type="string" indexed="true" stored="true"
> docValues=
> "true" multiValued="true" />
>
> My query looks like
> {!join method=crossCollection from=PersonID to=PersonID fromIndex=document
> v='type:(pdf)'}
>
> Is this feature not working on multivalue fields? Is there any other
> option?
>
> Regards
> Sergio Maroto
>