You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sukanta Dey <Su...@gettyimages.com> on 2013/11/22 09:15:52 UTC
RE: Need help on Joining and sorting syntax and limitations between
multiple documents in solr-4.4.0
Hi Team,
I am attaching all the required files we are using to get the VJOIN functionality along with the actual requirement statement.
Hope this would help you understand better the requirement for VJOIN functionality.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Wednesday, September 04, 2013 1:50 PM
To: 'solr-user@lucene.apache.org'
Cc: Sukanta Dey
Subject: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
In my project I am going to use Apache solr-4.4.0 version for searching. While doing that I need to join between multiple solr documents within the same core on one of the common field across the documents.
Though I successfully join the documents using solr-4.4.0 join syntax, it is returning me the expected result, but, since my next requirement is to sort the returned result on basis of the fields from the documents
Involved in join condition's "from" clause, which I was not able to get. Let me explain the problem in detail along with the files I am using ...
1) Files being used :
a. Picklist_1.xml
--------------------------
<add><doc>
<field name="describedObjectId">t1324838</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">956</field>
<field name="siteId">130712901</field>
<field name="en">Draft</field>
<field name="gr">Draoft</field>
</doc></add>
b. Picklist_2.xml
---------------------------
<add><doc>
<field name="describedObjectId">t1324837</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">87749</field>
<field name="siteId">130712901</field>
<field name="en">New</field>
<field name="gr">Neuo</field>
</doc></add>
c. AssetID_1.xml
-------------------------------
<add><doc>
<field name="def14227_picklist">t1324837</field>
<field name="describedObjectId">a180894808</field>
<field name="describedObjectType">1</field>
<field name="isMetadataComplete">true</field>
<field name="lastUpdateDate">2013-09-02T09:28:18Z</field>
<field name="ownerId">130713716</field>
<field name="siteId">130712901</field>
</doc></add>
d. AssetID_2.xml
--------------------------------
<add><doc>
<field name="def14227_picklist">t1324838</field>
<field name="describedObjectId">a171658357</field>
<field name="describedObjectType">1</field>
<field name="ownerId">130713716</field>
<field name="rGroupId">2283961</field>
<field name="rGroupId">2290309</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rRuleId">13503796</field>
<field name="rRuleId">15485964</field>
<field name="rUgpId">38052</field>
<field name="rUgpId">41133</field>
<field name="siteId">130712901</field>
</doc></add>
2) Requirement:
--------------------------------------------
i. It needs to have a join between the files using "def14227_picklist" field from AssetID_1.xml and AssetID_2.xml and "describedObjectId" field from Picklist_1.xml and Picklist_2.xml files.
ii. After joining we need to have all the fields from the files AssetID_*.xml and "en","gr" fields from Picklist_*.xml files.
iii. While joining we also sort the result based on the "en" field value.
3) I was trying with "q={!join from=inner_id to=outer_id}zzz:vvv" syntax but no luck.
Any help/suggestion would be appreciated.
Thanks,
Sukanta Dey
RE: Need help on Joining and sorting syntax and limitations between
multiple documents in solr-4.4.0
Posted by Sukanta Dey <Su...@gettyimages.com>.
Hi Colm/Yaan,
Based on the comments in the support ticket it seems vjoin has some constraints like
I. "from" and "to" keys need to be integer whereas in our case these essentially be string values
II. Join needs to be happen between multiple cores, in our case it requires to be in the same core
III. Finally, vjoin could have an adverse effect as far as the performance is concerned
Considering the above points we require an alternative solution for vjoin or else some modifications for vjoin which could suffice our requirements.
Please, let us know your thoughts/suggestions on the same.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Wednesday, November 27, 2013 7:07 PM
To: 'Colm Pruvot'; Yann Yu; 'Greg Harris'
Cc: 'solr-user@lucene.apache.org'; Souvik Mazumder; Sukanta Dey; Katherine Schuyler
Subject: RE: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
As per the latest updates in the support ticket in Lucid portal we have some concerns as below
1. The join key id's seem to have to be integers. It says they require longs, but I am having trouble with anything but an integer as the "from" and "to" key values.
--regarding the above comment, we need to have these fields as non-numeric instead of numeric value which was discussed in first call with you.
2. You have to separate this on two collections. This doesn't work as join does in the same way; it is a function returning a value. That means that documents that don't match just get a value of zero and still return. That means all four documents will return if they are all in the same collection.
--starting from the requirement discussion we are emphasizing on the fact that join needs to be performed between the documents which will reside in the same core not in different cores.
3. You need a "join" cache in your joined collection.
--could you please explain a bit more on the above like, what we need to do to have this implementation on our side and what is the utility/feature of join cache
Also, we tried the vjoin operation with the syntax given by Greg in the ticket but it is not working as per our expectation.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Tuesday, November 26, 2013 3:20 PM
To: 'Colm Pruvot'; Yann Yu; 'Greg Harris'
Cc: 'solr-user@lucene.apache.org'; Sukanta Dey; Souvik Mazumder
Subject: RE: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
Attaching the updated files as per the comments in the ticket. You can now try the VJOIN operation on the updated files.
It would be also helpful for us if you send the correct VJOIN syntax with the inputs from the updated files.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Friday, November 22, 2013 1:46 PM
To: 'Colm Pruvot'; Yann Yu; 'Greg Harris'
Cc: 'solr-user@lucene.apache.org'; Sukanta Dey
Subject: RE: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
I am attaching all the required files we are using to get the VJOIN functionality along with the actual requirement statement.
Hope this would help you understand better the requirement for VJOIN functionality.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Wednesday, September 04, 2013 1:50 PM
To: 'solr-user@lucene.apache.org'
Cc: Sukanta Dey
Subject: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
In my project I am going to use Apache solr-4.4.0 version for searching. While doing that I need to join between multiple solr documents within the same core on one of the common field across the documents.
Though I successfully join the documents using solr-4.4.0 join syntax, it is returning me the expected result, but, since my next requirement is to sort the returned result on basis of the fields from the documents
Involved in join condition's "from" clause, which I was not able to get. Let me explain the problem in detail along with the files I am using ...
1) Files being used :
a. Picklist_1.xml
--------------------------
<add><doc>
<field name="describedObjectId">t1324838</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">956</field>
<field name="siteId">130712901</field>
<field name="en">Draft</field>
<field name="gr">Draoft</field>
</doc></add>
b. Picklist_2.xml
---------------------------
<add><doc>
<field name="describedObjectId">t1324837</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">87749</field>
<field name="siteId">130712901</field>
<field name="en">New</field>
<field name="gr">Neuo</field>
</doc></add>
c. AssetID_1.xml
-------------------------------
<add><doc>
<field name="def14227_picklist">t1324837</field>
<field name="describedObjectId">a180894808</field>
<field name="describedObjectType">1</field>
<field name="isMetadataComplete">true</field>
<field name="lastUpdateDate">2013-09-02T09:28:18Z</field>
<field name="ownerId">130713716</field>
<field name="siteId">130712901</field>
</doc></add>
d. AssetID_2.xml
--------------------------------
<add><doc>
<field name="def14227_picklist">t1324838</field>
<field name="describedObjectId">a171658357</field>
<field name="describedObjectType">1</field>
<field name="ownerId">130713716</field>
<field name="rGroupId">2283961</field>
<field name="rGroupId">2290309</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rRuleId">13503796</field>
<field name="rRuleId">15485964</field>
<field name="rUgpId">38052</field>
<field name="rUgpId">41133</field>
<field name="siteId">130712901</field>
</doc></add>
2) Requirement:
--------------------------------------------
i. It needs to have a join between the files using "def14227_picklist" field from AssetID_1.xml and AssetID_2.xml and "describedObjectId" field from Picklist_1.xml and Picklist_2.xml files.
ii. After joining we need to have all the fields from the files AssetID_*.xml and "en","gr" fields from Picklist_*.xml files.
iii. While joining we also sort the result based on the "en" field value.
3) I was trying with "q={!join from=inner_id to=outer_id}zzz:vvv" syntax but no luck.
Any help/suggestion would be appreciated.
Thanks,
Sukanta Dey
RE: Need help on Joining and sorting syntax and limitations between
multiple documents in solr-4.4.0
Posted by Sukanta Dey <Su...@gettyimages.com>.
Hi Team,
As per the latest updates in the support ticket in Lucid portal we have some concerns as below
1. The join key id's seem to have to be integers. It says they require longs, but I am having trouble with anything but an integer as the "from" and "to" key values.
--regarding the above comment, we need to have these fields as non-numeric instead of numeric value which was discussed in first call with you.
2. You have to separate this on two collections. This doesn't work as join does in the same way; it is a function returning a value. That means that documents that don't match just get a value of zero and still return. That means all four documents will return if they are all in the same collection.
--starting from the requirement discussion we are emphasizing on the fact that join needs to be performed between the documents which will reside in the same core not in different cores.
3. You need a "join" cache in your joined collection.
--could you please explain a bit more on the above like, what we need to do to have this implementation on our side and what is the utility/feature of join cache
Also, we tried the vjoin operation with the syntax given by Greg in the ticket but it is not working as per our expectation.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Tuesday, November 26, 2013 3:20 PM
To: 'Colm Pruvot'; Yann Yu; 'Greg Harris'
Cc: 'solr-user@lucene.apache.org'; Sukanta Dey; Souvik Mazumder
Subject: RE: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
Attaching the updated files as per the comments in the ticket. You can now try the VJOIN operation on the updated files.
It would be also helpful for us if you send the correct VJOIN syntax with the inputs from the updated files.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Friday, November 22, 2013 1:46 PM
To: 'Colm Pruvot'; Yann Yu; 'Greg Harris'
Cc: 'solr-user@lucene.apache.org'; Sukanta Dey
Subject: RE: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
I am attaching all the required files we are using to get the VJOIN functionality along with the actual requirement statement.
Hope this would help you understand better the requirement for VJOIN functionality.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Wednesday, September 04, 2013 1:50 PM
To: 'solr-user@lucene.apache.org'
Cc: Sukanta Dey
Subject: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
In my project I am going to use Apache solr-4.4.0 version for searching. While doing that I need to join between multiple solr documents within the same core on one of the common field across the documents.
Though I successfully join the documents using solr-4.4.0 join syntax, it is returning me the expected result, but, since my next requirement is to sort the returned result on basis of the fields from the documents
Involved in join condition's "from" clause, which I was not able to get. Let me explain the problem in detail along with the files I am using ...
1) Files being used :
a. Picklist_1.xml
--------------------------
<add><doc>
<field name="describedObjectId">t1324838</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">956</field>
<field name="siteId">130712901</field>
<field name="en">Draft</field>
<field name="gr">Draoft</field>
</doc></add>
b. Picklist_2.xml
---------------------------
<add><doc>
<field name="describedObjectId">t1324837</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">87749</field>
<field name="siteId">130712901</field>
<field name="en">New</field>
<field name="gr">Neuo</field>
</doc></add>
c. AssetID_1.xml
-------------------------------
<add><doc>
<field name="def14227_picklist">t1324837</field>
<field name="describedObjectId">a180894808</field>
<field name="describedObjectType">1</field>
<field name="isMetadataComplete">true</field>
<field name="lastUpdateDate">2013-09-02T09:28:18Z</field>
<field name="ownerId">130713716</field>
<field name="siteId">130712901</field>
</doc></add>
d. AssetID_2.xml
--------------------------------
<add><doc>
<field name="def14227_picklist">t1324838</field>
<field name="describedObjectId">a171658357</field>
<field name="describedObjectType">1</field>
<field name="ownerId">130713716</field>
<field name="rGroupId">2283961</field>
<field name="rGroupId">2290309</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rRuleId">13503796</field>
<field name="rRuleId">15485964</field>
<field name="rUgpId">38052</field>
<field name="rUgpId">41133</field>
<field name="siteId">130712901</field>
</doc></add>
2) Requirement:
--------------------------------------------
i. It needs to have a join between the files using "def14227_picklist" field from AssetID_1.xml and AssetID_2.xml and "describedObjectId" field from Picklist_1.xml and Picklist_2.xml files.
ii. After joining we need to have all the fields from the files AssetID_*.xml and "en","gr" fields from Picklist_*.xml files.
iii. While joining we also sort the result based on the "en" field value.
3) I was trying with "q={!join from=inner_id to=outer_id}zzz:vvv" syntax but no luck.
Any help/suggestion would be appreciated.
Thanks,
Sukanta Dey
RE: Need help on Joining and sorting syntax and limitations between
multiple documents in solr-4.4.0
Posted by Sukanta Dey <Su...@gettyimages.com>.
Hi Team,
Attaching the updated files as per the comments in the ticket. You can now try the VJOIN operation on the updated files.
It would be also helpful for us if you send the correct VJOIN syntax with the inputs from the updated files.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Friday, November 22, 2013 1:46 PM
To: 'Colm Pruvot'; Yann Yu; 'Greg Harris'
Cc: 'solr-user@lucene.apache.org'; Sukanta Dey
Subject: RE: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
I am attaching all the required files we are using to get the VJOIN functionality along with the actual requirement statement.
Hope this would help you understand better the requirement for VJOIN functionality.
Thanks,
Sukanta
From: Sukanta Dey
Sent: Wednesday, September 04, 2013 1:50 PM
To: 'solr-user@lucene.apache.org'
Cc: Sukanta Dey
Subject: Need help on Joining and sorting syntax and limitations between multiple documents in solr-4.4.0
Hi Team,
In my project I am going to use Apache solr-4.4.0 version for searching. While doing that I need to join between multiple solr documents within the same core on one of the common field across the documents.
Though I successfully join the documents using solr-4.4.0 join syntax, it is returning me the expected result, but, since my next requirement is to sort the returned result on basis of the fields from the documents
Involved in join condition's "from" clause, which I was not able to get. Let me explain the problem in detail along with the files I am using ...
1) Files being used :
a. Picklist_1.xml
--------------------------
<add><doc>
<field name="describedObjectId">t1324838</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">956</field>
<field name="siteId">130712901</field>
<field name="en">Draft</field>
<field name="gr">Draoft</field>
</doc></add>
b. Picklist_2.xml
---------------------------
<add><doc>
<field name="describedObjectId">t1324837</field>
<field name="describedObjectType">7</field>
<field name="picklistItemId">87749</field>
<field name="siteId">130712901</field>
<field name="en">New</field>
<field name="gr">Neuo</field>
</doc></add>
c. AssetID_1.xml
-------------------------------
<add><doc>
<field name="def14227_picklist">t1324837</field>
<field name="describedObjectId">a180894808</field>
<field name="describedObjectType">1</field>
<field name="isMetadataComplete">true</field>
<field name="lastUpdateDate">2013-09-02T09:28:18Z</field>
<field name="ownerId">130713716</field>
<field name="siteId">130712901</field>
</doc></add>
d. AssetID_2.xml
--------------------------------
<add><doc>
<field name="def14227_picklist">t1324838</field>
<field name="describedObjectId">a171658357</field>
<field name="describedObjectType">1</field>
<field name="ownerId">130713716</field>
<field name="rGroupId">2283961</field>
<field name="rGroupId">2290309</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rGroupPermissionLevel">7</field>
<field name="rRuleId">13503796</field>
<field name="rRuleId">15485964</field>
<field name="rUgpId">38052</field>
<field name="rUgpId">41133</field>
<field name="siteId">130712901</field>
</doc></add>
2) Requirement:
--------------------------------------------
i. It needs to have a join between the files using "def14227_picklist" field from AssetID_1.xml and AssetID_2.xml and "describedObjectId" field from Picklist_1.xml and Picklist_2.xml files.
ii. After joining we need to have all the fields from the files AssetID_*.xml and "en","gr" fields from Picklist_*.xml files.
iii. While joining we also sort the result based on the "en" field value.
3) I was trying with "q={!join from=inner_id to=outer_id}zzz:vvv" syntax but no luck.
Any help/suggestion would be appreciated.
Thanks,
Sukanta Dey