You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Abhijit Pawar <ab...@gmail.com> on 2018/05/27 18:48:57 UTC
How to merge child documents using DataImportHandler
Hello,
I am using DataImportHandler to index data from mongoDB.
Here's how my data-source-config file looks like:
<dataConfig>
<dataSource name="mongod" type="JdbcDataSource"
driver="com.mongodb.jdbc.MongoDriver" url="mongodb://<<IP
Address>>:27017/<<DB>>"/>
<document name="products">
entityA(Root Entity) - *products*
entityB (child=true,pk=unique field) - *skus*
entityC - *attributevalues*
entityD - *attributenames*
entityE(child=true,pk=unique field) - *skupricelist*
When data is indexed separate *skupricelist* documents are created for
each attribute (since *skupricelist* is child of *skus* and under
*attributenames*).How can I merge / join the all those skupricelist
documents with all attributes in same document?
example :
Right now the documents created are as follows:
Separate document 1
{
'PRODUCT NAME':'ABC',
'SKU NAME':'ABC-1',
'Color':'Red',
'SKUPricelist':'SKUPricelistA'
}
Separate document 2
{
'PRODUCT NAME':'ABC',
'SKU':'ABC-1',
'Size':'10',
'SKUPricelist':'SKUPricelistA'
}
Separate document 3
{
'PRODUCT NAME':'ABC',
'SKU':'ABC-1',
'Type':'Leather',
'SKUPricelist':'SKUPricelistA'
}
Is there a way I can join them like this?
{
'PRODUCT NAME':'ABC',
'SKU':'ABC-1',
'Color':'Red',
'Size':'10',
'Type':'Leather',
'SKUPricelist':'SKUPricelistA'
}
Thank You.
Regards,
Abhijit
Re: [EXTERNAL] Re: How to merge child documents using
DataImportHandler
Posted by David M Giannone <Da...@gm.com>.
?
Sent via the Samsung Galaxy S® 6, an AT&T 4G LTE smartphone
-------- Original message --------
From: Mikhail Khludnev <mk...@apache.org>
Date: 5/27/18 3:23 PM (GMT-05:00)
To: solr-user <so...@lucene.apache.org>
Subject: [EXTERNAL] Re: How to merge child documents using DataImportHandler
Hello, Abhijit.
Have you tried to drop some of child=true? They usually cause slicing to
separate documents, rather than default "merge to root" mode.
On Sun, May 27, 2018 at 9:48 PM, Abhijit Pawar <ab...@gmail.com>
wrote:
>
> Hello,
>
> I am using DataImportHandler to index data from mongoDB.
>
> Here's how my data-source-config file looks like:
>
> <dataConfig>
> <dataSource name="mongod" type="JdbcDataSource"
> driver="com.mongodb.jdbc.MongoDriver" url="mongodb://<<IP
> Address>>:27017/<<DB>>"/>
> <document name="products">
> entityA(Root Entity) - *products*
> entityB (child=true,pk=unique field) - *skus*
> entityC - *attributevalues*
> entityD - *attributenames*
> entityE(child=true,pk=unique field) - *skupricelist*
>
>
> When data is indexed separate *skupricelist* documents are created for
> each attribute (since *skupricelist* is child of *skus* and under
> *attributenames*).How can I merge / join the all those skupricelist
> documents with all attributes in same document?
>
> example :
> Right now the documents created are as follows:
>
> Separate document 1
> {
> 'PRODUCT NAME':'ABC',
> 'SKU NAME':'ABC-1',
> 'Color':'Red',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Separate document 2
> {
> 'PRODUCT NAME':'ABC',
> 'SKU':'ABC-1',
> 'Size':'10',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Separate document 3
> {
> 'PRODUCT NAME':'ABC',
> 'SKU':'ABC-1',
> 'Type':'Leather',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Is there a way I can join them like this?
>
> {
> 'PRODUCT NAME':'ABC',
> 'SKU':'ABC-1',
> 'Color':'Red',
> 'Size':'10',
> 'Type':'Leather',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Thank You.
> Regards,
>
> Abhijit
>
--
Sincerely yours
Mikhail Khludnev
Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.
Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.
Re: How to merge child documents using DataImportHandler
Posted by Abhijit Pawar <ab...@gmail.com>.
Hi Mikhail,
Yes I already tried that dropping child=true for skupricelist document.
However then it does not index data from that collection at all.
I need it as I am inheriting some properties from skus collection and some
from attributevalues and attributenames collection.
Also here data from skus, attributevalues and attributenames collecitions
is already merged under same document.
However data from skupricelist data is split into separate documents for
every attribute.
Regards,
Abhijit
On Sun, May 27, 2018 at 2:24 PM Mikhail Khludnev <mk...@apache.org> wrote:
> Hello, Abhijit.
> Have you tried to drop some of child=true? They usually cause slicing to
> separate documents, rather than default "merge to root" mode.
>
> On Sun, May 27, 2018 at 9:48 PM, Abhijit Pawar <abhijit.ibizsoft@gmail.com
> >
> wrote:
>
> >
> > Hello,
> >
> > I am using DataImportHandler to index data from mongoDB.
> >
> > Here's how my data-source-config file looks like:
> >
> > <dataConfig>
> > <dataSource name="mongod" type="JdbcDataSource"
> > driver="com.mongodb.jdbc.MongoDriver" url="mongodb://<<IP
> > Address>>:27017/<<DB>>"/>
> > <document name="products">
> > entityA(Root Entity) - *products*
> > entityB (child=true,pk=unique field) - *skus*
> > entityC - *attributevalues*
> > entityD - *attributenames*
> > entityE(child=true,pk=unique field) - *skupricelist*
> >
> >
> > When data is indexed separate *skupricelist* documents are created for
> > each attribute (since *skupricelist* is child of *skus* and under
> > *attributenames*).How can I merge / join the all those skupricelist
> > documents with all attributes in same document?
> >
> > example :
> > Right now the documents created are as follows:
> >
> > Separate document 1
> > {
> > 'PRODUCT NAME':'ABC',
> > 'SKU NAME':'ABC-1',
> > 'Color':'Red',
> > 'SKUPricelist':'SKUPricelistA'
> > }
> >
> > Separate document 2
> > {
> > 'PRODUCT NAME':'ABC',
> > 'SKU':'ABC-1',
> > 'Size':'10',
> > 'SKUPricelist':'SKUPricelistA'
> > }
> >
> > Separate document 3
> > {
> > 'PRODUCT NAME':'ABC',
> > 'SKU':'ABC-1',
> > 'Type':'Leather',
> > 'SKUPricelist':'SKUPricelistA'
> > }
> >
> > Is there a way I can join them like this?
> >
> > {
> > 'PRODUCT NAME':'ABC',
> > 'SKU':'ABC-1',
> > 'Color':'Red',
> > 'Size':'10',
> > 'Type':'Leather',
> > 'SKUPricelist':'SKUPricelistA'
> > }
> >
> > Thank You.
> > Regards,
> >
> > Abhijit
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>
Re: How to merge child documents using DataImportHandler
Posted by Mikhail Khludnev <mk...@apache.org>.
Hello, Abhijit.
Have you tried to drop some of child=true? They usually cause slicing to
separate documents, rather than default "merge to root" mode.
On Sun, May 27, 2018 at 9:48 PM, Abhijit Pawar <ab...@gmail.com>
wrote:
>
> Hello,
>
> I am using DataImportHandler to index data from mongoDB.
>
> Here's how my data-source-config file looks like:
>
> <dataConfig>
> <dataSource name="mongod" type="JdbcDataSource"
> driver="com.mongodb.jdbc.MongoDriver" url="mongodb://<<IP
> Address>>:27017/<<DB>>"/>
> <document name="products">
> entityA(Root Entity) - *products*
> entityB (child=true,pk=unique field) - *skus*
> entityC - *attributevalues*
> entityD - *attributenames*
> entityE(child=true,pk=unique field) - *skupricelist*
>
>
> When data is indexed separate *skupricelist* documents are created for
> each attribute (since *skupricelist* is child of *skus* and under
> *attributenames*).How can I merge / join the all those skupricelist
> documents with all attributes in same document?
>
> example :
> Right now the documents created are as follows:
>
> Separate document 1
> {
> 'PRODUCT NAME':'ABC',
> 'SKU NAME':'ABC-1',
> 'Color':'Red',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Separate document 2
> {
> 'PRODUCT NAME':'ABC',
> 'SKU':'ABC-1',
> 'Size':'10',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Separate document 3
> {
> 'PRODUCT NAME':'ABC',
> 'SKU':'ABC-1',
> 'Type':'Leather',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Is there a way I can join them like this?
>
> {
> 'PRODUCT NAME':'ABC',
> 'SKU':'ABC-1',
> 'Color':'Red',
> 'Size':'10',
> 'Type':'Leather',
> 'SKUPricelist':'SKUPricelistA'
> }
>
> Thank You.
> Regards,
>
> Abhijit
>
--
Sincerely yours
Mikhail Khludnev