You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mauricio Aristizabal <ar...@gmail.com> on 2008/11/10 20:53:33 UTC
DataImportHandler - composite multivalued fields for merging db rows?
Hi, I'm setting up a Solr 1.3 server for our existing app, and have an issue
importing.
We have a marketing system and want to make campaigns more searchable. Each
campaign has a number of terms which specify either a fixed or percentage
commission (2 separate fields). In addition each term points to a tracker
that can be of type CLICK, IMPRESSION, etc.
I need each campaign to be a document. Since a campaign can have multiple
terms (with one tracker each) I've setup the fields from terms and tracker
as multivalued. However this just gets me 3 separate lists, and even though
there is a relationship between their elements, I can't be certain that
element 1 in the commissionfixed list goes with element 1 in the type list.
The ideal thing would be to encapsulate each terms with its tracker in some
sort of composite (or at least tokenized) field, and then make that
multi-valued. Is this possible somehow? I've been looking at the
documentation for fieldType without luck.
Here are my simplified schema and data-config files:
<entity name="campaign" pk="id" >
<field column="name"/>
<entity name="terms" pk="id" query="select * from terms where
campaign_id='${campaign.id}'">
<field column="commissionfixed"/>
<field column="commissionpercent"/>
<entity name="tracker" pk="id" query="select type from tracker
where id='${terms.tracker_id}'">
<field column="type"/>
</entity>
</entity>
</entity>
<field name="name" type="text" indexed="true" stored="true"/>
<field name="type" type="text" multiValued="true" indexed="true"
stored="true"/>
<field name="commissionfixed" type="float" multiValued="true"
stored="true"/>
<field name="commissionpercent" type="float" multiValued="true"
stored="true"/>
Thanks.
Re: DataImportHandler - composite multivalued fields for merging db rows?
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
Any component can use the VariableResolver . If you write a custom
transformer that too can use it
On Wed, Nov 12, 2008 at 4:40 AM, Mauricio Aristizabal <ar...@gmail.com> wrote:
> Thanks Noble, that worked beautifully.
>
> What wasn't initially apparent to me was that a transformer can use the same
> notation as in the query attribute (e.g. ${parententityname.fieldname} ) to
> access fields from parent entities, which allows them to merge data from
> multiple related rows, not just different columns.
>
> -Mauricio
>
>
>
> On Mon, Nov 10, 2008 at 8:27 PM, Noble Paul നോബിള് नोब्ळ् <
> noble.paul@gmail.com> wrote:
>
>> create a separate field commissionfixed_type and use a
>> TemplateTransformer to merge two values in the entity 'tracker'
>>
>> On Tue, Nov 11, 2008 at 1:23 AM, Mauricio Aristizabal <ar...@gmail.com>
>> wrote:
>> > Hi, I'm setting up a Solr 1.3 server for our existing app, and have an
>> issue
>> > importing.
>> >
>> > We have a marketing system and want to make campaigns more searchable.
>> Each
>> > campaign has a number of terms which specify either a fixed or percentage
>> > commission (2 separate fields). In addition each term points to a
>> tracker
>> > that can be of type CLICK, IMPRESSION, etc.
>> >
>> > I need each campaign to be a document. Since a campaign can have
>> multiple
>> > terms (with one tracker each) I've setup the fields from terms and
>> tracker
>> > as multivalued. However this just gets me 3 separate lists, and even
>> though
>> > there is a relationship between their elements, I can't be certain that
>> > element 1 in the commissionfixed list goes with element 1 in the type
>> list.
>> >
>> > The ideal thing would be to encapsulate each terms with its tracker in
>> some
>> > sort of composite (or at least tokenized) field, and then make that
>> > multi-valued. Is this possible somehow? I've been looking at the
>> > documentation for fieldType without luck.
>> >
>> > Here are my simplified schema and data-config files:
>> >
>> > <entity name="campaign" pk="id" >
>> > <field column="name"/>
>> > <entity name="terms" pk="id" query="select * from terms where
>> > campaign_id='${campaign.id}'">
>> > <field column="commissionfixed"/>
>> > <field column="commissionpercent"/>
>> > <entity name="tracker" pk="id" query="select type from
>> tracker
>> > where id='${terms.tracker_id}'">
>> > <field column="type"/>
>> > </entity>
>> > </entity>
>> > </entity>
>> >
>> > <field name="name" type="text" indexed="true" stored="true"/>
>> > <field name="type" type="text" multiValued="true" indexed="true"
>> > stored="true"/>
>> > <field name="commissionfixed" type="float" multiValued="true"
>> > stored="true"/>
>> > <field name="commissionpercent" type="float" multiValued="true"
>> > stored="true"/>
>> >
>> >
>> > Thanks.
>> >
>>
>>
>>
>> --
>> --Noble Paul
>>
>
--
--Noble Paul
Re: DataImportHandler - composite multivalued fields for merging db rows?
Posted by Mauricio Aristizabal <ar...@gmail.com>.
Thanks Noble, that worked beautifully.
What wasn't initially apparent to me was that a transformer can use the same
notation as in the query attribute (e.g. ${parententityname.fieldname} ) to
access fields from parent entities, which allows them to merge data from
multiple related rows, not just different columns.
-Mauricio
On Mon, Nov 10, 2008 at 8:27 PM, Noble Paul നോബിള് नोब्ळ् <
noble.paul@gmail.com> wrote:
> create a separate field commissionfixed_type and use a
> TemplateTransformer to merge two values in the entity 'tracker'
>
> On Tue, Nov 11, 2008 at 1:23 AM, Mauricio Aristizabal <ar...@gmail.com>
> wrote:
> > Hi, I'm setting up a Solr 1.3 server for our existing app, and have an
> issue
> > importing.
> >
> > We have a marketing system and want to make campaigns more searchable.
> Each
> > campaign has a number of terms which specify either a fixed or percentage
> > commission (2 separate fields). In addition each term points to a
> tracker
> > that can be of type CLICK, IMPRESSION, etc.
> >
> > I need each campaign to be a document. Since a campaign can have
> multiple
> > terms (with one tracker each) I've setup the fields from terms and
> tracker
> > as multivalued. However this just gets me 3 separate lists, and even
> though
> > there is a relationship between their elements, I can't be certain that
> > element 1 in the commissionfixed list goes with element 1 in the type
> list.
> >
> > The ideal thing would be to encapsulate each terms with its tracker in
> some
> > sort of composite (or at least tokenized) field, and then make that
> > multi-valued. Is this possible somehow? I've been looking at the
> > documentation for fieldType without luck.
> >
> > Here are my simplified schema and data-config files:
> >
> > <entity name="campaign" pk="id" >
> > <field column="name"/>
> > <entity name="terms" pk="id" query="select * from terms where
> > campaign_id='${campaign.id}'">
> > <field column="commissionfixed"/>
> > <field column="commissionpercent"/>
> > <entity name="tracker" pk="id" query="select type from
> tracker
> > where id='${terms.tracker_id}'">
> > <field column="type"/>
> > </entity>
> > </entity>
> > </entity>
> >
> > <field name="name" type="text" indexed="true" stored="true"/>
> > <field name="type" type="text" multiValued="true" indexed="true"
> > stored="true"/>
> > <field name="commissionfixed" type="float" multiValued="true"
> > stored="true"/>
> > <field name="commissionpercent" type="float" multiValued="true"
> > stored="true"/>
> >
> >
> > Thanks.
> >
>
>
>
> --
> --Noble Paul
>
Re: DataImportHandler - composite multivalued fields for merging db rows?
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
create a separate field commissionfixed_type and use a
TemplateTransformer to merge two values in the entity 'tracker'
On Tue, Nov 11, 2008 at 1:23 AM, Mauricio Aristizabal <ar...@gmail.com> wrote:
> Hi, I'm setting up a Solr 1.3 server for our existing app, and have an issue
> importing.
>
> We have a marketing system and want to make campaigns more searchable. Each
> campaign has a number of terms which specify either a fixed or percentage
> commission (2 separate fields). In addition each term points to a tracker
> that can be of type CLICK, IMPRESSION, etc.
>
> I need each campaign to be a document. Since a campaign can have multiple
> terms (with one tracker each) I've setup the fields from terms and tracker
> as multivalued. However this just gets me 3 separate lists, and even though
> there is a relationship between their elements, I can't be certain that
> element 1 in the commissionfixed list goes with element 1 in the type list.
>
> The ideal thing would be to encapsulate each terms with its tracker in some
> sort of composite (or at least tokenized) field, and then make that
> multi-valued. Is this possible somehow? I've been looking at the
> documentation for fieldType without luck.
>
> Here are my simplified schema and data-config files:
>
> <entity name="campaign" pk="id" >
> <field column="name"/>
> <entity name="terms" pk="id" query="select * from terms where
> campaign_id='${campaign.id}'">
> <field column="commissionfixed"/>
> <field column="commissionpercent"/>
> <entity name="tracker" pk="id" query="select type from tracker
> where id='${terms.tracker_id}'">
> <field column="type"/>
> </entity>
> </entity>
> </entity>
>
> <field name="name" type="text" indexed="true" stored="true"/>
> <field name="type" type="text" multiValued="true" indexed="true"
> stored="true"/>
> <field name="commissionfixed" type="float" multiValued="true"
> stored="true"/>
> <field name="commissionpercent" type="float" multiValued="true"
> stored="true"/>
>
>
> Thanks.
>
--
--Noble Paul