You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by neha <pn...@yahoo.com> on 2011/03/29 21:01:09 UTC
Concatenate multivalued DIH fields
I have two multivalued DIH fields "fname" and "lname". I want to concatenate
each of the fname and lname pairs to get a third multivalued DIH field
"name".
I tried this :
But the result is : [Lars L., Helle K., Thomas A., Jes] [Thomsen, Iversen,
Brinck, Olesen], instead of Lars L. Thomsen, Helle K. Iverson, Thomas A
Brinck, Jes Oleson.
Is there a way to iterate through the multivalued fields or is there
something more simple to do this.
Thanks,
Neha
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2749988.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by alexei <ac...@gmail.com>.
Thank you for your replies guys.
Personally I like the flatten="true" option.
However there is still one issue with it. After stripping the tags away,
I get "JoeSmith" instead of "Joe Smith" with a space (solr 3.1).
Did anyone run into this issue?
Cheers,
Alexei
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2967017.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by jimtronic <ji...@gmail.com>.
I solved this problem using the flatten="true" attribute.
Given this schema
<people>
<person>
<names>
<name>
<firstName>Joe</firstName>
<lastName>Smith</firstName>
</name>
</names>
</person>
</people>
<field column="attr_names" xpath="/people/person/names/name" flatten="true"
/>
attr_names is a multiValued field in my schema.xml. The flatten attribute
tells solr to take all the text from the specified node and below.
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2875435.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by neha <pn...@yahoo.com>.
I would appreciate any help on this issue, I am unable to proceed without
getting this solved.
Thanks,
Neha
-----
Thanks,
Neha
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2758485.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by neha <pn...@yahoo.com>.
I wrote this snippet but get an exception
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2753910.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by neha <pn...@yahoo.com>.
I am getting data from an xml file. If possible would you be able to guide me
with a code snippet for script transformer for doing this(I am sorry if this
very basic, I am a newbie to Solr).
Thanks,
Neha
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2753763.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by Stefan Matheis <ma...@googlemail.com>.
suggesting an alternative way .. adjust your query and use the
rdbms-concat function?
On Wed, Mar 30, 2011 at 3:22 PM, neha <pn...@yahoo.com> wrote:
> HI, when i tried to use template transformer, it concatenates the entire
> multivalued field with other, not each element of the multivalued fields.
> [Lars L., Helle K., Thomas A., Jes] [Thomsen, Iversen, Brinck, Olesen],
> instead of Lars L. Thomsen, Helle K. Iverson, Thomas A Brinck, Jes Oleson.
>
> Regarding the script transformer function, it should be able to take the two
> multivalued rows as input and send the concatenated result out. I am not
> sure this is possible with script transformer. Please could you suggest an
> alternative.
>
> Thanks,
> Neha
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2753604.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Concatenate multivalued DIH fields
Posted by neha <pn...@yahoo.com>.
HI, when i tried to use template transformer, it concatenates the entire
multivalued field with other, not each element of the multivalued fields.
[Lars L., Helle K., Thomas A., Jes] [Thomsen, Iversen, Brinck, Olesen],
instead of Lars L. Thomsen, Helle K. Iverson, Thomas A Brinck, Jes Oleson.
Regarding the script transformer function, it should be able to take the two
multivalued rows as input and send the concatenated result out. I am not
sure this is possible with script transformer. Please could you suggest an
alternative.
Thanks,
Neha
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2753604.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by Stefan Matheis <ma...@googlemail.com>.
Neha, If you just need to combine them .. w/o further logic,
http://wiki.apache.org/solr/DataImportHandler#TemplateTransformer
should be enough
The first Example for ScriptTransformer should be pretty clear, no?
The Function (which you define as ScriptTransformer) will retrieve the
current row as parameter, you could work on this row and have to
return it finally.
in between, you could modify the row .. add, modify, remove values,
the row object is a java.util.HashMap instance
Regards
Stefan
On Tue, Mar 29, 2011 at 9:50 PM, neha <pn...@yahoo.com> wrote:
> Thank you for ur reply, but is there more documentation on Script
> Transformer?? I am newbie to Solr DIH. Can I send two rows as parameters to
> the script transformer function. Also what is the syntax to call the script
> transformer in the DIH field?? The documentation is not very clear about it.
>
> Thanks,
> Neha
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2750237.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Concatenate multivalued DIH fields
Posted by neha <pn...@yahoo.com>.
Thank you for ur reply, but is there more documentation on Script
Transformer?? I am newbie to Solr DIH. Can I send two rows as parameters to
the script transformer function. Also what is the syntax to call the script
transformer in the DIH field?? The documentation is not very clear about it.
Thanks,
Neha
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2750237.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by Markus Jelsma <ma...@openindex.io>.
Haven't tried your use case but i believe DIH's ScriptTransformer can do the
trick. It seems to operate on rows so you can fetch both fields and add a
concatenated field.
http://wiki.apache.org/solr/DataImportHandler#ScriptTransformer
> I have two multivalued DIH fields "fname" and "lname". I want to
> concatenate each of the fname and lname pairs to get a third multivalued
> DIH field "name".
>
> I tried this :
>
>
>
>
> But the result is : [Lars L., Helle K., Thomas A., Jes] [Thomsen, Iversen,
> Brinck, Olesen], instead of Lars L. Thomsen, Helle K. Iverson, Thomas A
> Brinck, Jes Oleson.
>
> Is there a way to iterate through the multivalued fields or is there
> something more simple to do this.
>
> Thanks,
> Neha
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp27
> 49988p2749988.html Sent from the Solr - User mailing list archive at
> Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by Lance Norskog <go...@gmail.com>.
The XPathEntityProcessor allows you to use an external XSL transform
file. In that you can do anything you want. Another option is to use
the script transformer:
http://wiki.apache.org/solr/DataImportHandler#ScriptTransformer
On Wed, Apr 6, 2011 at 12:16 PM, alexei <ac...@gmail.com> wrote:
> Hi Everyone,
>
> I am having an identical problem with concatenating author's first and last
> names stored in an xml blob.
> Because this field is multivalued copyfield does not work.
>
> Does anyone have a solution?
>
> Regards,
> Alexei
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2786506.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Lance Norskog
goksron@gmail.com
Re: Concatenate multivalued DIH fields
Posted by alexei <ac...@gmail.com>.
Hi Everyone,
I am having an identical problem with concatenating author's first and last
names stored in an xml blob.
Because this field is multivalued copyfield does not work.
Does anyone have a solution?
Regards,
Alexei
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2786506.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by neha <pn...@yahoo.com>.
Hi, When there are multiple authors instead of concatenating, it append list
of last names after the first names.
Vincent
White
Bakken
Sjaastad
Maurice B.
Linda R.
Inger Johanne
Ottar
Thanks,
Neha
-----
Thanks,
Neha
--
View this message in context: http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2760520.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Concatenate multivalued DIH fields
Posted by Marcelo Iturbe <ma...@santiago.cl>.
Hello,
In the schema.xml file you can contenate much easier.
<field name="name" type="text" indexed="true" stored="true"
multiValued="true" />
<copyField source="fname" dest="name" />
<copyField source="lname" dest="name" />
</fields>
Hope this helps
On Tue, Mar 29, 2011 at 3:01 PM, neha <pn...@yahoo.com> wrote:
> I have two multivalued DIH fields "fname" and "lname". I want to
> concatenate
> each of the fname and lname pairs to get a third multivalued DIH field
> "name".
>
> I tried this :
>
>
>
>
> But the result is : [Lars L., Helle K., Thomas A., Jes] [Thomsen, Iversen,
> Brinck, Olesen], instead of Lars L. Thomsen, Helle K. Iverson, Thomas A
> Brinck, Jes Oleson.
>
> Is there a way to iterate through the multivalued fields or is there
> something more simple to do this.
>
> Thanks,
> Neha
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Concatenate-multivalued-DIH-fields-tp2749988p2749988.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>