You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "anurag.jain" <an...@gmail.com> on 2013/01/17 18:57:52 UTC

MultiValue

my json file look like

[ { "last_name" : "jain", "training_skill":["c", "c++", "php,java,.net"] }]

can u please suggest me how should i declare field in schema for
"trainingskill" field



please reply 

urgent





--
View this message in context: http://lucene.472066.n3.nabble.com/MultiValue-tp4034305.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: MultiValue

Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Try my suggested field definition and see if it helps with faceting. It
should. Try it on a small example or a fake schema.

But I would still recommend escalating the problem up the chain to an
architect or similar. Because I bet that data is stored in multiple places
(e.g. in the database) and you will hit a real problem later when you will
try to match a particular data/configuration set back to original sources.

Otherwise, like suggested somewhere else in the chain, you can also look at
update.chain and Request Processors. But you will have to write one
yourself for this situation.

Regards,
   Alex.

Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD book)


On Thu, Jan 17, 2013 at 2:50 PM, anurag.jain <an...@gmail.com> wrote:

> @Alexandre Rafalovitch Thanks.
>
> yeah you got my point.
>
>
> training_skill:["c", "c++", "php", "java", ".net"]
> but it is not possible for me to split "php,java,.net"  because data can
> very and data is very large. i mean i have to perform on 50000 line  data.
>
> it might come["c++,php,java",".net","c#,ruby", "python  java"] like that.
>
> so i have to perform on this list. just want to ignore [ " , ]
>

Re: MultiValue

Posted by "anurag.jain" <an...@gmail.com>.
@Alexandre Rafalovitch Thanks. 

yeah you got my point.


training_skill:["c", "c++", "php", "java", ".net"]
but it is not possible for me to split "php,java,.net"  because data can
very and data is very large. i mean i have to perform on 50000 line  data. 

it might come["c++,php,java",".net","c#,ruby", "python  java"] like that. 

so i have to perform on this list. just want to ignore [ " , ] 











--
View this message in context: http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4034339.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: MultiValue

Posted by Alexandre Rafalovitch <ar...@gmail.com>.
I think the problem here is that the list has 3-values, but the last one is
actually a set of several as well. Anurag seem to be able to split them
into separate values whether they came as individual array items or as part
of joint list. So, we have a mix of multiValue submission and desire to
split it out.

The correct solution I suspect would be to normalize everything to just be
training_skill:["c", "c++", "php", "java", ".net"] before this hits Solr.

However, since he wants this for facets and as a training exercise, one
could remember that facets values come from the tokens, not stored value.
So, it might be possible to do this:
    <field name="test" type="comaSplit" indexed="true" stored="true"
multiValued="true"/>
    <fieldType name="comaSplit" class="solr.TextField"
 positionIncrementGap="100" >
        <analyzer>
           <tokenizer class="solr.PatternTokenizerFactory" pattern="," />
        </analyzer>
    </fieldType>

I think the filter code will probably just aggregate all tokens despite the
fact that they are spread over multiple values.

Regards,
   Alex.

Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD book)


On Thu, Jan 17, 2013 at 2:33 PM, Gora Mohanty <go...@mimirtech.com> wrote:

> On 18 January 2013 00:31, anurag.jain <an...@gmail.com> wrote:
> >
> >   [ { "last_name" : "jain", "training_skill":["c", "c++",
> "php,java,.net"]
> > }
> > ]
> >
> > actually i want to tokenize in   c c++ php java .net
>
> What do you mean by "tokenize" in this case? It has
> been a while since I had occasion to use JSON input,
> and also do not remember which Solr version introduced
> this, but with a JSON array mapped to a multi-valued
> Solr field, you should get one value per entry in the array.
> http://wiki.apache.org/solr/UpdateJSON#Update_Commands
> seems to be in agreement.
>
> > so through this i can make them as facet.
> >
> >
> > but problem is in list
> > "training_skill":["c", "c++", *"php,java,.net"*]
>
> Faceting should be straightforward. Are you not
> seeing the behaviour described above? Could
> you describe the issues that you are facing in
> more detail?
>
> Regards,
> Gora
>

Re: MultiValue

Posted by Gora Mohanty <go...@mimirtech.com>.
On 18 January 2013 00:31, anurag.jain <an...@gmail.com> wrote:
>
>   [ { "last_name" : "jain", "training_skill":["c", "c++", "php,java,.net"]
> }
> ]
>
> actually i want to tokenize in   c c++ php java .net

What do you mean by "tokenize" in this case? It has
been a while since I had occasion to use JSON input,
and also do not remember which Solr version introduced
this, but with a JSON array mapped to a multi-valued
Solr field, you should get one value per entry in the array.
http://wiki.apache.org/solr/UpdateJSON#Update_Commands
seems to be in agreement.

> so through this i can make them as facet.
>
>
> but problem is in list
> "training_skill":["c", "c++", *"php,java,.net"*]

Faceting should be straightforward. Are you not
seeing the behaviour described above? Could
you describe the issues that you are facing in
more detail?

Regards,
Gora

Re: MultiValue

Posted by "anurag.jain" <an...@gmail.com>.
actually  [ { "last_name" : "jain", "training_skill":*["c", "c++",
"php,java,.net"]* }  ]   training_skill is list. and if i want to store in
string field type then it will include [ and , also. so how to avoid ? or it
will not. 


or do you have any other field type definition through which my work will be
easy. 





--
View this message in context: http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4034327.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: MultiValue

Posted by Dikchant Sahi <co...@gmail.com>.
You mean to say that the problem is with json which is being ingested.

What you are trying to achieve is that you want to split the values on the
basis of comma and index it as multiple value.

What problem you are facing in indexing json in format Solr expects. If you
don't have control over it, probably you can try playing with custom
processors.




On Fri, Jan 18, 2013 at 12:31 AM, anurag.jain <an...@gmail.com> wrote:

>   [ { "last_name" : "jain", "training_skill":["c", "c++", "php,java,.net"]
> }
> ]
>
> actually i want to tokenize in   c c++ php java .net
>
>
> so through this i can make them as facet.
>
>
> but problem is in list
> "training_skill":["c", "c++", *"php,java,.net"*]
>
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4034316.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: MultiValue

Posted by "anurag.jain" <an...@gmail.com>.
  [ { "last_name" : "jain", "training_skill":["c", "c++", "php,java,.net"] }
]

actually i want to tokenize in   c c++ php java .net


so through this i can make them as facet.


but problem is in list
"training_skill":["c", "c++", *"php,java,.net"*]






--
View this message in context: http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4034316.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: MultiValue

Posted by Dikchant Sahi <co...@gmail.com>.
you just need to make the field as multivalued.

<field name="last_name" type="string" indexed="true" stored="true" * */>
<field name="trainingskill" type="string" indexed="true" stored="true"
*multiValued="true"
*/>

type should be set based on your search requirements.

On Thu, Jan 17, 2013 at 11:27 PM, anurag.jain <an...@gmail.com> wrote:

> my json file look like
>
> [ { "last_name" : "jain", "training_skill":["c", "c++", "php,java,.net"] }]
>
> can u please suggest me how should i declare field in schema for
> "trainingskill" field
>
>
>
> please reply
>
> urgent
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/MultiValue-tp4034305.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: MultiValue

Posted by manju16832003 <ma...@gmail.com>.
Hi All,
I managed to *solve* the issue I had posted earlier with respect to
multiValued.

Here is the Query suppose to configured this way in *data-config.xml *
Description: in the below, first query has associated table images. Each
person would have many images. Here the JSON/XML would return all the images
associated with the person in block.

<document name="manju">
	  <entity name="list" dataSource="manjudb" query="select
member_id,MemberName FROM member">
		<entity name="imagelist" dataSource="manjudb" query="SELECT imagepath FROM
images WHERE member_id='${list.member_id}'">
		</entity>
	  </entity>
</document>

and *schema.xml* for the above query fields looks like

<field name="member_id" type="int" indexed="true" stored="true"
required="true"/> 
<field name="MemberName" type="string" indexed="true" stored="true"/>
<field name="imagepath" type="string" indexed="true" stored="true"
<b>multiValued="true"* />

Output:
"response": {
    "numFound": 3,
    "start": 0,
    "docs": [
      {
        "MemberName": "Vettel",
        "member_id": 1,
        "_version_": 1434904021528739800
      },
      {
        "MemberName": "Schumacher",
        "member_id": 2,
        "imagepath": [ //As you could see here that Three rows from the
table *images* returned as one JSON object.
          "c:\\\\images\\etc\\test.jpg",
          "c:\\\\images\\etc\\test211.jpg",
          "c:\\\\images\\etc\\test2343434.jpg",
          "C:\\\\manju"
        ],
        "_version_": 1434904021541322800
      },
      {
        "MemberName": "J.Button",
        "member_id": 3,
        "_version_": 1434904021543420000
      }
    ]
  }
}

Thanks






--
View this message in context: http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4062863.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: MultiValue

Posted by manju16832003 <ma...@gmail.com>.
Hi All,
I wanted to achieve something similar

[ { "last_name" : "jain", "training_skill":["c", "c++", "php,java,.net"] }] 

[{"person_id" : "111", "documents" :
["doc_1","doc_2","doc_3","doc4,doc5,doc6"]}]
Meaning one person could have more than one document.

how do we need to write the query, and configure data fields.

My current data-config.xml is
<dataConfig>
       <dataSource type="JdbcDataSource" name="ds-carlist"
driver="com.mysql.jdbc.Driver" url="jdbc:mysql://127.0.0.1/solr_db"
user="root" password="root"/>

	<document name="person">
	 <entity name="list" dataSource="ds-person" query="SELECT a.id,b.docs FROM
person as a,docs as b
WHERE a.id = b.person_id">

	 </entity>
	</document>
</dataConfig>

and data field declearion in schema.xml is
   <field name="id" type="int" indexed="true" stored="true"
required="true"/> 
   <field name="docs" type="string" indexed="true" stored="true"
multiValued="true"/>


Currently I don't see any errors and the no output. 
I'm bit confused how to configure above scenario in Solr.

Thank you



--
View this message in context: http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4062788.html
Sent from the Solr - User mailing list archive at Nabble.com.