Indexing & search list of Key/Value pairs

[amid <am...@donanza.com>]

Hi,

I'm using solr and trying to index a list of key/value pairs, the key
contains a string with a skill and the value is the years of experience
(i.e. someone with 5 years of php and 2 years of ruby).

I want to be able to create a query which return all document with a
specific skill and range of years,
i.e. php with 2-4 years

Is there a good way to index the list of skills pair so we can query it
easily?

Thanks,
Ami



--
View this message in context: http://lucene.472066.n3.nabble.com/Indexing-search-list-of-Key-Value-pairs-tp4156206.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Indexing & search list of Key/Value pairs

[amid <am...@donanza.com>]

Hi,

Found the best way to do it (for the ones which will read it in the future).
Starting from Solr 4.8 nested documents can be used so for the document we
can created child document with the key & value as fields for each ley,
using block join queries will close to loop and give the ability to search
document with a nested document matching the query.

Hope this will help.
Thanks,
Ami



--
View this message in context: http://lucene.472066.n3.nabble.com/Indexing-search-list-of-Key-Value-pairs-tp4156206p4212357.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Indexing & search list of Key/Value pairs

[amid <am...@donanza.com>]

Hi Jack,

This can really improve the solution, still it's less robust than an option
to search a "pair" (for example if the skills are indexed with an index key,
a query like "(skill1:php skill1_years:[2 TO *]) OR (skill2:php
skill2_years:[2 TO *] ...).
The solution I describe is not really good as it can create allot of
subqueries and I assume can kill the performance.

Wondered if payloads can be used so the term will be the skill and the
payload will contain the amount of years. If I can create a query (or
implement a custom one) which can search by skill & payload together this
can be a very good solution.
Is this achievable? if so do you know of any demo or good starting point to
explore it?

Many thanks,
Ami




--
View this message in context: http://lucene.472066.n3.nabble.com/Indexing-search-list-of-Key-Value-pairs-tp4156206p4156241.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Indexing & search list of Key/Value pairs

[Jack Krupansky <ja...@basetechnology.com>]

You can certainly have a separate multivalued text field, like "skills" that 
can have arbitrary text values like "PHP", "Ruby, "Software Development", 
"Agile Methodology", "Agile Development", "Cat Herding", etc., that are 
analyzed, lower cased, stemmed, etc.

As far as the dynamic field names, technically they can have spaces and 
special characters and case sensitive, I would suggest that they be 
"normalized" as lower case, with underscores for special characters and 
spaces, such as:

skills:agile software_development:[10 TO *]

That would match somebody with "Agile Methodology" or "Agile Development" 
AND 10 or more years of "Software Development".

-- Jack Krupansky

-----Original Message----- 
From: amid
Sent: Monday, September 1, 2014 12:50 PM
To: solr-user@lucene.apache.org
Subject: Re: Indexing & search list of Key/Value pairs

Hi Jack,

Thanks for the fast response.
I assume that using this technique will have the following limitations:
1) Skill characters will be limited
2) Field name are not analyze and will not be able to get the full search
pack (synonym, analyzers...)

Am i right?

If so do you familiar with other techniques? (Don't have problem with
customizing implementation of parsers, scoring, etc)

Many thanks,
Ami



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexing-search-list-of-Key-Value-pairs-tp4156206p4156219.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Re: Indexing & search list of Key/Value pairs

[amid <am...@donanza.com>]

Hi Jack,

Thanks for the fast response.
I assume that using this technique will have the following limitations:
1) Skill characters will be limited 
2) Field name are not analyze and will not be able to get the full search
pack (synonym, analyzers...)

Am i right?

If so do you familiar with other techniques? (Don't have problem with
customizing implementation of parsers, scoring, etc)

Many thanks,
Ami



--
View this message in context: http://lucene.472066.n3.nabble.com/Indexing-search-list-of-Key-Value-pairs-tp4156206p4156219.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Indexing & search list of Key/Value pairs

[Jack Krupansky <ja...@basetechnology.com>]

Solr supports multivalued fields, but really only for scalar, not structured 
values. And trying to manage two or more multivalued fields in parallel is 
also problematic. Better to simply use dynamic fields, such as name the 
field "xyz_skill" and the value is the number of years. Then you can simply 
query:

php_skill:[5 TO *] AND ruby_skill:[2 TO *]

-- Jack Krupansky

-----Original Message----- 
From: amid
Sent: Monday, September 1, 2014 12:24 PM
To: solr-user@lucene.apache.org
Subject: Indexing & search list of Key/Value pairs

Hi,

I'm using solr and trying to index a list of key/value pairs, the key
contains a string with a skill and the value is the years of experience
(i.e. someone with 5 years of php and 2 years of ruby).

I want to be able to create a query which return all document with a
specific skill and range of years,
i.e. php with 2-4 years

Is there a good way to index the list of skills pair so we can query it
easily?

Thanks,
Ami



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexing-search-list-of-Key-Value-pairs-tp4156206.html
Sent from the Solr - User mailing list archive at Nabble.com.