You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sheldon P <te...@gmail.com> on 2012/08/24 21:33:38 UTC
How do I represent a group of customer key/value pairs
I've just started to learn Solr and I have a question about modeling data
in the schema.xml.
I'm using SolrJ to interact with my Solr server. It's easy for me to store
key/value paris where the key is known. For example, if I have:
title="Some book title"
author="The authors name"
I can represent that data in the schema.xml file like this:
<field name="title" type="text_general" indexed="true"
stored="true"/>
<field name="author" type="text_general" indexed="true"
stored="true"/>
I also have data that is stored as a Java HashMap, where the keys are
unknown:
Map<String, String> map = new HashMap<String, String>();
map.put("some unknown key", "some unknown data");
map.put("another unknown key", "more unknown data");
I would prefer to store that data in Solr without losing its hierarchy.
For example:
<field name="map" type="maptype" indexed="true" stored="true"/>
<field name="some unknown key" type="text_general" indexed="true"
stored="true"/>
<field name="another unknown key" type="text_general" indexed="true"
stored="true"/>
</field>
Then I could search for "some unknown key", and receive "some unknown data".
Is this possible in Solr? What is the best way to store this kind of data?
Re: How do I represent a group of customer key/value pairs
Posted by Lance Norskog <go...@gmail.com>.
I do not understand exactly the data modeling problem.
PathHierarchyTokenizerFactory may be what you're looking for. You
might have to combine this with a charfilter or some token filters to
get exactly what you want. Maybe have two fields, one which only saves
the leaf words and the other that only saves the tree words?
On Sat, Aug 25, 2012 at 7:45 PM, Sheldon P <sp...@gmail.com> wrote:
> Thanks Lance. It looks like it's worth investigating. I've already
> started down the path of using a bean with "@Field(map_*)" on my
> HashMap setter. This defect tipped me off on this functionality:
> https://issues.apache.org/jira/browse/SOLR-1357
> This technique provides me with a mechanism to store the HashMap data,
> but flattens the structure. I'll play with the ideas provided on
> "http://wiki.apache.org/solr/HierarchicalFaceting". If anyone has
> some sample code (java + schema.xml) they can point me too that does
> "Hierarchical Faceting" I would very much appreciate it.
>
>
> On Sat, Aug 25, 2012 at 6:42 PM, Lance Norskog <go...@gmail.com> wrote:
>> There are more advanced ways to embed hierarchy in records. This describes them:
>>
>> http://wiki.apache.org/solr/HierarchicalFaceting
>>
>> (This is a great page, never noticed it.)
>>
>> On Fri, Aug 24, 2012 at 8:12 PM, Sheldon P <sp...@gmail.com> wrote:
>>> Thanks for the prompt reply Jack. Could you point me towards any code
>>> examples of that technique?
>>>
>>>
>>> On Fri, Aug 24, 2012 at 4:31 PM, Jack Krupansky <ja...@basetechnology.com> wrote:
>>>> The general rule in Solr is simple: denormalize your data.
>>>>
>>>> If you have some maps (or tables) and a set of keys (columns) for each map
>>>> (table), define fields with names like <map-name>_<key-name>, such as
>>>> "map1_name", "map2_name", "map1_field1", "map2_field1". Solr has dynamic
>>>> fields, so you can define "<map-name>_*" to have a desired type - if all the
>>>> keys have the same type.
>>>>
>>>> -- Jack Krupansky
>>>>
>>>> -----Original Message----- From: Sheldon P
>>>> Sent: Friday, August 24, 2012 3:33 PM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: How do I represent a group of customer key/value pairs
>>>>
>>>>
>>>> I've just started to learn Solr and I have a question about modeling data
>>>> in the schema.xml.
>>>>
>>>> I'm using SolrJ to interact with my Solr server. It's easy for me to store
>>>> key/value paris where the key is known. For example, if I have:
>>>>
>>>> title="Some book title"
>>>> author="The authors name"
>>>>
>>>>
>>>> I can represent that data in the schema.xml file like this:
>>>>
>>>> <field name="title" type="text_general" indexed="true"
>>>> stored="true"/>
>>>> <field name="author" type="text_general" indexed="true"
>>>> stored="true"/>
>>>>
>>>> I also have data that is stored as a Java HashMap, where the keys are
>>>> unknown:
>>>>
>>>> Map<String, String> map = new HashMap<String, String>();
>>>> map.put("some unknown key", "some unknown data");
>>>> map.put("another unknown key", "more unknown data");
>>>>
>>>>
>>>> I would prefer to store that data in Solr without losing its hierarchy.
>>>> For example:
>>>>
>>>> <field name="map" type="maptype" indexed="true" stored="true"/>
>>>>
>>>> <field name="some unknown key" type="text_general" indexed="true"
>>>> stored="true"/>
>>>>
>>>> <field name="another unknown key" type="text_general" indexed="true"
>>>> stored="true"/>
>>>>
>>>> </field>
>>>>
>>>>
>>>> Then I could search for "some unknown key", and receive "some unknown data".
>>>>
>>>> Is this possible in Solr? What is the best way to store this kind of data?
>>
>>
>>
>> --
>> Lance Norskog
>> goksron@gmail.com
--
Lance Norskog
goksron@gmail.com
Re: How do I represent a group of customer key/value pairs
Posted by Sheldon P <sp...@gmail.com>.
Thanks Lance. It looks like it's worth investigating. I've already
started down the path of using a bean with "@Field(map_*)" on my
HashMap setter. This defect tipped me off on this functionality:
https://issues.apache.org/jira/browse/SOLR-1357
This technique provides me with a mechanism to store the HashMap data,
but flattens the structure. I'll play with the ideas provided on
"http://wiki.apache.org/solr/HierarchicalFaceting". If anyone has
some sample code (java + schema.xml) they can point me too that does
"Hierarchical Faceting" I would very much appreciate it.
On Sat, Aug 25, 2012 at 6:42 PM, Lance Norskog <go...@gmail.com> wrote:
> There are more advanced ways to embed hierarchy in records. This describes them:
>
> http://wiki.apache.org/solr/HierarchicalFaceting
>
> (This is a great page, never noticed it.)
>
> On Fri, Aug 24, 2012 at 8:12 PM, Sheldon P <sp...@gmail.com> wrote:
>> Thanks for the prompt reply Jack. Could you point me towards any code
>> examples of that technique?
>>
>>
>> On Fri, Aug 24, 2012 at 4:31 PM, Jack Krupansky <ja...@basetechnology.com> wrote:
>>> The general rule in Solr is simple: denormalize your data.
>>>
>>> If you have some maps (or tables) and a set of keys (columns) for each map
>>> (table), define fields with names like <map-name>_<key-name>, such as
>>> "map1_name", "map2_name", "map1_field1", "map2_field1". Solr has dynamic
>>> fields, so you can define "<map-name>_*" to have a desired type - if all the
>>> keys have the same type.
>>>
>>> -- Jack Krupansky
>>>
>>> -----Original Message----- From: Sheldon P
>>> Sent: Friday, August 24, 2012 3:33 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: How do I represent a group of customer key/value pairs
>>>
>>>
>>> I've just started to learn Solr and I have a question about modeling data
>>> in the schema.xml.
>>>
>>> I'm using SolrJ to interact with my Solr server. It's easy for me to store
>>> key/value paris where the key is known. For example, if I have:
>>>
>>> title="Some book title"
>>> author="The authors name"
>>>
>>>
>>> I can represent that data in the schema.xml file like this:
>>>
>>> <field name="title" type="text_general" indexed="true"
>>> stored="true"/>
>>> <field name="author" type="text_general" indexed="true"
>>> stored="true"/>
>>>
>>> I also have data that is stored as a Java HashMap, where the keys are
>>> unknown:
>>>
>>> Map<String, String> map = new HashMap<String, String>();
>>> map.put("some unknown key", "some unknown data");
>>> map.put("another unknown key", "more unknown data");
>>>
>>>
>>> I would prefer to store that data in Solr without losing its hierarchy.
>>> For example:
>>>
>>> <field name="map" type="maptype" indexed="true" stored="true"/>
>>>
>>> <field name="some unknown key" type="text_general" indexed="true"
>>> stored="true"/>
>>>
>>> <field name="another unknown key" type="text_general" indexed="true"
>>> stored="true"/>
>>>
>>> </field>
>>>
>>>
>>> Then I could search for "some unknown key", and receive "some unknown data".
>>>
>>> Is this possible in Solr? What is the best way to store this kind of data?
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
Re: How do I represent a group of customer key/value pairs
Posted by Lance Norskog <go...@gmail.com>.
There are more advanced ways to embed hierarchy in records. This describes them:
http://wiki.apache.org/solr/HierarchicalFaceting
(This is a great page, never noticed it.)
On Fri, Aug 24, 2012 at 8:12 PM, Sheldon P <sp...@gmail.com> wrote:
> Thanks for the prompt reply Jack. Could you point me towards any code
> examples of that technique?
>
>
> On Fri, Aug 24, 2012 at 4:31 PM, Jack Krupansky <ja...@basetechnology.com> wrote:
>> The general rule in Solr is simple: denormalize your data.
>>
>> If you have some maps (or tables) and a set of keys (columns) for each map
>> (table), define fields with names like <map-name>_<key-name>, such as
>> "map1_name", "map2_name", "map1_field1", "map2_field1". Solr has dynamic
>> fields, so you can define "<map-name>_*" to have a desired type - if all the
>> keys have the same type.
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Sheldon P
>> Sent: Friday, August 24, 2012 3:33 PM
>> To: solr-user@lucene.apache.org
>> Subject: How do I represent a group of customer key/value pairs
>>
>>
>> I've just started to learn Solr and I have a question about modeling data
>> in the schema.xml.
>>
>> I'm using SolrJ to interact with my Solr server. It's easy for me to store
>> key/value paris where the key is known. For example, if I have:
>>
>> title="Some book title"
>> author="The authors name"
>>
>>
>> I can represent that data in the schema.xml file like this:
>>
>> <field name="title" type="text_general" indexed="true"
>> stored="true"/>
>> <field name="author" type="text_general" indexed="true"
>> stored="true"/>
>>
>> I also have data that is stored as a Java HashMap, where the keys are
>> unknown:
>>
>> Map<String, String> map = new HashMap<String, String>();
>> map.put("some unknown key", "some unknown data");
>> map.put("another unknown key", "more unknown data");
>>
>>
>> I would prefer to store that data in Solr without losing its hierarchy.
>> For example:
>>
>> <field name="map" type="maptype" indexed="true" stored="true"/>
>>
>> <field name="some unknown key" type="text_general" indexed="true"
>> stored="true"/>
>>
>> <field name="another unknown key" type="text_general" indexed="true"
>> stored="true"/>
>>
>> </field>
>>
>>
>> Then I could search for "some unknown key", and receive "some unknown data".
>>
>> Is this possible in Solr? What is the best way to store this kind of data?
--
Lance Norskog
goksron@gmail.com
Re: How do I represent a group of customer key/value pairs
Posted by Sheldon P <sp...@gmail.com>.
Thanks for the prompt reply Jack. Could you point me towards any code
examples of that technique?
On Fri, Aug 24, 2012 at 4:31 PM, Jack Krupansky <ja...@basetechnology.com> wrote:
> The general rule in Solr is simple: denormalize your data.
>
> If you have some maps (or tables) and a set of keys (columns) for each map
> (table), define fields with names like <map-name>_<key-name>, such as
> "map1_name", "map2_name", "map1_field1", "map2_field1". Solr has dynamic
> fields, so you can define "<map-name>_*" to have a desired type - if all the
> keys have the same type.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Sheldon P
> Sent: Friday, August 24, 2012 3:33 PM
> To: solr-user@lucene.apache.org
> Subject: How do I represent a group of customer key/value pairs
>
>
> I've just started to learn Solr and I have a question about modeling data
> in the schema.xml.
>
> I'm using SolrJ to interact with my Solr server. It's easy for me to store
> key/value paris where the key is known. For example, if I have:
>
> title="Some book title"
> author="The authors name"
>
>
> I can represent that data in the schema.xml file like this:
>
> <field name="title" type="text_general" indexed="true"
> stored="true"/>
> <field name="author" type="text_general" indexed="true"
> stored="true"/>
>
> I also have data that is stored as a Java HashMap, where the keys are
> unknown:
>
> Map<String, String> map = new HashMap<String, String>();
> map.put("some unknown key", "some unknown data");
> map.put("another unknown key", "more unknown data");
>
>
> I would prefer to store that data in Solr without losing its hierarchy.
> For example:
>
> <field name="map" type="maptype" indexed="true" stored="true"/>
>
> <field name="some unknown key" type="text_general" indexed="true"
> stored="true"/>
>
> <field name="another unknown key" type="text_general" indexed="true"
> stored="true"/>
>
> </field>
>
>
> Then I could search for "some unknown key", and receive "some unknown data".
>
> Is this possible in Solr? What is the best way to store this kind of data?
Re: How do I represent a group of customer key/value pairs
Posted by Jack Krupansky <ja...@basetechnology.com>.
The general rule in Solr is simple: denormalize your data.
If you have some maps (or tables) and a set of keys (columns) for each map
(table), define fields with names like <map-name>_<key-name>, such as
"map1_name", "map2_name", "map1_field1", "map2_field1". Solr has dynamic
fields, so you can define "<map-name>_*" to have a desired type - if all the
keys have the same type.
-- Jack Krupansky
-----Original Message-----
From: Sheldon P
Sent: Friday, August 24, 2012 3:33 PM
To: solr-user@lucene.apache.org
Subject: How do I represent a group of customer key/value pairs
I've just started to learn Solr and I have a question about modeling data
in the schema.xml.
I'm using SolrJ to interact with my Solr server. It's easy for me to store
key/value paris where the key is known. For example, if I have:
title="Some book title"
author="The authors name"
I can represent that data in the schema.xml file like this:
<field name="title" type="text_general" indexed="true"
stored="true"/>
<field name="author" type="text_general" indexed="true"
stored="true"/>
I also have data that is stored as a Java HashMap, where the keys are
unknown:
Map<String, String> map = new HashMap<String, String>();
map.put("some unknown key", "some unknown data");
map.put("another unknown key", "more unknown data");
I would prefer to store that data in Solr without losing its hierarchy.
For example:
<field name="map" type="maptype" indexed="true" stored="true"/>
<field name="some unknown key" type="text_general" indexed="true"
stored="true"/>
<field name="another unknown key" type="text_general" indexed="true"
stored="true"/>
</field>
Then I could search for "some unknown key", and receive "some unknown data".
Is this possible in Solr? What is the best way to store this kind of data?