You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jason Toy <ja...@gmail.com> on 2011/10/06 17:27:58 UTC
what is the recommended way to store locations?
In our current system ,we have 3 fields for location, city, state, and
country. People in our system search for one of those 3 strings.
So a user can search for "San Francisco" or "California". In solr I store
those 3 fields as strings and when a search happens I search with an OR
statement across those 3 fields.
Is there a more efficient way to store this data storage wise and/or speed
wise? We don't currently plan to use any spacial features like "3 miles
near SF".
Re: what is the recommended way to store locations?
Posted by Robert Brown <ro...@intelcompute.com>.
Expanding CA to California sounds like a use for a synonyms config
file? you can then do that translation at index and query time, if
needed.
On Thu, 6 Oct 2011 12:01:33 -0400, Jason Toy <ja...@gmail.com>
wrote:
> Hi Otis,
> Thanks for the response. So just to make sure I understand clearly, so I
> would store a location field of either text or ngram fields
> of the format "San Francisco, California, United States" and use full text
> search against that so someone could search for San Francisco or California
> and get that hit?
> I've also added some code in the application level so that if someone
> searches for CA, it gets expanded to California during search time, would it
> be better to store this in the doc directly or keep it in application code?
>
> Jason
>
>
> On Thu, Oct 6, 2011 at 11:34 AM, Otis Gospodnetic <
> otis_gospodnetic@yahoo.com> wrote:
>
>> Jason,
>>
>> That sounds pretty simple and works well if you plan on allowing
>> fielded/structured search.
>> If not, you could alternatively stick all geo values in a single text field
>> and avoid dealing with multiple fields.
>>
>> You may also want to use ngram fields instead of text if you want to still
>> match that San Fransisco oops typo.
>>
>> Otis
>> ----
>> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
>> Lucene ecosystem search :: http://search-lucene.com/
>>
>>
>> ----- Original Message -----
>> > From: Jason Toy <ja...@gmail.com>
>> > To: solr-user@lucene.apache.org
>> > Cc:
>> > Sent: Thursday, October 6, 2011 11:27 AM
>> > Subject: what is the recommended way to store locations?
>> >
>> > In our current system ,we have 3 fields for location, city, state, and
>> > country. People in our system search for one of those 3 strings.
>> > So a user can search for "San Francisco" or "California".
>> > In solr I store
>> > those 3 fields as strings and when a search happens I search with an OR
>> > statement across those 3 fields.
>> >
>> > Is there a more efficient way to store this data storage wise and/or
>> speed
>> > wise? We don't currently plan to use any spacial features like "3
>> > miles
>> > near SF".
>> >
>>
Re: what is the recommended way to store locations?
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Yes, sounds right.
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
----- Original Message -----
> From: Jason Toy <ja...@gmail.com>
> To: solr-user@lucene.apache.org; Otis Gospodnetic <ot...@yahoo.com>
> Cc:
> Sent: Thursday, October 6, 2011 12:01 PM
> Subject: Re: what is the recommended way to store locations?
>
> Hi Otis,
> Thanks for the response. So just to make sure I understand clearly, so I
> would store a location field of either text or ngram fields
> of the format "San Francisco, California, United States" and use full
> text
> search against that so someone could search for San Francisco or California
> and get that hit?
> I've also added some code in the application level so that if someone
> searches for CA, it gets expanded to California during search time, would it
> be better to store this in the doc directly or keep it in application code?
>
> Jason
>
>
> On Thu, Oct 6, 2011 at 11:34 AM, Otis Gospodnetic <
> otis_gospodnetic@yahoo.com> wrote:
>
>> Jason,
>>
>> That sounds pretty simple and works well if you plan on allowing
>> fielded/structured search.
>> If not, you could alternatively stick all geo values in a single text field
>> and avoid dealing with multiple fields.
>>
>> You may also want to use ngram fields instead of text if you want to still
>> match that San Fransisco oops typo.
>>
>> Otis
>> ----
>> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
>> Lucene ecosystem search :: http://search-lucene.com/
>>
>>
>> ----- Original Message -----
>> > From: Jason Toy <ja...@gmail.com>
>> > To: solr-user@lucene.apache.org
>> > Cc:
>> > Sent: Thursday, October 6, 2011 11:27 AM
>> > Subject: what is the recommended way to store locations?
>> >
>> > In our current system ,we have 3 fields for location, city, state,
> and
>> > country. People in our system search for one of those 3 strings.
>> > So a user can search for "San Francisco" or
> "California".
>> > In solr I store
>> > those 3 fields as strings and when a search happens I search with an
> OR
>> > statement across those 3 fields.
>> >
>> > Is there a more efficient way to store this data storage wise and/or
>> speed
>> > wise? We don't currently plan to use any spacial features like
> "3
>> > miles
>> > near SF".
>> >
>>
>
>
>
> --
> - sent from my mobile
> 6176064373
>
Re: what is the recommended way to store locations?
Posted by Jason Toy <ja...@gmail.com>.
Hi Otis,
Thanks for the response. So just to make sure I understand clearly, so I
would store a location field of either text or ngram fields
of the format "San Francisco, California, United States" and use full text
search against that so someone could search for San Francisco or California
and get that hit?
I've also added some code in the application level so that if someone
searches for CA, it gets expanded to California during search time, would it
be better to store this in the doc directly or keep it in application code?
Jason
On Thu, Oct 6, 2011 at 11:34 AM, Otis Gospodnetic <
otis_gospodnetic@yahoo.com> wrote:
> Jason,
>
> That sounds pretty simple and works well if you plan on allowing
> fielded/structured search.
> If not, you could alternatively stick all geo values in a single text field
> and avoid dealing with multiple fields.
>
> You may also want to use ngram fields instead of text if you want to still
> match that San Fransisco oops typo.
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
>
>
> ----- Original Message -----
> > From: Jason Toy <ja...@gmail.com>
> > To: solr-user@lucene.apache.org
> > Cc:
> > Sent: Thursday, October 6, 2011 11:27 AM
> > Subject: what is the recommended way to store locations?
> >
> > In our current system ,we have 3 fields for location, city, state, and
> > country. People in our system search for one of those 3 strings.
> > So a user can search for "San Francisco" or "California".
> > In solr I store
> > those 3 fields as strings and when a search happens I search with an OR
> > statement across those 3 fields.
> >
> > Is there a more efficient way to store this data storage wise and/or
> speed
> > wise? We don't currently plan to use any spacial features like "3
> > miles
> > near SF".
> >
>
--
- sent from my mobile
6176064373
Re: what is the recommended way to store locations?
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Jason,
That sounds pretty simple and works well if you plan on allowing fielded/structured search.
If not, you could alternatively stick all geo values in a single text field and avoid dealing with multiple fields.
You may also want to use ngram fields instead of text if you want to still match that San Fransisco oops typo.
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
----- Original Message -----
> From: Jason Toy <ja...@gmail.com>
> To: solr-user@lucene.apache.org
> Cc:
> Sent: Thursday, October 6, 2011 11:27 AM
> Subject: what is the recommended way to store locations?
>
> In our current system ,we have 3 fields for location, city, state, and
> country. People in our system search for one of those 3 strings.
> So a user can search for "San Francisco" or "California".
> In solr I store
> those 3 fields as strings and when a search happens I search with an OR
> statement across those 3 fields.
>
> Is there a more efficient way to store this data storage wise and/or speed
> wise? We don't currently plan to use any spacial features like "3
> miles
> near SF".
>
RE: what is the recommended way to store locations?
Posted by Rob Brown <ro...@intelcompute.com>.
Just to throw this out there, we use UK postal data for locations, 9m
records and do location lookups/searches by grouping on the first part
of the post code.
Works really well, but apologies for going even more off-topic :)
-----Original Message-----
From: Jaeger, Jay - DOT <Ja...@dot.wi.gov>
Reply-to: solr-user@lucene.apache.org
To: solr-user@lucene.apache.org <so...@lucene.apache.org>
Subject: RE: what is the recommended way to store locations?
Date: Thu, 6 Oct 2011 16:13:06 -0500
We do much the same (along with name, address, postal code, etc.).
However, we use AND when we search: the more data someone can provide, the fewer and more applicable their search results.
JRJ
-----Original Message-----
From: Jason Toy [mailto:jasontoy@gmail.com]
Sent: Thursday, October 06, 2011 10:28 AM
To: solr-user@lucene.apache.org
Subject: what is the recommended way to store locations?
In our current system ,we have 3 fields for location, city, state, and
country. People in our system search for one of those 3 strings.
So a user can search for "San Francisco" or "California". In solr I store
those 3 fields as strings and when a search happens I search with an OR
statement across those 3 fields.
Is there a more efficient way to store this data storage wise and/or speed
wise? We don't currently plan to use any spacial features like "3 miles
near SF".
RE: what is the recommended way to store locations?
Posted by "Jaeger, Jay - DOT" <Ja...@dot.wi.gov>.
We do much the same (along with name, address, postal code, etc.).
However, we use AND when we search: the more data someone can provide, the fewer and more applicable their search results.
JRJ
-----Original Message-----
From: Jason Toy [mailto:jasontoy@gmail.com]
Sent: Thursday, October 06, 2011 10:28 AM
To: solr-user@lucene.apache.org
Subject: what is the recommended way to store locations?
In our current system ,we have 3 fields for location, city, state, and
country. People in our system search for one of those 3 strings.
So a user can search for "San Francisco" or "California". In solr I store
those 3 fields as strings and when a search happens I search with an OR
statement across those 3 fields.
Is there a more efficient way to store this data storage wise and/or speed
wise? We don't currently plan to use any spacial features like "3 miles
near SF".
Re: what is the recommended way to store locations?
Posted by Bill Bell <bi...@gmail.com>.
You could client-side Google Geocoding on why the user typed in.
Then get the lat,long returned from Google, and do a geo spatial search
On 10/6/11 9:27 AM, "Jason Toy" <ja...@gmail.com> wrote:
>In our current system ,we have 3 fields for location, city, state, and
>country. People in our system search for one of those 3 strings.
>So a user can search for "San Francisco" or "California". In solr I store
>those 3 fields as strings and when a search happens I search with an OR
>statement across those 3 fields.
>
>Is there a more efficient way to store this data storage wise and/or speed
>wise? We don't currently plan to use any spacial features like "3 miles
>near SF".