You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Kumaravel Kandasami <ku...@gmail.com> on 2010/01/15 23:03:01 UTC
Design Question - Dynamic Field Names (*)
Need to your suggestion in best designing the following requirement.
- We have two indexes.
Index 1: "name_index",
Fields:
"id" - indexed, not stored
"field_name" - indexed, stored.
Index 2: "trans_index',
Fields(Dynamic Schema):
"id" - indexed, not stored
"*" - indexed, stored.
(Dynamic field names of the trans_index is the same as the "field_name" from
the name_index.)
- Requirement:
User would select the field he wants to query from the "name_index".
Once he selects the one of the values from the 'field_name' (from the
name_index), he queries the trans_index using the field_name.
- Issue:
When indexing the name_index field:"field_name" we are using the analyzer
that would lowercase, strip spaces etc.
Example: "First Name", "firstName" values are all stored and indexed as
'firstname'.
However, when we store field names in the trans_index we would be storing as
it is ... without analyzing.
So User queries like 'firstname:a*' might not match.
- Possible Solution:
We are planning to have an custom analyzer that we would use while indexing
(configured in the schema.xml) file. As well the crawler program would use
the
same analyzer to create field names.
Is there any better design solutions ?
Kumar _/|\_
www.saisk.com
kumar@saisk.com
"making a profound difference with knowledge and creativity..."
Re: Design Question - Dynamic Field Names (*)
Posted by Chris Hostetter <ho...@fucit.org>.
: - We are indexing CSV files and generating field names dynamically from the
: "header" line.
: User should be able to *list all the possible header names* (i.e. dynamic
: field names), and filter results based on some of the field names.
: - Also, list* all possible values* associated to for a given field name.
#1) the LukeRequestHandler can list all field names in the index.
#2) the TermsComponent or Faceting can list all *indexed* values in a
given field ... which one you'll want to use depends largely on what you
want to do with that list.
-Hoss
Re: Design Question - Dynamic Field Names (*)
Posted by Kumaravel Kandasami <ku...@gmail.com>.
First Thanks for the response.
Yes, mostly likely we want to optimize to one index file. I think it is
possible, coming from the RDBMS world we might be over complicating the
solution.
*Requirement:*
- We are indexing CSV files and generating field names dynamically from the
"header" line.
User should be able to *list all the possible header names* (i.e. dynamic
field names), and filter results based on some of the field names.
- Also, list* all possible values* associated to for a given field name.
Kumar _/|\_
www.saisk.com
kumar@saisk.com
"making a profound difference with knowledge and creativity..."
On Tue, Jan 19, 2010 at 1:33 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> On Sat, Jan 16, 2010 at 3:33 AM, Kumaravel Kandasami <
> kumaravel.kandasami@gmail.com> wrote:
>
> > Need to your suggestion in best designing the following requirement.
> >
> > - We have two indexes.
> > Index 1: "name_index",
> > Fields:
> > "id" - indexed, not stored
> > "field_name" - indexed, stored.
> >
> > Index 2: "trans_index',
> > Fields(Dynamic Schema):
> > "id" - indexed, not stored
> > "*" - indexed, stored.
> >
> > (Dynamic field names of the trans_index is the same as the "field_name"
> > from
> > the name_index.)
> >
> > - Requirement:
> >
> > User would select the field he wants to query from the "name_index".
> > Once he selects the one of the values from the 'field_name' (from the
> > name_index), he queries the trans_index using the field_name.
> >
> >
> > - Issue:
> >
> > When indexing the name_index field:"field_name" we are using the analyzer
> > that would lowercase, strip spaces etc.
> > Example: "First Name", "firstName" values are all stored and indexed as
> > 'firstname'.
> >
> > However, when we store field names in the trans_index we would be storing
> > as
> > it is ... without analyzing.
> > So User queries like 'firstname:a*' might not match.
> >
> > - Possible Solution:
> >
> > We are planning to have an custom analyzer that we would use while
> indexing
> > (configured in the schema.xml) file. As well the crawler program would
> use
> > the
> > same analyzer to create field names.
> >
> > Is there any better design solutions ?
> >
> >
> Your scenario sounds quite strange and it is still not clear why you are
> doing all this. Perhaps the solution doesn't even require two indexes? Can
> you describe the actual problem so that we can be of more help?
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: Design Question - Dynamic Field Names (*)
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Sat, Jan 16, 2010 at 3:33 AM, Kumaravel Kandasami <
kumaravel.kandasami@gmail.com> wrote:
> Need to your suggestion in best designing the following requirement.
>
> - We have two indexes.
> Index 1: "name_index",
> Fields:
> "id" - indexed, not stored
> "field_name" - indexed, stored.
>
> Index 2: "trans_index',
> Fields(Dynamic Schema):
> "id" - indexed, not stored
> "*" - indexed, stored.
>
> (Dynamic field names of the trans_index is the same as the "field_name"
> from
> the name_index.)
>
> - Requirement:
>
> User would select the field he wants to query from the "name_index".
> Once he selects the one of the values from the 'field_name' (from the
> name_index), he queries the trans_index using the field_name.
>
>
> - Issue:
>
> When indexing the name_index field:"field_name" we are using the analyzer
> that would lowercase, strip spaces etc.
> Example: "First Name", "firstName" values are all stored and indexed as
> 'firstname'.
>
> However, when we store field names in the trans_index we would be storing
> as
> it is ... without analyzing.
> So User queries like 'firstname:a*' might not match.
>
> - Possible Solution:
>
> We are planning to have an custom analyzer that we would use while indexing
> (configured in the schema.xml) file. As well the crawler program would use
> the
> same analyzer to create field names.
>
> Is there any better design solutions ?
>
>
Your scenario sounds quite strange and it is still not clear why you are
doing all this. Perhaps the solution doesn't even require two indexes? Can
you describe the actual problem so that we can be of more help?
--
Regards,
Shalin Shekhar Mangar.