You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Thomas Nguyen <th...@ign.com> on 2010/03/04 01:56:36 UTC
Can Solr Create New Indexes?
Is there a setting in the config I can set to have Solr create a new
Lucene index if the dataDir is empty on startup? I'd like to open our
Solr system to allow other developers here to add new cores without
having to use the Lucene API directly to create the indexes.
Re: weighted search and index
Posted by Lance Norskog <go...@gmail.com>.
Boosting by convention is "flat" at 1.0. Usually people boost with
numbers like 3 or 5 or 20.
On Wed, Mar 3, 2010 at 6:34 PM, Jianbin Dai <jd...@huawei.com> wrote:
> Hi Erick,
>
> Each doc contains some keywords that are indexed. However each keyword is
> associated with a weight to represent its importance. In my example,
> D1: fruit 0.8, apple 0.4, banana 0.2
>
> The keyword fruit is the most important keyword, which means I really really
> want it to be matched in a search result, but banana is less important (It
> would be good to be matched though).
>
> Hope that explains.
>
> Thanks.
>
> JB
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, March 03, 2010 6:23 PM
> To: solr-user@lucene.apache.org
> Subject: Re: weighted search and index
>
> Then I'm totally lost as to what you're trying to accomplish. Perhaps
> a higher-level statement of the problem would help.
>
> Because no matter how often I look at your point <2>, I don't see
> what relevance the numbers have if you're not using them to
> boost at index time. Why are they even there?
>
> Erick
>
> On Wed, Mar 3, 2010 at 8:54 PM, Jianbin Dai <jd...@huawei.com> wrote:
>
>> Thank you very much Erick!
>>
>> 1. I used boost in search, but I don't know exactly what's the best way to
>> boost, for such as Sports 0.8, golf 0.5 in my example, would it be
>> sports^0.8 AND golf^0.5 ?
>>
>>
>> 2. I cannot use boost in indexing. Because the weight of the value
> changes,
>> not the field, look at this example again,
>>
>> C1: fruit 0.8, apple 0.4, banana 0.2
>> C2: music 0.9, pop song 0.6, Britney Spears 0.4
>>
>> There is no good way to boost it during indexing.
>>
>> Thanks.
>>
>> JB
>>
>>
>> -----Original Message-----
>> From: Erick Erickson [mailto:erickerickson@gmail.com]
>> Sent: Wednesday, March 03, 2010 5:45 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: weighted search and index
>>
>> You have to provide some more details to get meaningful help.
>>
>> You say "I was trying to use boosting". How? At index time?
>> Search time? Both? Can you provide some code snippets?
>> What does your schema look like for the relevant field(s)?
>>
>> You say "but seems not working right". What does that mean? No hits?
>> Hits not ordered as you expect? Have you tried putting "&debugQuery=on" on
>> your URL and examined the return values?
>>
>> Have you looked at your index with the admin page and/or Luke to see if
>> the data in the index is as you expect?
>>
>> As far as I know, boosts are multiplicative. So boosting by a value less
>> than
>> 1 will actually decrease the ranking. But see the Lucene scoring, See:
>>
>>
> http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity.
>>
> html<http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Simila
> rity.%0Ahtml>
>>
>> And remember, that boosting will *tend* to move a hit up or down in the
>> ranking, not position it absolutely.
>>
>> HTH
>> Erick
>>
>> On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai <jd...@huawei.com> wrote:
>>
>> > Hi,
>> >
>> > I am trying to use solr for a content match application.
>> >
>> > A content is described by a set of keywords with weights associated,
> eg.,
>> >
>> > C1: fruit 0.8, apple 0.4, banana 0.2
>> > C2: music 0.9, pop song 0.6, Britney Spears 0.4
>> >
>> > Those contents would be indexed in solr.
>> > In the search, I also have a set of keywords with weights:
>> >
>> > Query: Sports 0.8, golf 0.5
>> >
>> > I am trying to find the closest matching contents for this query.
>> >
>> > My question is how to index the contents with weighted scores, and how
> to
>> > write search query. I was trying to use boosting, but seems not working
>> > right.
>> >
>> > Thanks.
>> >
>> > Jianbin
>> >
>> >
>> >
>>
>>
>
>
--
Lance Norskog
goksron@gmail.com
RE: weighted search and index
Posted by Jianbin Dai <jd...@huawei.com>.
Thanks! Will try it.
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, March 04, 2010 5:59 AM
To: solr-user@lucene.apache.org
Subject: Re: weighted search and index
OK, lights are finally dawning. I think what you want is payloads,
see:
http://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payload
s/
<http://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payloa
ds/>for
your index-time term boosting. Query time boosting is as you
indicated....
HTH
Erick
On Wed, Mar 3, 2010 at 9:34 PM, Jianbin Dai <jd...@huawei.com> wrote:
> Hi Erick,
>
> Each doc contains some keywords that are indexed. However each keyword is
> associated with a weight to represent its importance. In my example,
> D1: fruit 0.8, apple 0.4, banana 0.2
>
> The keyword fruit is the most important keyword, which means I really
> really
> want it to be matched in a search result, but banana is less important (It
> would be good to be matched though).
>
> Hope that explains.
>
> Thanks.
>
> JB
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, March 03, 2010 6:23 PM
> To: solr-user@lucene.apache.org
> Subject: Re: weighted search and index
>
> Then I'm totally lost as to what you're trying to accomplish. Perhaps
> a higher-level statement of the problem would help.
>
> Because no matter how often I look at your point <2>, I don't see
> what relevance the numbers have if you're not using them to
> boost at index time. Why are they even there?
>
> Erick
>
> On Wed, Mar 3, 2010 at 8:54 PM, Jianbin Dai <jd...@huawei.com> wrote:
>
> > Thank you very much Erick!
> >
> > 1. I used boost in search, but I don't know exactly what's the best way
> to
> > boost, for such as Sports 0.8, golf 0.5 in my example, would it be
> > sports^0.8 AND golf^0.5 ?
> >
> >
> > 2. I cannot use boost in indexing. Because the weight of the value
> changes,
> > not the field, look at this example again,
> >
> > C1: fruit 0.8, apple 0.4, banana 0.2
> > C2: music 0.9, pop song 0.6, Britney Spears 0.4
> >
> > There is no good way to boost it during indexing.
> >
> > Thanks.
> >
> > JB
> >
> >
> > -----Original Message-----
> > From: Erick Erickson [mailto:erickerickson@gmail.com]
> > Sent: Wednesday, March 03, 2010 5:45 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: weighted search and index
> >
> > You have to provide some more details to get meaningful help.
> >
> > You say "I was trying to use boosting". How? At index time?
> > Search time? Both? Can you provide some code snippets?
> > What does your schema look like for the relevant field(s)?
> >
> > You say "but seems not working right". What does that mean? No hits?
> > Hits not ordered as you expect? Have you tried putting "&debugQuery=on"
> on
> > your URL and examined the return values?
> >
> > Have you looked at your index with the admin page and/or Luke to see if
> > the data in the index is as you expect?
> >
> > As far as I know, boosts are multiplicative. So boosting by a value less
> > than
> > 1 will actually decrease the ranking. But see the Lucene scoring, See:
> >
> >
>
http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity
> .
> >
> html<
> http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Simila
> rity.%0Ahtml>
> >
> > And remember, that boosting will *tend* to move a hit up or down in the
> > ranking, not position it absolutely.
> >
> > HTH
> > Erick
> >
> > On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai <jd...@huawei.com> wrote:
> >
> > > Hi,
> > >
> > > I am trying to use solr for a content match application.
> > >
> > > A content is described by a set of keywords with weights associated,
> eg.,
> > >
> > > C1: fruit 0.8, apple 0.4, banana 0.2
> > > C2: music 0.9, pop song 0.6, Britney Spears 0.4
> > >
> > > Those contents would be indexed in solr.
> > > In the search, I also have a set of keywords with weights:
> > >
> > > Query: Sports 0.8, golf 0.5
> > >
> > > I am trying to find the closest matching contents for this query.
> > >
> > > My question is how to index the contents with weighted scores, and how
> to
> > > write search query. I was trying to use boosting, but seems not
working
> > > right.
> > >
> > > Thanks.
> > >
> > > Jianbin
> > >
> > >
> > >
> >
> >
>
>
Re: weighted search and index
Posted by Erick Erickson <er...@gmail.com>.
OK, lights are finally dawning. I think what you want is payloads,
see:
http://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payloads/
<http://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payloads/>for
your index-time term boosting. Query time boosting is as you
indicated....
HTH
Erick
On Wed, Mar 3, 2010 at 9:34 PM, Jianbin Dai <jd...@huawei.com> wrote:
> Hi Erick,
>
> Each doc contains some keywords that are indexed. However each keyword is
> associated with a weight to represent its importance. In my example,
> D1: fruit 0.8, apple 0.4, banana 0.2
>
> The keyword fruit is the most important keyword, which means I really
> really
> want it to be matched in a search result, but banana is less important (It
> would be good to be matched though).
>
> Hope that explains.
>
> Thanks.
>
> JB
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, March 03, 2010 6:23 PM
> To: solr-user@lucene.apache.org
> Subject: Re: weighted search and index
>
> Then I'm totally lost as to what you're trying to accomplish. Perhaps
> a higher-level statement of the problem would help.
>
> Because no matter how often I look at your point <2>, I don't see
> what relevance the numbers have if you're not using them to
> boost at index time. Why are they even there?
>
> Erick
>
> On Wed, Mar 3, 2010 at 8:54 PM, Jianbin Dai <jd...@huawei.com> wrote:
>
> > Thank you very much Erick!
> >
> > 1. I used boost in search, but I don't know exactly what's the best way
> to
> > boost, for such as Sports 0.8, golf 0.5 in my example, would it be
> > sports^0.8 AND golf^0.5 ?
> >
> >
> > 2. I cannot use boost in indexing. Because the weight of the value
> changes,
> > not the field, look at this example again,
> >
> > C1: fruit 0.8, apple 0.4, banana 0.2
> > C2: music 0.9, pop song 0.6, Britney Spears 0.4
> >
> > There is no good way to boost it during indexing.
> >
> > Thanks.
> >
> > JB
> >
> >
> > -----Original Message-----
> > From: Erick Erickson [mailto:erickerickson@gmail.com]
> > Sent: Wednesday, March 03, 2010 5:45 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: weighted search and index
> >
> > You have to provide some more details to get meaningful help.
> >
> > You say "I was trying to use boosting". How? At index time?
> > Search time? Both? Can you provide some code snippets?
> > What does your schema look like for the relevant field(s)?
> >
> > You say "but seems not working right". What does that mean? No hits?
> > Hits not ordered as you expect? Have you tried putting "&debugQuery=on"
> on
> > your URL and examined the return values?
> >
> > Have you looked at your index with the admin page and/or Luke to see if
> > the data in the index is as you expect?
> >
> > As far as I know, boosts are multiplicative. So boosting by a value less
> > than
> > 1 will actually decrease the ranking. But see the Lucene scoring, See:
> >
> >
> http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity
> .
> >
> html<
> http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Simila
> rity.%0Ahtml>
> >
> > And remember, that boosting will *tend* to move a hit up or down in the
> > ranking, not position it absolutely.
> >
> > HTH
> > Erick
> >
> > On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai <jd...@huawei.com> wrote:
> >
> > > Hi,
> > >
> > > I am trying to use solr for a content match application.
> > >
> > > A content is described by a set of keywords with weights associated,
> eg.,
> > >
> > > C1: fruit 0.8, apple 0.4, banana 0.2
> > > C2: music 0.9, pop song 0.6, Britney Spears 0.4
> > >
> > > Those contents would be indexed in solr.
> > > In the search, I also have a set of keywords with weights:
> > >
> > > Query: Sports 0.8, golf 0.5
> > >
> > > I am trying to find the closest matching contents for this query.
> > >
> > > My question is how to index the contents with weighted scores, and how
> to
> > > write search query. I was trying to use boosting, but seems not working
> > > right.
> > >
> > > Thanks.
> > >
> > > Jianbin
> > >
> > >
> > >
> >
> >
>
>
RE: weighted search and index
Posted by Jianbin Dai <jd...@huawei.com>.
Hi Erick,
Each doc contains some keywords that are indexed. However each keyword is
associated with a weight to represent its importance. In my example,
D1: fruit 0.8, apple 0.4, banana 0.2
The keyword fruit is the most important keyword, which means I really really
want it to be matched in a search result, but banana is less important (It
would be good to be matched though).
Hope that explains.
Thanks.
JB
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Wednesday, March 03, 2010 6:23 PM
To: solr-user@lucene.apache.org
Subject: Re: weighted search and index
Then I'm totally lost as to what you're trying to accomplish. Perhaps
a higher-level statement of the problem would help.
Because no matter how often I look at your point <2>, I don't see
what relevance the numbers have if you're not using them to
boost at index time. Why are they even there?
Erick
On Wed, Mar 3, 2010 at 8:54 PM, Jianbin Dai <jd...@huawei.com> wrote:
> Thank you very much Erick!
>
> 1. I used boost in search, but I don't know exactly what's the best way to
> boost, for such as Sports 0.8, golf 0.5 in my example, would it be
> sports^0.8 AND golf^0.5 ?
>
>
> 2. I cannot use boost in indexing. Because the weight of the value
changes,
> not the field, look at this example again,
>
> C1: fruit 0.8, apple 0.4, banana 0.2
> C2: music 0.9, pop song 0.6, Britney Spears 0.4
>
> There is no good way to boost it during indexing.
>
> Thanks.
>
> JB
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, March 03, 2010 5:45 PM
> To: solr-user@lucene.apache.org
> Subject: Re: weighted search and index
>
> You have to provide some more details to get meaningful help.
>
> You say "I was trying to use boosting". How? At index time?
> Search time? Both? Can you provide some code snippets?
> What does your schema look like for the relevant field(s)?
>
> You say "but seems not working right". What does that mean? No hits?
> Hits not ordered as you expect? Have you tried putting "&debugQuery=on" on
> your URL and examined the return values?
>
> Have you looked at your index with the admin page and/or Luke to see if
> the data in the index is as you expect?
>
> As far as I know, boosts are multiplicative. So boosting by a value less
> than
> 1 will actually decrease the ranking. But see the Lucene scoring, See:
>
>
http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity.
>
html<http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Simila
rity.%0Ahtml>
>
> And remember, that boosting will *tend* to move a hit up or down in the
> ranking, not position it absolutely.
>
> HTH
> Erick
>
> On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai <jd...@huawei.com> wrote:
>
> > Hi,
> >
> > I am trying to use solr for a content match application.
> >
> > A content is described by a set of keywords with weights associated,
eg.,
> >
> > C1: fruit 0.8, apple 0.4, banana 0.2
> > C2: music 0.9, pop song 0.6, Britney Spears 0.4
> >
> > Those contents would be indexed in solr.
> > In the search, I also have a set of keywords with weights:
> >
> > Query: Sports 0.8, golf 0.5
> >
> > I am trying to find the closest matching contents for this query.
> >
> > My question is how to index the contents with weighted scores, and how
to
> > write search query. I was trying to use boosting, but seems not working
> > right.
> >
> > Thanks.
> >
> > Jianbin
> >
> >
> >
>
>
Re: weighted search and index
Posted by Erick Erickson <er...@gmail.com>.
Then I'm totally lost as to what you're trying to accomplish. Perhaps
a higher-level statement of the problem would help.
Because no matter how often I look at your point <2>, I don't see
what relevance the numbers have if you're not using them to
boost at index time. Why are they even there?
Erick
On Wed, Mar 3, 2010 at 8:54 PM, Jianbin Dai <jd...@huawei.com> wrote:
> Thank you very much Erick!
>
> 1. I used boost in search, but I don't know exactly what's the best way to
> boost, for such as Sports 0.8, golf 0.5 in my example, would it be
> sports^0.8 AND golf^0.5 ?
>
>
> 2. I cannot use boost in indexing. Because the weight of the value changes,
> not the field, look at this example again,
>
> C1: fruit 0.8, apple 0.4, banana 0.2
> C2: music 0.9, pop song 0.6, Britney Spears 0.4
>
> There is no good way to boost it during indexing.
>
> Thanks.
>
> JB
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, March 03, 2010 5:45 PM
> To: solr-user@lucene.apache.org
> Subject: Re: weighted search and index
>
> You have to provide some more details to get meaningful help.
>
> You say "I was trying to use boosting". How? At index time?
> Search time? Both? Can you provide some code snippets?
> What does your schema look like for the relevant field(s)?
>
> You say "but seems not working right". What does that mean? No hits?
> Hits not ordered as you expect? Have you tried putting "&debugQuery=on" on
> your URL and examined the return values?
>
> Have you looked at your index with the admin page and/or Luke to see if
> the data in the index is as you expect?
>
> As far as I know, boosts are multiplicative. So boosting by a value less
> than
> 1 will actually decrease the ranking. But see the Lucene scoring, See:
>
> http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity.
> html<http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity.%0Ahtml>
>
> And remember, that boosting will *tend* to move a hit up or down in the
> ranking, not position it absolutely.
>
> HTH
> Erick
>
> On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai <jd...@huawei.com> wrote:
>
> > Hi,
> >
> > I am trying to use solr for a content match application.
> >
> > A content is described by a set of keywords with weights associated, eg.,
> >
> > C1: fruit 0.8, apple 0.4, banana 0.2
> > C2: music 0.9, pop song 0.6, Britney Spears 0.4
> >
> > Those contents would be indexed in solr.
> > In the search, I also have a set of keywords with weights:
> >
> > Query: Sports 0.8, golf 0.5
> >
> > I am trying to find the closest matching contents for this query.
> >
> > My question is how to index the contents with weighted scores, and how to
> > write search query. I was trying to use boosting, but seems not working
> > right.
> >
> > Thanks.
> >
> > Jianbin
> >
> >
> >
>
>
RE: weighted search and index
Posted by Jianbin Dai <jd...@huawei.com>.
Thank you very much Erick!
1. I used boost in search, but I don't know exactly what's the best way to
boost, for such as Sports 0.8, golf 0.5 in my example, would it be
sports^0.8 AND golf^0.5 ?
2. I cannot use boost in indexing. Because the weight of the value changes,
not the field, look at this example again,
C1: fruit 0.8, apple 0.4, banana 0.2
C2: music 0.9, pop song 0.6, Britney Spears 0.4
There is no good way to boost it during indexing.
Thanks.
JB
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Wednesday, March 03, 2010 5:45 PM
To: solr-user@lucene.apache.org
Subject: Re: weighted search and index
You have to provide some more details to get meaningful help.
You say "I was trying to use boosting". How? At index time?
Search time? Both? Can you provide some code snippets?
What does your schema look like for the relevant field(s)?
You say "but seems not working right". What does that mean? No hits?
Hits not ordered as you expect? Have you tried putting "&debugQuery=on" on
your URL and examined the return values?
Have you looked at your index with the admin page and/or Luke to see if
the data in the index is as you expect?
As far as I know, boosts are multiplicative. So boosting by a value less
than
1 will actually decrease the ranking. But see the Lucene scoring, See:
http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity.
html
And remember, that boosting will *tend* to move a hit up or down in the
ranking, not position it absolutely.
HTH
Erick
On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai <jd...@huawei.com> wrote:
> Hi,
>
> I am trying to use solr for a content match application.
>
> A content is described by a set of keywords with weights associated, eg.,
>
> C1: fruit 0.8, apple 0.4, banana 0.2
> C2: music 0.9, pop song 0.6, Britney Spears 0.4
>
> Those contents would be indexed in solr.
> In the search, I also have a set of keywords with weights:
>
> Query: Sports 0.8, golf 0.5
>
> I am trying to find the closest matching contents for this query.
>
> My question is how to index the contents with weighted scores, and how to
> write search query. I was trying to use boosting, but seems not working
> right.
>
> Thanks.
>
> Jianbin
>
>
>
Re: weighted search and index
Posted by Erick Erickson <er...@gmail.com>.
You have to provide some more details to get meaningful help.
You say "I was trying to use boosting". How? At index time?
Search time? Both? Can you provide some code snippets?
What does your schema look like for the relevant field(s)?
You say "but seems not working right". What does that mean? No hits?
Hits not ordered as you expect? Have you tried putting "&debugQuery=on" on
your URL and examined the return values?
Have you looked at your index with the admin page and/or Luke to see if
the data in the index is as you expect?
As far as I know, boosts are multiplicative. So boosting by a value less
than
1 will actually decrease the ranking. But see the Lucene scoring, See:
http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity.html
And remember, that boosting will *tend* to move a hit up or down in the
ranking, not position it absolutely.
HTH
Erick
On Wed, Mar 3, 2010 at 8:13 PM, Jianbin Dai <jd...@huawei.com> wrote:
> Hi,
>
> I am trying to use solr for a content match application.
>
> A content is described by a set of keywords with weights associated, eg.,
>
> C1: fruit 0.8, apple 0.4, banana 0.2
> C2: music 0.9, pop song 0.6, Britney Spears 0.4
>
> Those contents would be indexed in solr.
> In the search, I also have a set of keywords with weights:
>
> Query: Sports 0.8, golf 0.5
>
> I am trying to find the closest matching contents for this query.
>
> My question is how to index the contents with weighted scores, and how to
> write search query. I was trying to use boosting, but seems not working
> right.
>
> Thanks.
>
> Jianbin
>
>
>
RE: Can Solr Create New Indexes?
Posted by Thomas Nguyen <th...@ign.com>.
Ah that's the problem. Not sure why it didn't come to mind to follow
the call stack. Thanks for your help!
-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com]
Sent: Wednesday, March 03, 2010 5:20 PM
To: solr-user@lucene.apache.org
Subject: Re: Can Solr Create New Indexes?
I'm guessing the index folder itself already exists?
The data dir can be there, but the index dir itself must not be - that's
how it knows to create a new one.
Otherwise it thinks the empty dir is the index and cant find the files
it expects.
On 03/03/2010 08:15 PM, Thomas Nguyen wrote:
> Hmm I've tried starting Solr with no Lucene index in the dataDir.
> Here's the Exception I receive when starting Solr and when attempting
to
> add a document to the core:
>
>
> 2010-03-03 16:44:06,479 [main ] ERROR
> org.apache.solr.core.CoreContainer -
> java.lang.RuntimeException: java.io.FileNotFoundException: no
segments*
> file found in
>
org.apache.lucene.store.SimpleFSDirectory@C:\ign\test-solr\objectIndex\i
> ndex: files:
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1068)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:579)
> at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:428)
> at
> org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
> at
>
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.
> java:117)
> at
>
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:
> 83)
> at
> org.mortbay.jetty.servlet.FilterHolder.start(FilterHolder.java:71)
> at
>
org.mortbay.jetty.servlet.WebApplicationHandler.initializeServlets(WebAp
> plicationHandler.java:310)
> at
>
org.mortbay.jetty.servlet.WebApplicationContext.doStart(WebApplicationCo
> ntext.java:509)
> at
>
org.mortbay.jetty.plus.PlusWebAppContext.doStart(PlusWebAppContext.java:
> 149)
> at org.mortbay.util.Container.start(Container.java:72)
> at org.mortbay.http.HttpServer.doStart(HttpServer.java:708)
> at org.mortbay.jetty.plus.Server.doStart(Server.java:153)
> at org.mortbay.util.Container.start(Container.java:72)
> at org.mortbay.jetty.plus.Server.main(Server.java:202)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
> a:39)
> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
> Impl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.mortbay.start.Main.invokeMain(Main.java:151)
> at org.mortbay.start.Main.start(Main.java:476)
> at org.mortbay.start.Main.main(Main.java:94)
> Caused by: java.io.FileNotFoundException: no segments* file found in
>
org.apache.lucene.store.SimpleFSDirectory@C:\ign\test-solr\objectIndex\i
> ndex: files:
> at
>
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.j
> ava:655)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
> at
> org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
> at
> org.apache.lucene.index.IndexReader.open(IndexReader.java:403)
> at
>
org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexR
> eaderFactory.java:38)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1057)
> ... 21 more
>
> Before this point I've been using existing Lucene indexes (created by
> the Lucene API) with Solr without a problem.
>
>
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Wednesday, March 03, 2010 5:00 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Can Solr Create New Indexes?
>
> On 03/03/2010 07:56 PM, Thomas Nguyen wrote:
>
>> Is there a setting in the config I can set to have Solr create a new
>> Lucene index if the dataDir is empty on startup? I'd like to open
our
>> Solr system to allow other developers here to add new cores without
>> having to use the Lucene API directly to create the indexes.
>>
>>
>>
>>
> You don't have to use the Lucene API though?
>
> Solr creates the index if its not there ...
>
>
--
- Mark
http://www.lucidimagination.com
Re: Can Solr Create New Indexes?
Posted by Mark Miller <ma...@gmail.com>.
I'm guessing the index folder itself already exists?
The data dir can be there, but the index dir itself must not be - that's
how it knows to create a new one.
Otherwise it thinks the empty dir is the index and cant find the files
it expects.
On 03/03/2010 08:15 PM, Thomas Nguyen wrote:
> Hmm I've tried starting Solr with no Lucene index in the dataDir.
> Here's the Exception I receive when starting Solr and when attempting to
> add a document to the core:
>
>
> 2010-03-03 16:44:06,479 [main ] ERROR
> org.apache.solr.core.CoreContainer -
> java.lang.RuntimeException: java.io.FileNotFoundException: no segments*
> file found in
> org.apache.lucene.store.SimpleFSDirectory@C:\ign\test-solr\objectIndex\i
> ndex: files:
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1068)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:579)
> at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:428)
> at
> org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
> at
> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.
> java:117)
> at
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:
> 83)
> at
> org.mortbay.jetty.servlet.FilterHolder.start(FilterHolder.java:71)
> at
> org.mortbay.jetty.servlet.WebApplicationHandler.initializeServlets(WebAp
> plicationHandler.java:310)
> at
> org.mortbay.jetty.servlet.WebApplicationContext.doStart(WebApplicationCo
> ntext.java:509)
> at
> org.mortbay.jetty.plus.PlusWebAppContext.doStart(PlusWebAppContext.java:
> 149)
> at org.mortbay.util.Container.start(Container.java:72)
> at org.mortbay.http.HttpServer.doStart(HttpServer.java:708)
> at org.mortbay.jetty.plus.Server.doStart(Server.java:153)
> at org.mortbay.util.Container.start(Container.java:72)
> at org.mortbay.jetty.plus.Server.main(Server.java:202)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
> a:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
> Impl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.mortbay.start.Main.invokeMain(Main.java:151)
> at org.mortbay.start.Main.start(Main.java:476)
> at org.mortbay.start.Main.main(Main.java:94)
> Caused by: java.io.FileNotFoundException: no segments* file found in
> org.apache.lucene.store.SimpleFSDirectory@C:\ign\test-solr\objectIndex\i
> ndex: files:
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.j
> ava:655)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
> at
> org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
> at
> org.apache.lucene.index.IndexReader.open(IndexReader.java:403)
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexR
> eaderFactory.java:38)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1057)
> ... 21 more
>
> Before this point I've been using existing Lucene indexes (created by
> the Lucene API) with Solr without a problem.
>
>
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Wednesday, March 03, 2010 5:00 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Can Solr Create New Indexes?
>
> On 03/03/2010 07:56 PM, Thomas Nguyen wrote:
>
>> Is there a setting in the config I can set to have Solr create a new
>> Lucene index if the dataDir is empty on startup? I'd like to open our
>> Solr system to allow other developers here to add new cores without
>> having to use the Lucene API directly to create the indexes.
>>
>>
>>
>>
> You don't have to use the Lucene API though?
>
> Solr creates the index if its not there ...
>
>
--
- Mark
http://www.lucidimagination.com
RE: Can Solr Create New Indexes?
Posted by Thomas Nguyen <th...@ign.com>.
Hmm I've tried starting Solr with no Lucene index in the dataDir.
Here's the Exception I receive when starting Solr and when attempting to
add a document to the core:
2010-03-03 16:44:06,479 [main ] ERROR
org.apache.solr.core.CoreContainer -
java.lang.RuntimeException: java.io.FileNotFoundException: no segments*
file found in
org.apache.lucene.store.SimpleFSDirectory@C:\ign\test-solr\objectIndex\i
ndex: files:
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1068)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:579)
at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:428)
at
org.apache.solr.core.CoreContainer.load(CoreContainer.java:278)
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.
java:117)
at
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:
83)
at
org.mortbay.jetty.servlet.FilterHolder.start(FilterHolder.java:71)
at
org.mortbay.jetty.servlet.WebApplicationHandler.initializeServlets(WebAp
plicationHandler.java:310)
at
org.mortbay.jetty.servlet.WebApplicationContext.doStart(WebApplicationCo
ntext.java:509)
at
org.mortbay.jetty.plus.PlusWebAppContext.doStart(PlusWebAppContext.java:
149)
at org.mortbay.util.Container.start(Container.java:72)
at org.mortbay.http.HttpServer.doStart(HttpServer.java:708)
at org.mortbay.jetty.plus.Server.doStart(Server.java:153)
at org.mortbay.util.Container.start(Container.java:72)
at org.mortbay.jetty.plus.Server.main(Server.java:202)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.mortbay.start.Main.invokeMain(Main.java:151)
at org.mortbay.start.Main.start(Main.java:476)
at org.mortbay.start.Main.main(Main.java:94)
Caused by: java.io.FileNotFoundException: no segments* file found in
org.apache.lucene.store.SimpleFSDirectory@C:\ign\test-solr\objectIndex\i
ndex: files:
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.j
ava:655)
at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
at
org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
at
org.apache.lucene.index.IndexReader.open(IndexReader.java:403)
at
org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexR
eaderFactory.java:38)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1057)
... 21 more
Before this point I've been using existing Lucene indexes (created by
the Lucene API) with Solr without a problem.
-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com]
Sent: Wednesday, March 03, 2010 5:00 PM
To: solr-user@lucene.apache.org
Subject: Re: Can Solr Create New Indexes?
On 03/03/2010 07:56 PM, Thomas Nguyen wrote:
> Is there a setting in the config I can set to have Solr create a new
> Lucene index if the dataDir is empty on startup? I'd like to open our
> Solr system to allow other developers here to add new cores without
> having to use the Lucene API directly to create the indexes.
>
>
>
You don't have to use the Lucene API though?
Solr creates the index if its not there ...
--
- Mark
http://www.lucidimagination.com
Re: weighted search and index
Posted by Erick Erickson <er...@gmail.com>.
Huh?
On Thu, Mar 4, 2010 at 1:13 PM, Chris Hostetter <ho...@fucit.org>wrote:
>
> : Subject: weighted search and index
> : In-reply-to: <4B...@gmail.com>
>
> http://people.apache.org/~hossman/#threadhijack
> Thread Hijacking on Mailing Lists
>
> When starting a new discussion on a mailing list, please do not reply to
> an existing message, instead start a fresh email. Even if you change the
> subject line of your email, other mail headers still track which thread
> you replied to and your question is "hidden" in that thread and gets less
> attention. It makes following discussions in the mailing list archives
> particularly difficult.
> See Also: http://en.wikipedia.org/wiki/User:DonDiego/Thread_hijacking
>
>
>
> -Hoss
>
>
Re: weighted search and index
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: weighted search and index
: In-reply-to: <4B...@gmail.com>
http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
See Also: http://en.wikipedia.org/wiki/User:DonDiego/Thread_hijacking
-Hoss
weighted search and index
Posted by Jianbin Dai <jd...@huawei.com>.
Hi,
I am trying to use solr for a content match application.
A content is described by a set of keywords with weights associated, eg.,
C1: fruit 0.8, apple 0.4, banana 0.2
C2: music 0.9, pop song 0.6, Britney Spears 0.4
Those contents would be indexed in solr.
In the search, I also have a set of keywords with weights:
Query: Sports 0.8, golf 0.5
I am trying to find the closest matching contents for this query.
My question is how to index the contents with weighted scores, and how to
write search query. I was trying to use boosting, but seems not working
right.
Thanks.
Jianbin
Re: Can Solr Create New Indexes?
Posted by Mark Miller <ma...@gmail.com>.
On 03/03/2010 07:56 PM, Thomas Nguyen wrote:
> Is there a setting in the config I can set to have Solr create a new
> Lucene index if the dataDir is empty on startup? I'd like to open our
> Solr system to allow other developers here to add new cores without
> having to use the Lucene API directly to create the indexes.
>
>
>
You don't have to use the Lucene API though?
Solr creates the index if its not there ...
--
- Mark
http://www.lucidimagination.com