You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by Mark Miller <ma...@gmail.com> on 2009/08/04 02:21:22 UTC
Upgrading Lucene
When we upgrade to the latest Lucene 2.9, I've seen a few issues so far
that have to be resolved:
1. There is a new way to set constant score mode on prefix query. I've
got a patch for this.
2. The Highlighter has a new API and the RegexFragmenter needs to be
updated. I've got a patch for this.
3. LUCENE-1685 is going to cause another couple little breaks with the
Highlighter - waiting to commit that shortly and then I will fix those
as well. (https://issues.apache.org/jira/browse/LUCENE-1685)
4. You cannot instantiate MergePolicy with a no arg constructor anymore
- so that fails now. I don't have a fix for this at the moment.
Thats all I've seen so far.
--
- Mark
http://www.lucidimagination.com
Re: Upgrading Lucene
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Wed, Aug 5, 2009 at 9:29 PM, Mark Miller <ma...@gmail.com> wrote:
> Shalin Shekhar Mangar wrote:
>
>> On Tue, Aug 4, 2009 at 5:51 AM, Mark Miller <ma...@gmail.com>
>> wrote:
>>
>>
>>
>>> When we upgrade to the latest Lucene 2.9, I've seen a few issues so far
>>> that have to be resolved:
>>>
>>>
>>>
>>
>> When do we plan to upgrade?
>>
>>
>>
> Soon I hope? Anyone object to ASAP?
>
>
Nope, go ahead.
The SortField.getComparator and the FieldComparator API has also changed a
bit.
--
Regards,
Shalin Shekhar Mangar.
Re: Upgrading Lucene
Posted by Mark Miller <ma...@gmail.com>.
Shalin Shekhar Mangar wrote:
> On Tue, Aug 4, 2009 at 5:51 AM, Mark Miller <ma...@gmail.com> wrote:
>
>
>> When we upgrade to the latest Lucene 2.9, I've seen a few issues so far
>> that have to be resolved:
>>
>>
>
> When do we plan to upgrade?
>
>
Soon I hope? Anyone object to ASAP?
--
- Mark
http://www.lucidimagination.com
Re: Upgrading Lucene
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Aug 4, 2009 at 5:51 AM, Mark Miller <ma...@gmail.com> wrote:
> When we upgrade to the latest Lucene 2.9, I've seen a few issues so far
> that have to be resolved:
>
When do we plan to upgrade?
--
Regards,
Shalin Shekhar Mangar.
Re: Upgrading Lucene
Posted by Mark Miller <ma...@gmail.com>.
Grant Ingersoll wrote:
>
> On Aug 5, 2009, at 6:07 PM, Mark Miller wrote:
>
>> Mark Miller wrote:
>>> Grant Ingersoll wrote:
>>>>
>>>> On Aug 3, 2009, at 8:21 PM, Mark Miller wrote:
>>>>
>>>>>
>>>>> 4. You cannot instantiate MergePolicy with a no arg constructor
>>>>> anymore - so that fails now. I don't have a fix for this at the
>>>>> moment.
>>>>
>>>>
>>>> That sounds like a back compat break ;-)
>>>>
>>> It was - but they knew it would be and decided it was fine. The
>>> methods on the class were package private, so it appeared
>>> reasonable. The class was also labeled as expert and subject to
>>> sudden change. I guess it was fair game to break - I don't think
>>> this scenario was thought of, but I would think we can work around
>>> it. I havn't really thought about it yet myself though.
>>>
>>>
>> So this is a bit tricky I guess. The way they handled this in Lucene
>> Benchmark is:
>> writer.setMergePolicy((MergePolicy)
>> Class.forName(mergePolicy).getConstructor(new Class[] {
>> IndexWriter.class }).newInstance(new Object[] { writer }));
>>
>> Now if we handle it the same way, thats fine. But then you can't put
>> one of these in solr.home/lib. To do that, you have to use
>> SolrResourceLoader.newInstance - which requires a no arg constructor.
>
> There's a newInstance on SolrResourceLoader that can take args, if
> that helps.
Ah, nice - I think it does. I kept glossing over it and just saw:
public Object newInstance(String cname, String ... subpackages)
There is also public Object newInstance(String cName, String []
subPackages, Class[] params, Object[] args)
Thanks - it looks like it should work.
>
>
>> I think we can do something like: create an object with
>> SolrResourceLoader that loads the MergePolicy - it can use
>> Class.forName like above because it will use the Classloader of the
>> object that invoked it. But I'd rather not go there if we don't have
>> to. This is a pretty advanced plugin, and likely just intended for
>> picking between included Lucene impls. Do we want to make sure it can
>> still be loaded from solr.home/lib?
>
>
> We could require that any implementations take a writer. I think we
> need a better way of taking in arbitrary attributes. Basically,
> Spring or GUICE or whatever, but that isn't going to happen overnight.
>
Right - and we want to upgrade now :) Looks like we have the
functionality we need though, thanks !
--
- Mark
http://www.lucidimagination.com
Re: Upgrading Lucene
Posted by Grant Ingersoll <gs...@apache.org>.
On Aug 5, 2009, at 6:07 PM, Mark Miller wrote:
> Mark Miller wrote:
>> Grant Ingersoll wrote:
>>>
>>> On Aug 3, 2009, at 8:21 PM, Mark Miller wrote:
>>>
>>>>
>>>> 4. You cannot instantiate MergePolicy with a no arg constructor
>>>> anymore - so that fails now. I don't have a fix for this at the
>>>> moment.
>>>
>>>
>>> That sounds like a back compat break ;-)
>>>
>> It was - but they knew it would be and decided it was fine. The
>> methods on the class were package private, so it appeared
>> reasonable. The class was also labeled as expert and subject to
>> sudden change. I guess it was fair game to break - I don't think
>> this scenario was thought of, but I would think we can work around
>> it. I havn't really thought about it yet myself though.
>>
>>
> So this is a bit tricky I guess. The way they handled this in Lucene
> Benchmark is:
> writer.setMergePolicy((MergePolicy)
> Class.forName(mergePolicy).getConstructor(new Class[]
> { IndexWriter.class }).newInstance(new Object[] { writer }));
>
> Now if we handle it the same way, thats fine. But then you can't put
> one of these in solr.home/lib. To do that, you have to use
> SolrResourceLoader.newInstance - which requires a no arg constructor.
There's a newInstance on SolrResourceLoader that can take args, if
that helps.
> I think we can do something like: create an object with
> SolrResourceLoader that loads the MergePolicy - it can use
> Class.forName like above because it will use the Classloader of the
> object that invoked it. But I'd rather not go there if we don't have
> to. This is a pretty advanced plugin, and likely just intended for
> picking between included Lucene impls. Do we want to make sure it
> can still be loaded from solr.home/lib?
We could require that any implementations take a writer. I think we
need a better way of taking in arbitrary attributes. Basically,
Spring or GUICE or whatever, but that isn't going to happen overnight.
Re: Upgrading Lucene
Posted by Mark Miller <ma...@gmail.com>.
Mark Miller wrote:
> Grant Ingersoll wrote:
>>
>> On Aug 3, 2009, at 8:21 PM, Mark Miller wrote:
>>
>>>
>>> 4. You cannot instantiate MergePolicy with a no arg constructor
>>> anymore - so that fails now. I don't have a fix for this at the moment.
>>
>>
>> That sounds like a back compat break ;-)
>>
> It was - but they knew it would be and decided it was fine. The
> methods on the class were package private, so it appeared reasonable.
> The class was also labeled as expert and subject to sudden change. I
> guess it was fair game to break - I don't think this scenario was
> thought of, but I would think we can work around it. I havn't really
> thought about it yet myself though.
>
>
So this is a bit tricky I guess. The way they handled this in Lucene
Benchmark is:
writer.setMergePolicy((MergePolicy)
Class.forName(mergePolicy).getConstructor(new Class[] {
IndexWriter.class }).newInstance(new Object[] { writer }));
Now if we handle it the same way, thats fine. But then you can't put one
of these in solr.home/lib. To do that, you have to use
SolrResourceLoader.newInstance - which requires a no arg constructor. I
think we can do something like: create an object with SolrResourceLoader
that loads the MergePolicy - it can use Class.forName like above because
it will use the Classloader of the object that invoked it. But I'd
rather not go there if we don't have to. This is a pretty advanced
plugin, and likely just intended for picking between included Lucene
impls. Do we want to make sure it can still be loaded from solr.home/lib?
--
- Mark
http://www.lucidimagination.com
Re: Upgrading Lucene
Posted by Mark Miller <ma...@gmail.com>.
Grant Ingersoll wrote:
>
> On Aug 3, 2009, at 8:21 PM, Mark Miller wrote:
>
>>
>> 4. You cannot instantiate MergePolicy with a no arg constructor
>> anymore - so that fails now. I don't have a fix for this at the moment.
>
>
> That sounds like a back compat break ;-)
>
It was - but they knew it would be and decided it was fine. The methods
on the class were package private, so it appeared reasonable. The class
was also labeled as expert and subject to sudden change. I guess it was
fair game to break - I don't think this scenario was thought of, but I
would think we can work around it. I havn't really thought about it yet
myself though.
--
- Mark
http://www.lucidimagination.com
Re: Upgrading Lucene
Posted by Grant Ingersoll <gs...@apache.org>.
On Aug 3, 2009, at 8:21 PM, Mark Miller wrote:
>
> 4. You cannot instantiate MergePolicy with a no arg constructor
> anymore - so that fails now. I don't have a fix for this at the
> moment.
That sounds like a back compat break ;-)
This piece is starting to get a bit tricky. We have a couple of
places where we I think we need to allow for user args to somehow be
passed in on custom settings (the MergeScheduler comes to mind). I
think Jason R. opened a bug for one recently as well, but I'm not
finding it at the moment.