You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Dotan Cohen <do...@gmail.com> on 2013/03/01 10:25:49 UTC
Returning to Solr 4.0 from 4.1
Solr 4.1 has been giving up much trouble rejecting documents indexed.
While I try to work my way through this, I would like to move our
application back to Solr 4.0. However, now when I try to start Solr
with same index that was created with Solr 4.0 but has been running on
4.1 few a few days I get this error chain:
org.apache.solr.common.SolrException: Error opening new searcher
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
Caused by: java.lang.IllegalArgumentException: A SPI class of type
org.apache.lucene.codecs.Codec with name 'Lucene41' does not exist.
You need to add the corresponding JAR file supporting this SPI to your
classpath.The current classpath supports the following names:
[Lucene40, Lucene3x]
Obviously I'll not be installing Lucene41 in Solr 4.0, but is there
any way to work around this? Note that neither solrconf.xml nor
schema.xml have changed. Thanks.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Lance Norskog <go...@gmail.com>.
Yes, the SolrEntityProcessor can be used for this.
If you stored the original document bodies in the Solr index!
You can also download the documents in Json or CSV format and re-upload
those to old Solr. I don't know if CSV will work for your docs. If CSV
works, you can directly upload what you download. If you download JSON,
you have to "unwrap" the outermost structure and upload the data as an
array.
There are problems with the SolrEntityProcessor.1) It is
single-threaded. 2) If you 'copyField' to a field, and store that field,
you have to be sure not to reload the contents of the field, because you
will add a new copy from the 'source' field.
On 03/01/2013 04:48 AM, Alexandre Rafalovitch wrote:
> What about SolrEntityProcessor in DIH?
> https://wiki.apache.org/solr/DataImportHandler#SolrEntityProcessor
>
> Regards,
> Alex.
>
> Personal blog: http://blog.outerthoughts.com/
> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> - Time is the quality of nature that keeps events from happening all at
> once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
>
>
> On Fri, Mar 1, 2013 at 5:16 AM, Dotan Cohen <do...@gmail.com> wrote:
>
>> On Fri, Mar 1, 2013 at 11:59 AM, Rafał Kuć <r....@solr.pl> wrote:
>>> Hello!
>>>
>>> I assumed that re-indexing can be painful in your case, if it wouldn't
>>> you probably would re-index by now :) I guess (didn't test it myself),
>>> that you can create another collection inside your cluster, use the
>>> old codec for Lucene 4.0 (setting the version in solrconfig.xml should
>>> be enough) and re-indexing, but still re-indexing will have to be
>>> done. Or maybe someone knows a better way ?
>>>
>> Will I have to reindex via an external script bridging, such as a
>> Python script which requests N documents at a time, indexes them into
>> Solr 4.1, then requests another N documents to index? Or is there
>> internal Solr / Lucene facility for this? I've actually looked for
>> such a facility, but as I am unable to find such a thing I ask.
>>
>>
>> --
>> Dotan Cohen
>>
>> http://gibberish.co.il
>> http://what-is-what.com
>>
Re: Returning to Solr 4.0 from 4.1
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
What about SolrEntityProcessor in DIH?
https://wiki.apache.org/solr/DataImportHandler#SolrEntityProcessor
Regards,
Alex.
Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
On Fri, Mar 1, 2013 at 5:16 AM, Dotan Cohen <do...@gmail.com> wrote:
> On Fri, Mar 1, 2013 at 11:59 AM, Rafał Kuć <r....@solr.pl> wrote:
> > Hello!
> >
> > I assumed that re-indexing can be painful in your case, if it wouldn't
> > you probably would re-index by now :) I guess (didn't test it myself),
> > that you can create another collection inside your cluster, use the
> > old codec for Lucene 4.0 (setting the version in solrconfig.xml should
> > be enough) and re-indexing, but still re-indexing will have to be
> > done. Or maybe someone knows a better way ?
> >
>
> Will I have to reindex via an external script bridging, such as a
> Python script which requests N documents at a time, indexes them into
> Solr 4.1, then requests another N documents to index? Or is there
> internal Solr / Lucene facility for this? I've actually looked for
> such a facility, but as I am unable to find such a thing I ask.
>
>
> --
> Dotan Cohen
>
> http://gibberish.co.il
> http://what-is-what.com
>
Re: Returning to Solr 4.0 from 4.1
Posted by Dotan Cohen <do...@gmail.com>.
On Sat, Mar 2, 2013 at 9:32 PM, Upayavira <uv...@odoko.co.uk> wrote:
> What I'm questioning is whether the issue you see in 4.1 has been
> resolved in Subversion. While I would not expect 4.0 to read a 4.1
> index, the SVN branch/4.2 should be able to do so effortlessly.
>
> Upayavira
>
I see, thanks. Actually, running a clean 4.1 with no previous index
does not have the issues.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Upayavira <uv...@odoko.co.uk>.
What I'm questioning is whether the issue you see in 4.1 has been
resolved in Subversion. While I would not expect 4.0 to read a 4.1
index, the SVN branch/4.2 should be able to do so effortlessly.
Upayavira
On Sat, Mar 2, 2013, at 06:17 PM, Dotan Cohen wrote:
> On Fri, Mar 1, 2013 at 1:37 PM, Upayavira <uv...@odoko.co.uk> wrote:
> > Can you use a checkout from SVN? Does that resolve your issues? That is
> > what will become 4.2 when it is released soon:
> >
> > https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x/
> >
> > Upayavira
> >
>
> Thank you. Which feature of 4.2 are you suggesting for this issue? Can
> Solr 4.2 natively import from a Solr index?
>
>
> --
> Dotan Cohen
>
> http://gibberish.co.il
> http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Dotan Cohen <do...@gmail.com>.
On Fri, Mar 1, 2013 at 1:37 PM, Upayavira <uv...@odoko.co.uk> wrote:
> Can you use a checkout from SVN? Does that resolve your issues? That is
> what will become 4.2 when it is released soon:
>
> https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x/
>
> Upayavira
>
Thank you. Which feature of 4.2 are you suggesting for this issue? Can
Solr 4.2 natively import from a Solr index?
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Upayavira <uv...@odoko.co.uk>.
Can you use a checkout from SVN? Does that resolve your issues? That is
what will become 4.2 when it is released soon:
https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x/
Upayavira
On Fri, Mar 1, 2013, at 10:51 AM, Dotan Cohen wrote:
> On Fri, Mar 1, 2013 at 12:22 PM, Rafał Kuć <r....@solr.pl> wrote:
> > Hello!
> >
> > As far as I know you have to re-index using external tool.
> >
>
> Thank you Rafał. That is what I figured.
>
>
>
> --
> Dotan Cohen
>
> http://gibberish.co.il
> http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Dotan Cohen <do...@gmail.com>.
On Fri, Mar 1, 2013 at 12:22 PM, Rafał Kuć <r....@solr.pl> wrote:
> Hello!
>
> As far as I know you have to re-index using external tool.
>
Thank you Rafał. That is what I figured.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Rafał Kuć <r....@solr.pl>.
Hello!
As far as I know you have to re-index using external tool.
--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch
> On Fri, Mar 1, 2013 at 11:59 AM, Rafał Kuć <r....@solr.pl> wrote:
>> Hello!
>>
>> I assumed that re-indexing can be painful in your case, if it wouldn't
>> you probably would re-index by now :) I guess (didn't test it myself),
>> that you can create another collection inside your cluster, use the
>> old codec for Lucene 4.0 (setting the version in solrconfig.xml should
>> be enough) and re-indexing, but still re-indexing will have to be
>> done. Or maybe someone knows a better way ?
>>
> Will I have to reindex via an external script bridging, such as a
> Python script which requests N documents at a time, indexes them into
> Solr 4.1, then requests another N documents to index? Or is there
> internal Solr / Lucene facility for this? I've actually looked for
> such a facility, but as I am unable to find such a thing I ask.
Re: Returning to Solr 4.0 from 4.1
Posted by Dotan Cohen <do...@gmail.com>.
On Fri, Mar 1, 2013 at 11:59 AM, Rafał Kuć <r....@solr.pl> wrote:
> Hello!
>
> I assumed that re-indexing can be painful in your case, if it wouldn't
> you probably would re-index by now :) I guess (didn't test it myself),
> that you can create another collection inside your cluster, use the
> old codec for Lucene 4.0 (setting the version in solrconfig.xml should
> be enough) and re-indexing, but still re-indexing will have to be
> done. Or maybe someone knows a better way ?
>
Will I have to reindex via an external script bridging, such as a
Python script which requests N documents at a time, indexes them into
Solr 4.1, then requests another N documents to index? Or is there
internal Solr / Lucene facility for this? I've actually looked for
such a facility, but as I am unable to find such a thing I ask.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Rafał Kuć <r....@solr.pl>.
Hello!
I assumed that re-indexing can be painful in your case, if it wouldn't
you probably would re-index by now :) I guess (didn't test it myself),
that you can create another collection inside your cluster, use the
old codec for Lucene 4.0 (setting the version in solrconfig.xml should
be enough) and re-indexing, but still re-indexing will have to be
done. Or maybe someone knows a better way ?
--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch
> On Fri, Mar 1, 2013 at 11:28 AM, Rafał Kuć <r....@solr.pl> wrote:
>> Hello!
>>
>> I suppose the only way to make this work will be reindexing the data.
>> Solr 4.1 uses Lucene 4.1 as you know, which introduced new default
>> codec with stored fields compression and this is one of the reasons
>> you can't read that index with 4.0.
>>
> Thank you. My first inclination is to "reindex" the documents, but the
> only store of these documents is the Solr index itself. I am trying to
> find solutions to create a new core and to index the data in the old
> core into the new core. I'm not finding any good ways of going about
> this.
> Note that we are talking about ~18,000,000 (yes, 18 million) small
> documents similar to 'tweets' (mostly under 1 KiB each, very very few
> over 5 KiB).
Re: Returning to Solr 4.0 from 4.1
Posted by Dotan Cohen <do...@gmail.com>.
On Fri, Mar 1, 2013 at 11:28 AM, Rafał Kuć <r....@solr.pl> wrote:
> Hello!
>
> I suppose the only way to make this work will be reindexing the data.
> Solr 4.1 uses Lucene 4.1 as you know, which introduced new default
> codec with stored fields compression and this is one of the reasons
> you can't read that index with 4.0.
>
Thank you. My first inclination is to "reindex" the documents, but the
only store of these documents is the Solr index itself. I am trying to
find solutions to create a new core and to index the data in the old
core into the new core. I'm not finding any good ways of going about
this.
Note that we are talking about ~18,000,000 (yes, 18 million) small
documents similar to 'tweets' (mostly under 1 KiB each, very very few
over 5 KiB).
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
Re: Returning to Solr 4.0 from 4.1
Posted by Rafał Kuć <r....@solr.pl>.
Hello!
I suppose the only way to make this work will be reindexing the data.
Solr 4.1 uses Lucene 4.1 as you know, which introduced new default
codec with stored fields compression and this is one of the reasons
you can't read that index with 4.0.
--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch
> Solr 4.1 has been giving up much trouble rejecting documents indexed.
> While I try to work my way through this, I would like to move our
> application back to Solr 4.0. However, now when I try to start Solr
> with same index that was created with Solr 4.0 but has been running on
> 4.1 few a few days I get this error chain:
> org.apache.solr.common.SolrException: Error opening new searcher
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> Caused by: java.lang.IllegalArgumentException: A SPI class of type
> org.apache.lucene.codecs.Codec with name 'Lucene41' does not exist.
> You need to add the corresponding JAR file supporting this SPI to your
> classpath.The current classpath supports the following names:
> [Lucene40, Lucene3x]
> Obviously I'll not be installing Lucene41 in Solr 4.0, but is there
> any way to work around this? Note that neither solrconf.xml nor
> schema.xml have changed. Thanks.