You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Kayak28 <ka...@gmail.com> on 2019/11/06 01:24:04 UTC
Questions about corrupted Segments files.
Hello, Community members:
I am using Solr 7.7.2.
On the other day, while indexing to the Solr, my computer powered off.
As a result, there are corrupted segment files.
Is there any way to fix the corrupted segment files without re-indexing?
I have read a blog post (in Japanese) writing about checkIndex method which
can be used to determine/fix corrupted segment files, but when I tried to
run the following command, I got the error message.
So, I am not sure if checkIndex can actually fix the index files.
java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
-fix
ERROR: unexpected extra argument '-fix'
If anybody knows about either a way to fix corrupted segment files or a way
to use checkIndex '-fix' option correctly, could you please let me know?
Any clue will be very appreciated.
Sincerely,
Kaya Ota
Re: Questions about corrupted Segments files.
Posted by Kayak28 <ka...@gmail.com>.
Hello, Mr. Erick, Mr. Dmitry, and the community members:
Thank you for your advice.
I am going to try Luck and --exorcise option this weekend.
Again, I appreciated your replays.
Sincerely,
Kaya Ota
2019年11月6日(水) 22:36 Erick Erickson <er...@gmail.com>:
> If Luke doesn’t do the trick, use the -exorcise option and start your
> indexing process over again.
>
> Best,
> Erick
>
> > On Nov 6, 2019, at 6:24 AM, Dmitry Kan <dm...@gmail.com> wrote:
> >
> > Hi Kaya,
> >
> > Try luke:
> > http://dmitrykan.blogspot.com/2018/01/new-luke-on-javafx.html
> >
> > Best,
> >
> > Dmitry
> >
> > On Wed 6. Nov 2019 at 3.24, Kayak28 <ka...@gmail.com> wrote:
> > Hello, Community members:
> >
> > I am using Solr 7.7.2.
> > On the other day, while indexing to the Solr, my computer powered off.
> > As a result, there are corrupted segment files.
> >
> > Is there any way to fix the corrupted segment files without re-indexing?
> >
> > I have read a blog post (in Japanese) writing about checkIndex method
> which can be used to determine/fix corrupted segment files, but when I
> tried to run the following command, I got the error message.
> > So, I am not sure if checkIndex can actually fix the index files.
> >
> >
> > java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
> org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
> -fix
> >
> > ERROR: unexpected extra argument '-fix'
> >
> >
> >
> > If anybody knows about either a way to fix corrupted segment files or a
> way to use checkIndex '-fix' option correctly, could you please let me
> know?
> >
> > Any clue will be very appreciated.
> >
> > Sincerely,
> > Kaya Ota
> >
> >
> > --
> > --
> > Dmitry Kan
> > Luke Toolbox: http://github.com/DmitryKey/luke
> > Blog: http://dmitrykan.blogspot.com
> > Twitter: http://twitter.com/dmitrykan
> > SemanticAnalyzer: www.semanticanalyzer.info
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Questions about corrupted Segments files.
Posted by Erick Erickson <er...@gmail.com>.
If Luke doesn’t do the trick, use the -exorcise option and start your indexing process over again.
Best,
Erick
> On Nov 6, 2019, at 6:24 AM, Dmitry Kan <dm...@gmail.com> wrote:
>
> Hi Kaya,
>
> Try luke:
> http://dmitrykan.blogspot.com/2018/01/new-luke-on-javafx.html
>
> Best,
>
> Dmitry
>
> On Wed 6. Nov 2019 at 3.24, Kayak28 <ka...@gmail.com> wrote:
> Hello, Community members:
>
> I am using Solr 7.7.2.
> On the other day, while indexing to the Solr, my computer powered off.
> As a result, there are corrupted segment files.
>
> Is there any way to fix the corrupted segment files without re-indexing?
>
> I have read a blog post (in Japanese) writing about checkIndex method which can be used to determine/fix corrupted segment files, but when I tried to run the following command, I got the error message.
> So, I am not sure if checkIndex can actually fix the index files.
>
>
> java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index -fix
>
> ERROR: unexpected extra argument '-fix'
>
>
>
> If anybody knows about either a way to fix corrupted segment files or a way to use checkIndex '-fix' option correctly, could you please let me know?
>
> Any clue will be very appreciated.
>
> Sincerely,
> Kaya Ota
>
>
> --
> --
> Dmitry Kan
> Luke Toolbox: http://github.com/DmitryKey/luke
> Blog: http://dmitrykan.blogspot.com
> Twitter: http://twitter.com/dmitrykan
> SemanticAnalyzer: www.semanticanalyzer.info
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Questions about corrupted Segments files.
Posted by Dmitry Kan <dm...@gmail.com>.
Hi Kaya,
Try luke:
http://dmitrykan.blogspot.com/2018/01/new-luke-on-javafx.html
Best,
Dmitry
On Wed 6. Nov 2019 at 3.24, Kayak28 <ka...@gmail.com> wrote:
> Hello, Community members:
>
> I am using Solr 7.7.2.
> On the other day, while indexing to the Solr, my computer powered off.
> As a result, there are corrupted segment files.
>
> Is there any way to fix the corrupted segment files without re-indexing?
>
> I have read a blog post (in Japanese) writing about checkIndex method
> which can be used to determine/fix corrupted segment files, but when I
> tried to run the following command, I got the error message.
> So, I am not sure if checkIndex can actually fix the index files.
>
>
> java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
> org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
> -fix
>
>
> ERROR: unexpected extra argument '-fix'
>
>
>
> If anybody knows about either a way to fix corrupted segment files or a
> way to use checkIndex '-fix' option correctly, could you please let me
> know?
>
> Any clue will be very appreciated.
>
> Sincerely,
> Kaya Ota
>
>
>
--
--
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info
Re: Questions about corrupted Segments files.
Posted by Kayak28 <ka...@gmail.com>.
Hello, Mr. Erick Erickson and Community Members:
Thank you for your quick response.
I understand that "checkIndex -exorcise" does not actually fix the indices,
more like "first aid" kinda action.
And soon or later I need to re-index from my source.
What is the rule of thumb to recover from the corrupted segment files?
Sincerely,
Kaya Ota
2019年11月6日(水) 10:56 Erick Erickson <er...@gmail.com>:
> Well, the fix option “fixes” the index by deleting any segments it can’t
> read, thus also deleting the documents in that segment. So if you can get
> it to run, it’ll allow you to continue using the index, but you’ll have to
> re-index all your data anyway to get the documents in the deleted segments
> back. It’s intended as a stopgap, not a true repair.
>
> -fix has been renamed -exorcise to emphasize that it has serious
> consequences, see: https://issues.apache.org/jira/browse/LUCENE-5924
>
> Best,
> Erick
>
> > On Nov 5, 2019, at 8:24 PM, Kayak28 <ka...@gmail.com> wrote:
> >
> > Hello, Community members:
> >
> > I am using Solr 7.7.2.
> > On the other day, while indexing to the Solr, my computer powered off.
> > As a result, there are corrupted segment files.
> >
> > Is there any way to fix the corrupted segment files without re-indexing?
> >
> > I have read a blog post (in Japanese) writing about checkIndex method
> which
> > can be used to determine/fix corrupted segment files, but when I tried to
> > run the following command, I got the error message.
> > So, I am not sure if checkIndex can actually fix the index files.
> >
> >
> > java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
> > org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
> > -fix
> >
> >
> > ERROR: unexpected extra argument '-fix'
> >
> >
> >
> > If anybody knows about either a way to fix corrupted segment files or a
> way
> > to use checkIndex '-fix' option correctly, could you please let me know?
> >
> > Any clue will be very appreciated.
> >
> > Sincerely,
> > Kaya Ota
>
>
Re: Questions about corrupted Segments files.
Posted by Erick Erickson <er...@gmail.com>.
Well, the fix option “fixes” the index by deleting any segments it can’t read, thus also deleting the documents in that segment. So if you can get it to run, it’ll allow you to continue using the index, but you’ll have to re-index all your data anyway to get the documents in the deleted segments back. It’s intended as a stopgap, not a true repair.
-fix has been renamed -exorcise to emphasize that it has serious consequences, see: https://issues.apache.org/jira/browse/LUCENE-5924
Best,
Erick
> On Nov 5, 2019, at 8:24 PM, Kayak28 <ka...@gmail.com> wrote:
>
> Hello, Community members:
>
> I am using Solr 7.7.2.
> On the other day, while indexing to the Solr, my computer powered off.
> As a result, there are corrupted segment files.
>
> Is there any way to fix the corrupted segment files without re-indexing?
>
> I have read a blog post (in Japanese) writing about checkIndex method which
> can be used to determine/fix corrupted segment files, but when I tried to
> run the following command, I got the error message.
> So, I am not sure if checkIndex can actually fix the index files.
>
>
> java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
> org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
> -fix
>
>
> ERROR: unexpected extra argument '-fix'
>
>
>
> If anybody knows about either a way to fix corrupted segment files or a way
> to use checkIndex '-fix' option correctly, could you please let me know?
>
> Any clue will be very appreciated.
>
> Sincerely,
> Kaya Ota
Re: Questions about corrupted Segments files.
Posted by Dmitry Kan <dm...@gmail.com>.
Hi Kaya,
Try luke:
http://dmitrykan.blogspot.com/2018/01/new-luke-on-javafx.html
Best,
Dmitry
On Wed 6. Nov 2019 at 3.24, Kayak28 <ka...@gmail.com> wrote:
> Hello, Community members:
>
> I am using Solr 7.7.2.
> On the other day, while indexing to the Solr, my computer powered off.
> As a result, there are corrupted segment files.
>
> Is there any way to fix the corrupted segment files without re-indexing?
>
> I have read a blog post (in Japanese) writing about checkIndex method
> which can be used to determine/fix corrupted segment files, but when I
> tried to run the following command, I got the error message.
> So, I am not sure if checkIndex can actually fix the index files.
>
>
> java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
> org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
> -fix
>
>
> ERROR: unexpected extra argument '-fix'
>
>
>
> If anybody knows about either a way to fix corrupted segment files or a
> way to use checkIndex '-fix' option correctly, could you please let me
> know?
>
> Any clue will be very appreciated.
>
> Sincerely,
> Kaya Ota
>
>
>
--
--
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info