You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2021/09/03 01:02:39 UTC
[GitHub] [lucene] zacharymorn opened a new pull request #281: LUCENE-9662: Update concurrent index checking usage instructions
zacharymorn opened a new pull request #281:
URL: https://github.com/apache/lucene/pull/281
# Description
Update concurrent index checking usage instructions
# Sample output
```
> Task :lucene:core:CheckIndex.main() FAILED
ERROR: index path not specified
Usage: java org.apache.lucene.index.CheckIndex pathToIndex [-exorcise] [-slow] [-segment X] [-segment Y] [-threadCount X] [-dir-impl X]
-exorcise: actually write a new segments_N file, removing any problematic segments
-fast: just verify file checksums, omitting logical integrity checks
-slow: do additional slow checks; THIS IS VERY SLOW!
-codec X: when exorcising, codec to write the new segments_N file with
-verbose: print additional details
-segment X: only check the specified segments. This can be specified multiple
times, to check more than one segment, eg '-segment _2 -segment _a'.
You can't use this with the -exorcise option
-threadCount X: number of new threads created and used to check index concurrently.
When not specified, this will default to the number of CPU cores up to 4.
When '-threadCount 1' is used, index checking will be performed sequentially.
-dir-impl X: use a specific FSDirectory implementation. If no package is specified the org.apache.lucene.store package will be used.
**WARNING**: -exorcise *LOSES DATA*. This should only be used on an emergency basis as it will cause
documents (perhaps many) to be permanently removed from the index. Always make
a backup copy of your index before running this! Do not run this tool on an index
that is actively being written to. You have been warned!
Run without -exorcise, this tool will open the index, report version information
and report any exceptions it hits and what action it would take if -exorcise were
specified. With -exorcise, this tool will remove any segments that have issues and
write a new segments_N file. This means all documents contained in the affected
segments will be removed.
This tool exits with exit code 1 if the index cannot be opened or has any
corruption, else 0.
```
# Checklist
Please review the following and check all that apply:
- [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/lucene/HowToContribute) and my code conforms to the standards described there to the best of my ability.
- [x] I have created a Jira issue and added the issue ID to my pull request title.
- [x] I have given Lucene maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
- [x] I have developed this patch against the `main` branch.
- [x] I have run `./gradlew check`.
- [ ] I have added tests for my changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] zacharymorn commented on a change in pull request #281: LUCENE-9662: Update concurrent index checking usage instructions
Posted by GitBox <gi...@apache.org>.
zacharymorn commented on a change in pull request #281:
URL: https://github.com/apache/lucene/pull/281#discussion_r703985456
##########
File path: lucene/core/src/java/org/apache/lucene/index/CheckIndex.java
##########
@@ -3994,6 +3994,9 @@ public static Options parseOptions(String[] args) {
+ " -segment X: only check the specified segments. This can be specified multiple\n"
+ " times, to check more than one segment, eg '-segment _2 -segment _a'.\n"
+ " You can't use this with the -exorcise option\n"
+ + " -threadCount X: number of new threads created and used to check index concurrently.\n"
+ + " When not specified, this will default to the number of CPU cores up to 4.\n"
Review comment:
Sounds good. Updated the code and instruction here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] mikemccand commented on a change in pull request #281: LUCENE-9662: Update concurrent index checking usage instructions
Posted by GitBox <gi...@apache.org>.
mikemccand commented on a change in pull request #281:
URL: https://github.com/apache/lucene/pull/281#discussion_r703468423
##########
File path: lucene/core/src/java/org/apache/lucene/index/CheckIndex.java
##########
@@ -3994,6 +3994,9 @@ public static Options parseOptions(String[] args) {
+ " -segment X: only check the specified segments. This can be specified multiple\n"
+ " times, to check more than one segment, eg '-segment _2 -segment _a'.\n"
+ " You can't use this with the -exorcise option\n"
+ + " -threadCount X: number of new threads created and used to check index concurrently.\n"
Review comment:
Maybe just say `number of threads used to check index concurrently`? I.e. drop the "new" because then I start to wonder if `main` thread counts :)
##########
File path: lucene/core/src/java/org/apache/lucene/index/CheckIndex.java
##########
@@ -3994,6 +3994,9 @@ public static Options parseOptions(String[] args) {
+ " -segment X: only check the specified segments. This can be specified multiple\n"
+ " times, to check more than one segment, eg '-segment _2 -segment _a'.\n"
+ " You can't use this with the -exorcise option\n"
+ + " -threadCount X: number of new threads created and used to check index concurrently.\n"
+ + " When not specified, this will default to the number of CPU cores up to 4.\n"
Review comment:
Maybe we should change this default to not cap at `4`? Just use number of cores? This is command-line execution, which is typically done only once at a time (versus when `CheckIndex` is invoked from our tests...).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] zacharymorn merged pull request #281: LUCENE-9662: Update concurrent index checking usage instructions
Posted by GitBox <gi...@apache.org>.
zacharymorn merged pull request #281:
URL: https://github.com/apache/lucene/pull/281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] zacharymorn commented on pull request #281: LUCENE-9662: Update concurrent index checking usage instructions
Posted by GitBox <gi...@apache.org>.
zacharymorn commented on pull request #281:
URL: https://github.com/apache/lucene/pull/281#issuecomment-916599761
> Looks great! Thanks @zacharymorn!
Thanks Michael for the review and approval!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] zacharymorn commented on a change in pull request #281: LUCENE-9662: Update concurrent index checking usage instructions
Posted by GitBox <gi...@apache.org>.
zacharymorn commented on a change in pull request #281:
URL: https://github.com/apache/lucene/pull/281#discussion_r703985362
##########
File path: lucene/core/src/java/org/apache/lucene/index/CheckIndex.java
##########
@@ -3994,6 +3994,9 @@ public static Options parseOptions(String[] args) {
+ " -segment X: only check the specified segments. This can be specified multiple\n"
+ " times, to check more than one segment, eg '-segment _2 -segment _a'.\n"
+ " You can't use this with the -exorcise option\n"
+ + " -threadCount X: number of new threads created and used to check index concurrently.\n"
Review comment:
Ah yes sorry :D . Updated the instruction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org