You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Chris M. Hostetter (Jira)" <ji...@apache.org> on 2019/10/25 00:59:00 UTC
[jira] [Updated] (SOLR-13872) Backup can fail to read index files
w/NoSuchFileException during merges (SOLR-11616 regression)
[ https://issues.apache.org/jira/browse/SOLR-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris M. Hostetter updated SOLR-13872:
--------------------------------------
Attachment: index_churn.pl
Status: Open (was: Open)
This bug is fairly easy to reproduce using the "techproducts" example – I'm attaching an {{index_churn.pl}} script that I used to continuously add/delete documents with randomized commits & optimizes.
With this script and the commands below, it never takes more then a few minutes to trigger a failure – but any sort of heavy indexing/merging should be able to trigger it as well.
(Run the following commands in diff terminals)
# (ant clean && cd solr/ && ant server && bin/solr -e techproducts -noprompt) && perl ~/tmp/index_churn.pl
# watch -n1 -e "curl -D - -sS 'http://localhost:8983/solr/techproducts/replication?command=backup&location=/tmp'"
# (tail -F solr/example/techproducts/logs/solr.log | grep -m1 ERROR) ; echo "Found an error in logs"
...and wait for 3rd command to exit, Check the solr.log for details
I've confirmed this fails in:
- 7.6 (where I first noticed SOLR-11616 wasn't actual fixed)
- 7.2 (fix version for SOLR-11616)
- branch_8x (fb37f89af27d881efb0f7b40eff2dc76ef5bc2c4)
- master (5289fce4bf857861af3644f6240a8369aa9f5407)
Sample error from master...
{noformat}
2019-10-25 00:21:49.170 ERROR (Thread-83) [ ] o.a.s.h.SnapShooter Exception while creating snapshot => java.nio.file.NoSuchFileException: /home/hossman/lucene/alt_dev/solr/example/techproducts/solr/techproducts/data/index/segments_10l
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
java.nio.file.NoSuchFileException: /home/hossman/lucene/alt_dev/solr/example/techproducts/solr/techproducts/data/index/segments_10l
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:178) ~[?:?]
at java.nio.channels.FileChannel.open(FileChannel.java:292) ~[?:?]
at java.nio.channels.FileChannel.open(FileChannel.java:345) ~[?:?]
at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:237) ~[?:?]
at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:200) ~[?:?]
at org.apache.lucene.store.Directory.copyFrom(Directory.java:182) ~[?:?]
at org.apache.solr.core.backup.repository.LocalFileSystemRepository.copyFileFrom(LocalFileSystemRepository.java:145) ~[?:?]
at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:238) ~[?:?]
at org.apache.solr.handler.SnapShooter.lambda$createSnapAsync$2(SnapShooter.java:205) ~[?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
{noformat}
I have not yet dug very deep into the changes made in SOLR-11616 to try and understand wy they didn't fix the problem.
/cc [~varunthacker] to see if he has any ideas what the problem may be.
> Backup can fail to read index files w/NoSuchFileException during merges (SOLR-11616 regression)
> ------------------------------------------------------------------------------------------------
>
> Key: SOLR-13872
> URL: https://issues.apache.org/jira/browse/SOLR-13872
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Chris M. Hostetter
> Priority: Major
> Attachments: index_churn.pl
>
>
> SOLR-11616 purports to fix a bug in Solr's backup functionality that causes 'NoSuchFileException' errors when attempting to backup an index while it is undergoing indexing (and segment merging)
> Although SOLR-11616 is marked with "Fix Version: 7.2" it's pretty easy to demonstrate that this bug still exists on master, branch_8x, and even in 7.2 - so it seems less like the current problem is a "regression" and more that the original fix didn't work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org