You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Chris Harris <ry...@gmail.com> on 2009/09/28 23:36:48 UTC
FileNotFoundException in Java replication handler backups
Thanks to Noble Paul, I think I now understand the Java replication
handler's backup feature. It seems to work as expected on a toy index.
When trying it out on a copy of my production index (300GB-ish),
though, I'm getting FileNotFoundExceptions. These cancel the backup,
and delete the snapshot.yyyymmdd* directory. It seems reproducible, in
that every time I try to make a backup it will fail the same way.
This is Solr r815830. I'm not sure if this is something that would
potentially be addressed by SOLR-1458? (That patch is from after
r815830.)
For now I'm not using any event-based backup triggers; instead I'm
manually hitting
http://master_host:port/solr/replication?command=backup
This successfully sets off a snapshot, as seen in a thread dump.
However, after a while the snapshot fails. I'll paste in a couple of
stack traces below.
I haven't seen any other evidence that my index is corrupt; in
particular, searching the index and Java-based replication seem to be
working fine. I'm currently checking the index with the Lucene
CheckIndex tool to see if it detects any problems. I'll let you know
if this turns up any corruption.
Any suggestions about what to try next? Would you recommend trying
with a build with SOLR-1458 applied?
********************
Sep 28, 2009 9:32:18 AM org.apache.solr.handler.SnapShooter createSnapshot
SEVERE: Exception while creating snapshot
java.io.FileNotFoundException: Source
'E:\tomcat\solrstuff\solr\filingcore\data\index\_y0w.fnm' does not
exist
at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
Sep 28, 2009 10:39:43 AM org.apache.solr.handler.SnapShooter createSnapshot
SEVERE: Exception while creating snapshot
java.io.FileNotFoundException: Source
'E:\tomcat\solrstuff\solr\filingcore\data\index\segments_by' does not
exist
at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
Sep 28, 2009 11:52:08 AM org.apache.solr.handler.SnapShooter createSnapshot
SEVERE: Exception while creating snapshot
java.io.FileNotFoundException: Source
'E:\tomcat\solrstuff\solr\filingcore\data\index\_yby.nrm' does not
exist
at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
Re: FileNotFoundException in Java replication handler backups
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Sep 29, 2009 at 3:19 AM, Mark Miller <ma...@gmail.com> wrote:
> Looks like a bug to me. I don't see the commit point being reserved in
> the backup code - which means its likely be removed before its done
> being copied. Gotto reserve it using the delete policy to keep around
> for the full backup duration. I'd file a JIRA issue.
>
>
Definitely a bug. Chris, please open an issue. I'll try to work up a patch.
--
Regards,
Shalin Shekhar Mangar.
Re: FileNotFoundException in Java replication handler backups
Posted by Mark Miller <ma...@gmail.com>.
Mark Miller wrote:
> Looks like a bug to me. I don't see the commit point being reserved in
> the backup code - which means its likely be removed before its done
> being copied. Gotto reserve it using the delete policy to keep around
> for the full backup duration. I'd file a JIRA issue.
>
>
>
You might wait for Noble or Yonik to respond though - perhaps that issue
did fix this, and I just am not seeing it in the code. Now that I read
the issue, its a similar problem. Possible they missed fixing it for the
backup command though - I dunno.
--
- Mark
http://www.lucidimagination.com
Re: FileNotFoundException in Java replication handler backups
Posted by Mark Miller <ma...@gmail.com>.
Looks like a bug to me. I don't see the commit point being reserved in
the backup code - which means its likely be removed before its done
being copied. Gotto reserve it using the delete policy to keep around
for the full backup duration. I'd file a JIRA issue.
--
- Mark
http://www.lucidimagination.com
Chris Harris wrote:
> Thanks to Noble Paul, I think I now understand the Java replication
> handler's backup feature. It seems to work as expected on a toy index.
> When trying it out on a copy of my production index (300GB-ish),
> though, I'm getting FileNotFoundExceptions. These cancel the backup,
> and delete the snapshot.yyyymmdd* directory. It seems reproducible, in
> that every time I try to make a backup it will fail the same way.
>
> This is Solr r815830. I'm not sure if this is something that would
> potentially be addressed by SOLR-1458? (That patch is from after
> r815830.)
>
> For now I'm not using any event-based backup triggers; instead I'm
> manually hitting
>
> http://master_host:port/solr/replication?command=backup
>
> This successfully sets off a snapshot, as seen in a thread dump.
> However, after a while the snapshot fails. I'll paste in a couple of
> stack traces below.
>
> I haven't seen any other evidence that my index is corrupt; in
> particular, searching the index and Java-based replication seem to be
> working fine. I'm currently checking the index with the Lucene
> CheckIndex tool to see if it detects any problems. I'll let you know
> if this turns up any corruption.
>
> Any suggestions about what to try next? Would you recommend trying
> with a build with SOLR-1458 applied?
>
> ********************
>
> Sep 28, 2009 9:32:18 AM org.apache.solr.handler.SnapShooter createSnapshot
> SEVERE: Exception while creating snapshot
> java.io.FileNotFoundException: Source
> 'E:\tomcat\solrstuff\solr\filingcore\data\index\_y0w.fnm' does not
> exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
> at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
> at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
> at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
>
> Sep 28, 2009 10:39:43 AM org.apache.solr.handler.SnapShooter createSnapshot
> SEVERE: Exception while creating snapshot
> java.io.FileNotFoundException: Source
> 'E:\tomcat\solrstuff\solr\filingcore\data\index\segments_by' does not
> exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
> at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
> at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
> at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
>
>
> Sep 28, 2009 11:52:08 AM org.apache.solr.handler.SnapShooter createSnapshot
> SEVERE: Exception while creating snapshot
> java.io.FileNotFoundException: Source
> 'E:\tomcat\solrstuff\solr\filingcore\data\index\_yby.nrm' does not
> exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
> at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
> at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
> at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
>