[jira] Commented: (SOLR-1475) Java-based replication doesn't properly reserve its commit point during backups

["Yonik Seeley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760747#action_12760747 ] 

Yonik Seeley commented on SOLR-1475:
------------------------------------

Here are the various ways I've thought of doing it:
1. write your own copy that interleaves short term reservations
2. asynchronously (in a separate thread) make short term reservations while a different thread is actually doing copyFiles()
3. do a super-long reservation before invoking copyFiles() and then set it back to a short reservation after done (the problem being that if someone else made a super long reservation, it would be lost).

On a quick peek, it looks like you opted for #1

One thought on efficiency is to try and use FileChannel.transferTo() to potentially skip the user-space copy altogether.  We'd probably want to use relatively large block sizes too - 4 to 8MB perhaps?

> Java-based replication doesn't properly reserve its commit point during backups
> -------------------------------------------------------------------------------
>
>                 Key: SOLR-1475
>                 URL: https://issues.apache.org/jira/browse/SOLR-1475
>             Project: Solr
>          Issue Type: Bug
>          Components: replication (java)
>    Affects Versions: 1.4
>            Reporter: Chris Harris
>             Fix For: 1.4
>
>         Attachments: SOLR-1475.patch
>
>
> The issue title reflects Mark Miller's initial diagnosis of the problem.
> Here are my symptoms:
> This is regarding the backup feature of replication, as opposed to replication. Backups seem to work fine on toy indexes. When trying backups out on a copy of my production index (300GB-ish), though, I'm getting FileNotFoundExceptions. These cancel the backup, and delete the snapshot.yyyymmdd* directory. It seems reproducible, in that every time I try to make a backup of my large index it will fail the same way.
> This is Solr r815830. I'm not sure if this is something that would potentially be addressed by SOLR-1458? (That patch is from after r815830.)
> For now I'm not using any event-based backup triggers; instead I'm manually hitting
> http://master_host:port/solr/replication?command=backup
> This successfully sets off a snapshot, as seen in a thread dump.  However, after a while the snapshot fails. I'll paste in a couple of stack traces below.
> I haven't seen any other evidence that my index is corrupt; in particular, searching the index and Java-based replication seem to be working fine, and the Lucene CheckIndex tool did not report any problems with the index.
> ********************
> {code}
> Sep 28, 2009 9:32:18 AM org.apache.solr.handler.SnapShooter createSnapshot
> SEVERE: Exception while creating snapshot
> java.io.FileNotFoundException: Source
> 'E:\tomcat\solrstuff\solr\filingcore\data\index\_y0w.fnm' does not
> exist
>        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
>        at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
>        at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
>        at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
> Sep 28, 2009 10:39:43 AM org.apache.solr.handler.SnapShooter createSnapshot
> SEVERE: Exception while creating snapshot
> java.io.FileNotFoundException: Source
> 'E:\tomcat\solrstuff\solr\filingcore\data\index\segments_by' does not
> exist
>        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
>        at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
>        at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
>        at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
> Sep 28, 2009 11:52:08 AM org.apache.solr.handler.SnapShooter createSnapshot
> SEVERE: Exception while creating snapshot
> java.io.FileNotFoundException: Source
> 'E:\tomcat\solrstuff\solr\filingcore\data\index\_yby.nrm' does not
> exist
>        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:637)
>        at org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:587)
>        at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:83)
>        at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:61)
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.