You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jesse Yates (JIRA)" <ji...@apache.org> on 2012/11/29 22:16:58 UTC
[jira] [Created] (HBASE-7240) Cleanup old snapshots on start
Jesse Yates created HBASE-7240:
----------------------------------
Summary: Cleanup old snapshots on start
Key: HBASE-7240
URL: https://issues.apache.org/jira/browse/HBASE-7240
Project: HBase
Issue Type: Sub-task
Affects Versions: hbase-6055
Reporter: Jesse Yates
Fix For: hbase-6055
If the master is hard stopped (i.e. kill -9), the snapshot handler or SnapshotManager may not have a chance to cleanup after the snapshot, leaving extraneous files in the working snapshot directory (/hbase/.snapshot/.tmp directory).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7240) Cleanup old snapshots on start
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-7240:
-------------------------------
Attachment: hbase-7240.patch
Patch for this. Adds a start method to SnapshotManager that just does cleanup.
Didn't add a unit test since it required spinnning up a minicluster (or down and then up in an exisiting test) for just checking a couple line change that can be verified on sight easily.
> Cleanup old snapshots on start
> ------------------------------
>
> Key: HBASE-7240
> URL: https://issues.apache.org/jira/browse/HBASE-7240
> Project: HBase
> Issue Type: Sub-task
> Components: Client, master, regionserver, snapshots, Zookeeper
> Affects Versions: hbase-6055
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Fix For: hbase-6055
>
> Attachments: hbase-7240.patch
>
>
> If the master is hard stopped (i.e. kill -9), the snapshot handler or SnapshotManager may not have a chance to cleanup after the snapshot, leaving extraneous files in the working snapshot directory (/hbase/.snapshot/.tmp directory).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7240) Cleanup old snapshots on start
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506992#comment-13506992 ]
Jesse Yates commented on HBASE-7240:
------------------------------------
It would be a delete of the /hbase/snapshot/.tmp directory. Might change it we continue running snapshots across master failure sometime in the future, but for the moment, its just a single RPC.
> Cleanup old snapshots on start
> ------------------------------
>
> Key: HBASE-7240
> URL: https://issues.apache.org/jira/browse/HBASE-7240
> Project: HBase
> Issue Type: Sub-task
> Components: Client, master, regionserver, snapshots, Zookeeper
> Affects Versions: hbase-6055
> Reporter: Jesse Yates
> Fix For: hbase-6055
>
>
> If the master is hard stopped (i.e. kill -9), the snapshot handler or SnapshotManager may not have a chance to cleanup after the snapshot, leaving extraneous files in the working snapshot directory (/hbase/.snapshot/.tmp directory).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7240) Cleanup old snapshots on start
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506808#comment-13506808 ]
Jesse Yates commented on HBASE-7240:
------------------------------------
We could create a background chore that periodically checks this directory against the running snapshots, but its likely a rare occurrence that we fail a snapshot and can't cleanup after it. We probably just need to add a little cleanup mechanism to on startup (could just drop this into the SnapshotManager as it also plays well into a possible future goal of recovering snapshot attempts between master failures).
> Cleanup old snapshots on start
> ------------------------------
>
> Key: HBASE-7240
> URL: https://issues.apache.org/jira/browse/HBASE-7240
> Project: HBase
> Issue Type: Sub-task
> Components: Client, master, regionserver, snapshots, Zookeeper
> Affects Versions: hbase-6055
> Reporter: Jesse Yates
> Fix For: hbase-6055
>
>
> If the master is hard stopped (i.e. kill -9), the snapshot handler or SnapshotManager may not have a chance to cleanup after the snapshot, leaving extraneous files in the working snapshot directory (/hbase/.snapshot/.tmp directory).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7240) Cleanup old snapshots on start
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506989#comment-13506989 ]
Ted Yu commented on HBASE-7240:
-------------------------------
Cleanup at startup would be completed quickly, right ?
I assume this wouldn't affect total recovery time much.
> Cleanup old snapshots on start
> ------------------------------
>
> Key: HBASE-7240
> URL: https://issues.apache.org/jira/browse/HBASE-7240
> Project: HBase
> Issue Type: Sub-task
> Components: Client, master, regionserver, snapshots, Zookeeper
> Affects Versions: hbase-6055
> Reporter: Jesse Yates
> Fix For: hbase-6055
>
>
> If the master is hard stopped (i.e. kill -9), the snapshot handler or SnapshotManager may not have a chance to cleanup after the snapshot, leaving extraneous files in the working snapshot directory (/hbase/.snapshot/.tmp directory).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7240) Cleanup old snapshots on start
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-7240:
-------------------------------
Assignee: Jesse Yates
> Cleanup old snapshots on start
> ------------------------------
>
> Key: HBASE-7240
> URL: https://issues.apache.org/jira/browse/HBASE-7240
> Project: HBase
> Issue Type: Sub-task
> Components: Client, master, regionserver, snapshots, Zookeeper
> Affects Versions: hbase-6055
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Fix For: hbase-6055
>
>
> If the master is hard stopped (i.e. kill -9), the snapshot handler or SnapshotManager may not have a chance to cleanup after the snapshot, leaving extraneous files in the working snapshot directory (/hbase/.snapshot/.tmp directory).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7240) Cleanup old snapshots on start
Posted by "Matteo Bertozzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506939#comment-13506939 ]
Matteo Bertozzi commented on HBASE-7240:
----------------------------------------
+1 on just have the cleanup at startup
> Cleanup old snapshots on start
> ------------------------------
>
> Key: HBASE-7240
> URL: https://issues.apache.org/jira/browse/HBASE-7240
> Project: HBase
> Issue Type: Sub-task
> Components: Client, master, regionserver, snapshots, Zookeeper
> Affects Versions: hbase-6055
> Reporter: Jesse Yates
> Fix For: hbase-6055
>
>
> If the master is hard stopped (i.e. kill -9), the snapshot handler or SnapshotManager may not have a chance to cleanup after the snapshot, leaving extraneous files in the working snapshot directory (/hbase/.snapshot/.tmp directory).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira