[jira] [Work started] (HBASE-29572) Clean up output directory when copying the manifest fails during snapshot export

["huginn (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-29572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on HBASE-29572 started by huginn.
--------------------------------------
> Clean up output directory when copying the manifest fails during snapshot export
> --------------------------------------------------------------------------------
>
>                 Key: HBASE-29572
>                 URL: https://issues.apache.org/jira/browse/HBASE-29572
>             Project: HBase
>          Issue Type: Bug
>          Components: snapshots
>    Affects Versions: 2.4.11
>            Reporter: huginn
>            Assignee: huginn
>            Priority: Minor
>              Labels: pull-request-available
>
> If an export snapshot fails in copying .snapshotInfo and manifest, which may leave behind an empty snapshot directory under .snapshot (snapshot.export.skip.tmp = true) or .snapshot/.tmp (snapshot.export.skip.tmp = false) without .snapshotInfo file and manifest.
> SnapshotFileCache periodically refreshes the cache by traversing all snapshot directories under .snapshot and reading the .snapshotInfo file and manifest to update the cache. If it can not read .snapshotInfo from a snapshot directory,  it will throw CorruptedSnapshotException.  *getSnapshotsInProgress* may also result in such a situation when it attempts to retrieve the .snapshotInfo under each snapshot directory under .snapshot/.tmp.
> So we need to clean up the output snapshot dir before throwing exception and aborting this job if copying manifest fails.
> {code:java}
> cache = new SnapshotFileCache(fs, rootDir, workingFs, workingDir, cacheRefreshPeriod,
>         cacheRefreshPeriod, "snapshot-hfile-cleaner-cache-refresher",
>         new SnapshotFileCache.SnapshotFileInspector() {
>           @Override
>           public Collection<String> filesUnderSnapshot(final FileSystem fs, final Path snapshotDir)
>             throws IOException {
>             return SnapshotReferenceUtil.getHFileNames(conf, fs, snapshotDir);
>           }
> });
> public static Set<String> getHFileNames(final Configuration conf, final FileSystem fs,
>     final Path snapshotDir) throws IOException {
>     SnapshotDescription desc = SnapshotDescriptionUtils.readSnapshotInfo(fs, snapshotDir);
>     return getHFileNames(conf, fs, snapshotDir, desc);
> }
>   public static SnapshotDescription readSnapshotInfo(FileSystem fs, Path snapshotDir)
>     throws CorruptedSnapshotException {
>     Path snapshotInfo = new Path(snapshotDir, SNAPSHOTINFO_FILE);
>     try (FSDataInputStream in = fs.open(snapshotInfo)) {
>       return SnapshotDescription.parseFrom(in);
>     } catch (IOException e) {
>       throw new CorruptedSnapshotException("Couldn't read snapshot info from:" + snapshotInfo, e);
>     }
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)