You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Sadanand Shenoy (Jira)" <ji...@apache.org> on 2023/06/30 15:06:00 UTC
[jira] [Resolved] (HDDS-8967) Distcp using snapshots fails due to incorrect toSnapshot name
[ https://issues.apache.org/jira/browse/HDDS-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sadanand Shenoy resolved HDDS-8967.
-----------------------------------
Resolution: Duplicate
Duplicates HDDS-8968
> Distcp using snapshots fails due to incorrect toSnapshot name
> -------------------------------------------------------------
>
> Key: HDDS-8967
> URL: https://issues.apache.org/jira/browse/HDDS-8967
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Sadanand Shenoy
> Assignee: Sadanand Shenoy
> Priority: Major
>
> {code:java}
> $hadoop distcp -update -diff snap1 snap2 ofs://ozone1/vol2/buck2 ofs://ozone1/voldest1/buckdest1
> 23/06/30 07:32:22 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=true, deleteMissing=false, ignoreFailures=false, overwrite=false, append=false, useDiff=true, useRdiff=false, fromSnapshot=snap1, toSnapshot=snap2, skipCRC=false, blocking=true, numListstatusThreads=0, maxMaps=20, mapBandwidth=0.0, copyStrategy='uniformsize', preserveStatus=[], atomicWorkPath=null, logPath=null, sourceFileListing=null, sourcePaths=[ofs://ozone1/vol2/buck2], targetPath=ofs://ozone1/voldest1/buckdest1, filtersFile='null', blocksPerChunk=0, copyBufferSize=8192, verboseLog=false, directWrite=false, useiterator=false}, sourcePaths=[ofs://ozone1/vol2/buck2], targetPathExists=true, preserveRawXattrsfalse
> 23/06/30 07:32:22 WARN tools.DistCp: Failed to compute snapshot diff on ofs://ozone1/voldest1/buckdest1 at snapshot snap1
> java.io.IOException: KEY_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException:
> at org.apache.hadoop.tools.DistCpSync.getSnapshotDiffReport(DistCpSync.java:325)
> at org.apache.hadoop.tools.DistCpSync.checkNoChange(DistCpSync.java:373)
> at org.apache.hadoop.tools.DistCpSync.preSyncCheck(DistCpSync.java:113)
> at org.apache.hadoop.tools.DistCpSync.sync(DistCpSync.java:193)
> at org.apache.hadoop.tools.DistCp.prepareFileListing(DistCp.java:88)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:216)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:193)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:155)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:445)
> Caused by: KEY_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException:
> at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:713)
> at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.snapshotDiff(OzoneManagerProtocolClientSideTranslatorPB.java:1240)
> at org.apache.hadoop.ozone.client.rpc.RpcClient.snapshotDiff(RpcClient.java:996)
> at org.apache.hadoop.ozone.client.ObjectStore.snapshotDiff(ObjectStore.java:682)
> at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getSnapshotDiffReportOnceComplete(BasicRootedOzoneClientAdapterImpl.java:1363)
> at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getSnapshotDiffReport(BasicRootedOzoneClientAdapterImpl.java:1332)
> at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.getSnapshotDiffReport(BasicRootedOzoneFileSystem.java:1543)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.apache.hadoop.tools.DistCpSync.getSnapshotDiffReport(DistCpSync.java:322)
> ... 9 more
> 23/06/30 07:32:22 ERROR tools.DistCp: Exception encountered {code}
> The issue is that when distcp checks if target hasn't changed ,it sends an empty toSnapshot string and not "."
> {code:java}
> /**
> * Compute the snapshot diff on the given file system. Return true if the diff
> * is empty, i.e., no changes have happened in the FS.
> */
> private boolean checkNoChange(FileSystem fs, Path path) {
> try {
> final String from = getSnapshotName(context.getFromSnapshot());
> SnapshotDiffReport targetDiff = getSnapshotDiffReport(fs, path, from, "");
> if (!targetDiff.getDiffList().isEmpty()) {
> DistCp.LOG.warn("The target has been modified since snapshot "
> + context.getFromSnapshot());{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org