You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Sadanand Shenoy (Jira)" <ji...@apache.org> on 2023/06/30 10:29:00 UTC
[jira] [Created] (HDDS-8967) Distcp using snapshots fails due to incorrect toSnapshot name
Sadanand Shenoy created HDDS-8967:
-------------------------------------
Summary: Distcp using snapshots fails due to incorrect toSnapshot name
Key: HDDS-8967
URL: https://issues.apache.org/jira/browse/HDDS-8967
Project: Apache Ozone
Issue Type: Bug
Reporter: Sadanand Shenoy
Assignee: Sadanand Shenoy
{code:java}
$hadoop distcp -update -diff snap1 snap2 ofs://ozone1/vol2/buck2 ofs://ozone1/voldest1/buckdest1
23/06/30 07:32:22 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=true, deleteMissing=false, ignoreFailures=false, overwrite=false, append=false, useDiff=true, useRdiff=false, fromSnapshot=snap1, toSnapshot=snap2, skipCRC=false, blocking=true, numListstatusThreads=0, maxMaps=20, mapBandwidth=0.0, copyStrategy='uniformsize', preserveStatus=[], atomicWorkPath=null, logPath=null, sourceFileListing=null, sourcePaths=[ofs://ozone1/vol2/buck2], targetPath=ofs://ozone1/voldest1/buckdest1, filtersFile='null', blocksPerChunk=0, copyBufferSize=8192, verboseLog=false, directWrite=false, useiterator=false}, sourcePaths=[ofs://ozone1/vol2/buck2], targetPathExists=true, preserveRawXattrsfalse
23/06/30 07:32:22 WARN tools.DistCp: Failed to compute snapshot diff on ofs://ozone1/voldest1/buckdest1 at snapshot snap1
java.io.IOException: KEY_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException:
at org.apache.hadoop.tools.DistCpSync.getSnapshotDiffReport(DistCpSync.java:325)
at org.apache.hadoop.tools.DistCpSync.checkNoChange(DistCpSync.java:373)
at org.apache.hadoop.tools.DistCpSync.preSyncCheck(DistCpSync.java:113)
at org.apache.hadoop.tools.DistCpSync.sync(DistCpSync.java:193)
at org.apache.hadoop.tools.DistCp.prepareFileListing(DistCp.java:88)
at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:216)
at org.apache.hadoop.tools.DistCp.execute(DistCp.java:193)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:155)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:445)
Caused by: KEY_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException:
at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:713)
at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.snapshotDiff(OzoneManagerProtocolClientSideTranslatorPB.java:1240)
at org.apache.hadoop.ozone.client.rpc.RpcClient.snapshotDiff(RpcClient.java:996)
at org.apache.hadoop.ozone.client.ObjectStore.snapshotDiff(ObjectStore.java:682)
at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getSnapshotDiffReportOnceComplete(BasicRootedOzoneClientAdapterImpl.java:1363)
at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getSnapshotDiffReport(BasicRootedOzoneClientAdapterImpl.java:1332)
at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.getSnapshotDiffReport(BasicRootedOzoneFileSystem.java:1543)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.hadoop.tools.DistCpSync.getSnapshotDiffReport(DistCpSync.java:322)
... 9 more
23/06/30 07:32:22 ERROR tools.DistCp: Exception encountered {code}
The issue is that when distcp checks if target hasn't changed ,it sends an empty toSnapshot string and not "."
{code:java}
/**
* Compute the snapshot diff on the given file system. Return true if the diff
* is empty, i.e., no changes have happened in the FS.
*/
private boolean checkNoChange(FileSystem fs, Path path) {
try {
final String from = getSnapshotName(context.getFromSnapshot());
SnapshotDiffReport targetDiff = getSnapshotDiffReport(fs, path, from, "");
if (!targetDiff.getDiffList().isEmpty()) {
DistCp.LOG.warn("The target has been modified since snapshot "
+ context.getFromSnapshot());{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org