[jira] Created: (HDFS-1309) FileSystem.rename will fail silently

["Kris Nuttycombe (JIRA)" <ji...@apache.org>]

FileSystem.rename will fail silently
------------------------------------

                 Key: HDFS-1309
                 URL: https://issues.apache.org/jira/browse/HDFS-1309
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: hdfs client
    Affects Versions: 0.20.2
         Environment: Linux version 2.6.31-302-ec2 (buildd@yellow) (gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu7) ) #7-Ubuntu SMP Tue Oct 13 19:55:22 UTC 2009

            Reporter: Kris Nuttycombe
            Priority: Minor


Some filesystem operations (such as rename) will fail silently. In the attached example, a failure message will be written to the hadoop log, but it would be much better if the operation were to "fail fast" by throwing a checked exception and forcing the caller to handle the problem; failing to do so can easily lead to inadvertent data corruption.

  val coalesceBasePath = new Path(eventLog.basePath, coalesceTo)
  val backupBasePath = new Path(eventLog.basePath, relocateTo)

    eventLog.fs.mkdirs(backupBasePath)
    for (path <- coalesced; time <- HDFSEventLog.timePart(path, eventType)) {
      val backupPath = HDFSEventLog.path(backupBasePath, eventType, time)
      log.info("Relocating " + path + " to " + backupPath)
      eventLog.fs.rename(path, backupPath)
    }


INF [20100715-16:11:20.727] reporting: Relocating hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279226067707 to hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279226067707
INF [20100715-16:11:20.752] reporting: Relocating hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279226077707 to hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279226077707
INF [20100715-16:11:20.754] reporting: Relocating hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279226457707 to hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279226457707
INF [20100715-16:11:20.757] reporting: Relocating hdfs://localhost:9000/test-batchEventLog/metrics/metrics_1279229126727 to hdfs://localhost:9000/test-batchEventLog/pre-coalesce/metrics/metrics_1279229126727
Complete.
[knuttycombe@floorshow reporting (reporting-coalesce)]$ hadoop fs -ls hdfs://localhost:9000/test-batchEventLog/
Found 3 items
drwxr-xr-x   - knuttycombe supergroup          0 2010-07-15 14:54 /test-batchEventLog/coalesced
drwxr-xr-x   - knuttycombe supergroup          0 2010-07-15 14:35 /test-batchEventLog/metrics
drwxr-xr-x   - knuttycombe supergroup          0 2010-07-15 16:11 /test-batchEventLog/pre-coalesce
[knuttycombe@floorshow reporting (reporting-coalesce)]$ hadoop fs -ls hdfs://localhost:9000/test-batchEventLog/metrics
Found 4 items
-rw-r--r--   3 knuttycombe supergroup    2017122 2010-07-15 14:34 /test-batchEventLog/metrics/metrics_1279226067707
-rw-r--r--   3 knuttycombe supergroup    4122951 2010-07-15 14:34 /test-batchEventLog/metrics/metrics_1279226077707
-rw-r--r--   3 knuttycombe supergroup    5999912 2010-07-15 14:35 /test-batchEventLog/metrics/metrics_1279226457707
-rw-r--r--   3 knuttycombe supergroup    8638301 2010-07-15 14:26 /test-batchEventLog/metrics/metrics_1279229126727
[knuttycombe@floorshow reporting (reporting-coalesce)]$ hadoop fs -ls hdfs://localhost:9000/test-batchEventLog/pre-coalesce
[knuttycombe@floorshow reporting (reporting-coalesce)]$

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.