[jira] [Created] (HADOOP-11556) SwiftNativeFileSystem rename being interrupted during "fs -put"

["Lei (Eddy) Xu (JIRA)" <ji...@apache.org>]

Lei (Eddy) Xu created HADOOP-11556:
--------------------------------------

             Summary: SwiftNativeFileSystem rename being interrupted during "fs -put"
                 Key: HADOOP-11556
                 URL: https://issues.apache.org/jira/browse/HADOOP-11556
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs, fs/swift
    Affects Versions: 2.6.0
            Reporter: Lei (Eddy) Xu


When running {{hadoop fs -put  bar  swift://... }},  {{FsShell}} writes to {{bar._COPYING_}} first, and then renames it to "bar". 

However, it fails when the file is partitioned uploaded. 

{code}
15/02/06 21:00:49 DEBUG snative.SwiftNativeFileSystemStore: Source file appears to be partitioned. copying file and deleting children
15/02/06 21:00:49 DEBUG params.DefaultHttpParams: Set parameter http.method.retry-handler = org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@26bf4e60
15/02/06 21:00:49 DEBUG params.DefaultHttpParams: Set parameter http.connection.timeout = 15000
15/02/06 21:00:49 DEBUG params.DefaultHttpParams: Set parameter http.socket.timeout = 60000
15/02/06 21:00:49 DEBUG http.SwiftRestClient: COPY https://snet-storage101.iad3.clouddrive.com/v1/MossoCloudFS_946265/foo/bar._COPYING_
X-Auth-Token: ...
Destination: foo/bar
User-Agent: Apache Hadoop Swift Client 3.0.0-SNAPSHOT from ... by root source checksum ...

15/02/06 21:00:49 DEBUG httpclient.HttpConnection: Open connection to snet-storage101.iad3.clouddrive.com:443
15/02/06 21:00:49 DEBUG wire.header: >> "COPY /v1/MossoCloudFS_946265/foo/bar._COPYING_ HTTP/1.1[\r][\n]"
15/02/06 21:00:49 DEBUG httpclient.HttpMethodBase: Adding Host request header
15/02/06 21:00:49 DEBUG wire.header: >> "X-Auth-Token: ...[\r][\n]"
15/02/06 21:00:49 DEBUG wire.header: >> "Destination: foo/bar[\r][\n]"
15/02/06 21:00:49 DEBUG wire.header: >> "User-Agent: Apache Hadoop Swift Client 3.0.0-SNAPSHOT from ... by root source checksum ...[\r][\n]"
15/02/06 21:00:49 DEBUG wire.header: >> "Host: snet-storage101.iad3.clouddrive.com[\r][\n]"
15/02/06 21:00:49 DEBUG wire.header: >> "Content-Length: 0[\r][\n]"
15/02/06 21:00:49 DEBUG wire.header: >> "[\r][\n]"
15/02/06 21:00:49 DEBUG methods.EntityEnclosingMethod: Request body has not been specified
15/02/06 21:01:49 DEBUG httpclient.HttpMethodDirector: Closing the connection.
15/02/06 21:01:49 DEBUG httpclient.HttpMethodDirector: Method retry handler returned false. Automatic recovery will not be attempted
15/02/06 21:01:49 DEBUG httpclient.HttpConnection: Releasing connection back to connection manager.
15/02/06 21:01:49 DEBUG httpclient.HttpConnection: Releasing connection back to connection manager.
{code}

The {{bar}} file eventually shows up in Swift. However, the {{bar._COPYING_}} directories and the manifest file are not deleted.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)