[jira] [Commented] (HADOOP-17015) ABFS: Make PUT and POST operations idempotent

["Steve Loughran (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162756#comment-17162756 ] 

Steve Loughran commented on HADOOP-17015:
-----------------------------------------

the key uses of rename that I know of are renaming a task's file/dir to a known destination -the dest is the conflict point, rather than the source (FileOutputCommitter), but HBase also depends on rename semantics. It's why it switches to ZK (HBoss) for deployment in cloud infra where rename doesn't do atomic

> ABFS: Make PUT and POST operations idempotent
> ---------------------------------------------
>
>                 Key: HADOOP-17015
>                 URL: https://issues.apache.org/jira/browse/HADOOP-17015
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 3.2.1
>            Reporter: Sneha Vijayarajan
>            Assignee: Sneha Vijayarajan
>            Priority: Major
>             Fix For: 3.4.0
>
>
> Currently when a PUT or POST operation timeouts and the server has already successfully executed the operation, there is no check in driver to see if the operation did succeed or not and just retries the same operation again. This can cause driver to through invalid user errors.
>  
> Sample scenario:
>  # Rename request times out. Though server has successfully executed the operation.
>  # Driver retries rename and get source not found error.
> In the scenario, driver needs to check if rename is being retried and success if source if not found, but destination is present.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org