You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Raghu Angadi (JIRA)" <ji...@apache.org> on 2008/03/12 22:24:46 UTC

[jira] Created: (HADOOP-3007) DataNode pipelining : failure on mirror make results in failure on upstream datanode

DataNode pipelining : failure on mirror make results in failure on upstream datanode
------------------------------------------------------------------------------------

                 Key: HADOOP-3007
                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
             Project: Hadoop Core
          Issue Type: Bug
          Components: dfs
    Affects Versions: 0.16.0
            Reporter: Raghu Angadi
            Priority: Blocker
             Fix For: 0.16.2



When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.

I think there are two minor bugs in receiveBlock() :
# mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
# when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().

I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Comment: was deleted

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580595#action_12580595 ] 

Raghu Angadi commented on HADOOP-3007:
--------------------------------------

> client can get a fresh new set of 3 datanodes for the pipeline and start all over again.
This patch does not change anything when a client is writing data. This affect only when one datanode is transferring (re-replicating) block to more datanodes.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580998#action_12580998 ] 

Hadoop QA commented on HADOOP-3007:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
http://issues.apache.org/jira/secure/attachment/12378264/HADOOP-3007.patch
against trunk revision 619744.

    @author +1.  The patch does not contain any @author tags.

    tests included -1.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no tests are needed for this patch.

    javadoc +1.  The javadoc tool did not generate any warning messages.

    javac +1.  The applied patch does not generate any new javac compiler warnings.

    release audit +1.  The applied patch does not generate any new release audit warnings.

    findbugs +1.  The patch does not introduce any new Findbugs warnings.

    core tests +1.  The patch passed core unit tests.

    contrib tests +1.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2017/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2017/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2017/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2017/console

This message is automatically generated.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12581343#action_12581343 ] 

Hudson commented on HADOOP-3007:
--------------------------------

Integrated in Hadoop-trunk #436 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/436/])

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Description: 
When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.

I think there are two minor bugs in receiveBlock() :
# mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
# when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().

I am not sure if this should block 0.16.1, but should surely block 0.16.2.

  was:

When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.

I think there are two minor bugs in receiveBlock() :
# mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
# when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().

I am not sure if this should block 0.16.1, but should surely block 0.16.2.

        Summary: DataNode pipelining : failure on mirror results in failure on upstream datanode  (was: DataNode pipelining : failure on mirror make results in failure on upstream datanode)

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi reassigned HADOOP-3007:
------------------------------------

    Assignee: Raghu Angadi

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580616#action_12580616 ] 

dhruba borthakur commented on HADOOP-3007:
------------------------------------------

+1. Code looks good.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Attachment:     (was: HADOOP-3006.patch)

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Attachment: HADOOP-3007.patch

Attached patch does the following :

* Renames conflicting arguments for receiveBlock().
* Replaces mirrorOut's accesses with : {code}
try {
  access mirrorOut;
} catch (IOException e) {
  handleMirrorOutError(e);
} {code}
* Note that with HADOOP-1702, number of places where mirrorOut is written to goes down and reduces repeat of the above pattern.
* It does not change any other behaviour. If this goes after HADOOP-3033, the patch will be smaller since 'currentWrite' is removed there.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580593#action_12580593 ] 

dhruba borthakur commented on HADOOP-3007:
------------------------------------------

All code looks good except the first block.

This portion of code is used when the socket connection is being set up. At this point, it is better for the connection request to fail if any one mirror fails so that the client can get a fresh new set of 3 datanodes for the pipeline and start all over again.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580597#action_12580597 ] 

Raghu Angadi commented on HADOOP-3007:
--------------------------------------

Conceptually, there is not much difference between failure while connecting or while writing to mirror. In both cases, it just means the transfer did not happen to mirror.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Attachment: HADOOP-3006.patch

Patch updated with improved comments.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Attachment: HADOOP-3007.patch

Updated patch to latest 0.16 branch.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-3007) DataNode pipelining : failure on mirror results in failure on upstream datanode

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-3007:
---------------------------------

    Status: Patch Available  (was: Open)

Thanks Dhruba.

> DataNode pipelining : failure on mirror results in failure on upstream datanode
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3007
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3007.patch, HADOOP-3007.patch
>
>
> When datanode is transfering data to other datanodes (as opposed to a client write()), DN currently receiving the data is supposed to continued to receive even if the downstream (mirror) datanode fails.
> I think there are two minor bugs in receiveBlock() :
> # mirrorSock masks this.mirrorSock, which could be set to null by other members like receiveChunk() to indicate failure.
> # when this.mirrorSock is non null, failure to write to it at in receiveBlock() should be handled same way as in receiveChunk().
> I am not sure if this should block 0.16.1, but should surely block 0.16.2.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.