You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2008/09/18 02:05:44 UTC
[jira] Created: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
DFSClient should do lease recovery using data transfer port.
------------------------------------------------------------
Key: HADOOP-4198
URL: https://issues.apache.org/jira/browse/HADOOP-4198
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.18.0
Reporter: Konstantin Shvachko
Assignee: Tsz Wo (Nicholas), SZE
Priority: Blocker
Fix For: 0.18.1
HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
Lease recovery can be initialized by a name-node or by a client.
The problem was reported if lease recovery is initialized by a client running on an untrusted host.
The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4198:
-------------------------------------------
Release Note:
Changed DFSClient to use data transfer port for lease recovery.
Removed ClientDatanodeProtocol.
Hadoop Flags: [Incompatible change]
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.1
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-4198:
--------------------------------
Fix Version/s: (was: 0.18.1)
0.18.2
0.18.1 is being released. Moving this to 0.18.2.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4198) DFSClient should do
lease recovery using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632069#action_12632069 ]
szetszwo edited comment on HADOOP-4198 at 9/18/08 11:00 AM:
--------------------------------------------------------------------------
{noformat}
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
{noformat}
No new tests added since there are already a lot of tests for lease recovery and file creation.
was (Author: szetszwo):
{noformat}
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
{noformat}
No new tests added since there are already a lot test for lease recovery and file creation.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4198:
-------------------------------------------
Attachment: 4198_20080917.patch
4198_20080917.patch: get ride of ClientDatanodeProtocol
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.1
>
> Attachments: 4198_20080917.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4198:
-------------------------------------------
Attachment: 4198_20080917b.patch
4198_20080917b.patch: moved Datanode recoverBlock implementation to DataXceiver.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.1
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632333#action_12632333 ]
Konstantin Shvachko commented on HADOOP-4198:
---------------------------------------------
+1 Looks good.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632397#action_12632397 ]
Konstantin Shvachko commented on HADOOP-4198:
---------------------------------------------
We can set DATA_TRANSFER_VERSION to 14 in 0.18 and to 15 in 0.19. So they will be always incompatible as desired.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch, 4198_20080917b_0.18.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-4198:
------------------------------------
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
No longer seems to serve a useful purpose.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch, 4198_20080917b_0.18.patch, 4198_20080918.patch, 4198_20080918_0.18.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632042#action_12632042 ]
Konstantin Shvachko commented on HADOOP-4198:
---------------------------------------------
You still have the implementation of ClientDatanodeProtocol.recoverBlock in the DataNode class.
I think it makes sense to remove and move the logic (which is just print a log massage) to the other DataNode.recoverBlock() method.
This should make things simpler.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.1
>
> Attachments: 4198_20080917.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636746#action_12636746 ]
dhruba borthakur commented on HADOOP-4198:
------------------------------------------
Can somebody please explain why we do not need this patch anymore?
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch, 4198_20080917b_0.18.patch, 4198_20080918.patch, 4198_20080918_0.18.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4198:
-------------------------------------------
Attachment: 4198_20080918_0.18.patch
4198_20080918_0.18.patch: changed DATA_TRANSFER_VERSION from 11 to 14
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch, 4198_20080917b_0.18.patch, 4198_20080918.patch, 4198_20080918_0.18.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4198) DFSClient should do
lease recovery using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632034#action_12632034 ]
szetszwo edited comment on HADOOP-4198 at 9/17/08 5:23 PM:
-------------------------------------------------------------------------
4198_20080917.patch: get rid of ClientDatanodeProtocol
was (Author: szetszwo):
4198_20080917.patch: get ride of ClientDatanodeProtocol
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.1
>
> Attachments: 4198_20080917.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4198:
-------------------------------------------
Attachment: 4198_20080918.patch
4198_20080918.patch: changed DATA_TRANSFER_VERSION from 13 to 15
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch, 4198_20080917b_0.18.patch, 4198_20080918.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4198:
-------------------------------------------
Status: Patch Available (was: Open)
{noformat}
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
{noformat}
No new tests added since there are already a lot test for lease recovery and file creation.
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.1
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4198) DFSClient should do lease recovery
using data transfer port.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4198:
-------------------------------------------
Attachment: 4198_20080917b_0.18.patch
4198_20080917b_0.18.patch: patch for 0.18.
Note that *DATA_TRANSFER_VERSION* in trunk is changed from 13 to 14 but the one in 0.18 is changed from 11 to 12. Or should it change DATA_TRANSFER_VERSION from 11 to 14?
> DFSClient should do lease recovery using data transfer port.
> ------------------------------------------------------------
>
> Key: HADOOP-4198
> URL: https://issues.apache.org/jira/browse/HADOOP-4198
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: 4198_20080917.patch, 4198_20080917b.patch, 4198_20080917b_0.18.patch
>
>
> HADOOP-3283 introduced {{dfs.datanode.ipc.address}} which defines where the data-node rpc server runs.
> The rpc server on a data-node is used only for lease recovery (HADOOP-3310).
> Lease recovery can be initialized by a name-node or by a client.
> The problem was reported if lease recovery is initialized by a client running on an untrusted host.
> The port that the http server runs on is closed for the outside use and therefore lease recovery fails.
> Production level security model assumes that data-nodes are run on trusted nodes and therefore it is safe to have ports open for inter data-node communication.
> HDFS clients can run on arbitrary nodes and according to the security model can access only the ports that are externally open.
> We propose to use the standard data node port for lease recovery, which means that lease recovery will use {{DataXceiver}} and data transfer protocol rather than {{ClientDatanodeProtocol}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.