You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2007/09/18 02:03:44 UTC
[jira] Created: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
----------------------------------------------------------------------------------------------------------------------------------
Key: HADOOP-1914
URL: https://issues.apache.org/jira/browse/HADOOP-1914
Project: Hadoop
Issue Type: New Feature
Reporter: Hairong Kuang
Assignee: Hairong Kuang
For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
BlockLocations[] getBlocks(DatanodeID datanode, long size);
This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529587 ]
dhruba borthakur commented on HADOOP-1914:
------------------------------------------
Code look good. One comment: in the current code, the RPC layer has to serialize a list of BlockLocations. Instead, if we create a new object that has an array of BlockLocations inside it, and use this new object in the RPC, then we might be able to avoid redundant bytes being serialized by the RPC layer.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530804 ]
dhruba borthakur commented on HADOOP-1914:
------------------------------------------
+1. Code looks good.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Status: Patch Available (was: Open)
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Attachment: namenodeProtocol.patch
The patch includes:
1. First version of NamenodeProtocol that have one API, getBlocks.
2. Definition of the return value of getBlocks: BlockLocations
3. Change of Namenode & FSNamesystem to support getBlocks.
4. A JUnit test to test getBlocks.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Attachment: namenodeProtocol2.patch
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12528568 ]
Hairong Kuang commented on HADOOP-1914:
---------------------------------------
Yes I agree. It's quite simple to add these two APIs to NamenodeProtocol. But it's not trival to change SecondaryNamenode to use NamenodeProtocol. Do you mind that I create a seprate jira on this?
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531945 ]
Hudson commented on HADOOP-1914:
--------------------------------
Integrated in Hadoop-Nightly #258 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/258/])
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch, namenodeProtocol3.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531627 ]
Hadoop QA commented on HADOOP-1914:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12366861/namenodeProtocol2.patch
against trunk revision r581028.
@author +1. The patch does not contain any @author tags.
javadoc -1. The javadoc tool appears to have generated messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/859/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/859/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/859/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/859/console
This message is automatically generated.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1914:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Hairong!
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch, namenodeProtocol3.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531648 ]
Hadoop QA commented on HADOOP-1914:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12366866/namenodeProtocol3.patch
against trunk revision r581047.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/860/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/860/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/860/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/860/console
This message is automatically generated.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch, namenodeProtocol3.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Status: Open (was: Patch Available)
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Status: Open (was: Patch Available)
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch, namenodeProtocol3.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531088 ]
Hadoop QA commented on HADOOP-1914:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12366503/namenodeProtocol1.patch
against trunk revision r580166.
@author +1. The patch does not contain any @author tags.
javadoc -1. The javadoc tool appears to have generated messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/842/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/842/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/842/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/842/console
This message is automatically generated.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Status: Patch Available (was: Open)
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Component/s: dfs
Fix Version/s: 0.15.0
Affects Version/s: 0.14.1
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Status: Patch Available (was: Open)
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch, namenodeProtocol3.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Attachment: namenodeProtocol3.patch
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch, namenodeProtocol2.patch, namenodeProtocol3.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12528272 ]
dhruba borthakur commented on HADOOP-1914:
------------------------------------------
It might be worthwhile to move two existing API: rollFsEdit and rollFsImage from the client protocol to the NameNode procotol. These two APIs are used by the Secondary Namenode to cause periodic checkpointing.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1914) HDFS should have a NamenodeProtocol
to allow secondary namenodes and rebalancing processes to communicate with
a primary namenode
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-1914:
----------------------------------
Attachment: namenodeProtocol1.patch
This patch includes a class BlocksWithLocations that implements an array of BlockWithLocations. The class provides its own serialization/deserialization methods which are more efficient than the default RPC array (de)serialization methods.
> HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1914
> URL: https://issues.apache.org/jira/browse/HADOOP-1914
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: namenodeProtocol.patch, namenodeProtocol1.patch
>
>
> For the security purpose, it is nice to have a NamenodeProtocol to allow secondary namenodes and rebalacing processes to communicate with a primary namenode. In the first version the NamenodeProtocol is going to support one RPC:
> BlockLocations[] getBlocks(DatanodeID datanode, long size);
> This RPC allows a rebalancing process to fetch a partial blocks map from a namenode. It returns a list of blocks on the given datanode and its locations, whose total size is the given size.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.