You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2011/05/20 00:23:47 UTC
[jira] [Created] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Encapsulate individual DataTransferProtocol op header
-----------------------------------------------------
Key: HDFS-1966
URL: https://issues.apache.org/jira/browse/HDFS-1966
Project: Hadoop HDFS
Issue Type: Sub-task
Components: data-node, hdfs client
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1966:
-----------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.0
Release Note: Added header classes for individual DataTransferProtocol op headers.
Hadoop Flags: [Incompatible change, Reviewed]
Status: Resolved (was: Patch Available)
I have committed this.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.23.0
>
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1966:
-----------------------------------------
Attachment: h1966_20110519.patch
h1966_20110519.patch: added {{CopyBlockHeader}} for illustrating the idea.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040422#comment-13040422 ]
Hadoop QA commented on HDFS-1966:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12480686/h1966_20110527b.patch
against trunk revision 1128393.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/651//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/651//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/651//console
This message is automatically generated.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042918#comment-13042918 ]
Hudson commented on HDFS-1966:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #704 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/704/])
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.23.0
>
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1966:
-----------------------------------------
Attachment: h1966_20110527b.patch
h1966_20110527b.patch: changed all methods.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Jitendra Nath Pandey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042530#comment-13042530 ]
Jitendra Nath Pandey commented on HDFS-1966:
--------------------------------------------
I think DataTransferProtocol is getting too cluttered and it might be worthwhile to split it into several classes and interfaces. But that is beyond the scope of this jira.
+1 for the patch.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042741#comment-13042741 ]
Hudson commented on HDFS-1966:
------------------------------
Integrated in Hadoop-Hdfs-trunk #685 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/685/])
HDFS-1966. Encapsulate individual DataTransferProtocol op headers.
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1130367
Files :
* /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/protocol/DataTransferProtocol.java
* /hadoop/hdfs/trunk/CHANGES.txt
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.23.0
>
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1966:
-----------------------------------------
Attachment: h1966_20110526.patch
h1966_20110526.patch: reverted some other changes.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040895#comment-13040895 ]
Todd Lipcon commented on HDFS-1966:
-----------------------------------
looks like TestFiDataTransferProtocol2 failed. is that due to this patch or just flaky?
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1966:
-----------------------------------------
Status: Patch Available (was: Open)
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1966:
-----------------------------------------
Attachment: h1966_20110524.patch
h1966_20110524.patch: reverted some changes.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Jitendra Nath Pandey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042493#comment-13042493 ]
Jitendra Nath Pandey commented on HDFS-1966:
--------------------------------------------
1. What is the reason for defining header classes inside the Op enum?
2. I will recommend adding a factory to create right header object depending on the opcode. The factory could be useful at the receiving end.
3. Please add a few unit tests for serialization/de-serialization of the headers.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040908#comment-13040908 ]
Tsz Wo (Nicholas), SZE commented on HDFS-1966:
----------------------------------------------
TestFiDataTransferProtocol2 is flaky. It is nothing to do with the patch. See HDFS-1923.
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1966) Encapsulate individual
DataTransferProtocol op header
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042513#comment-13042513 ]
Tsz Wo (Nicholas), SZE commented on HDFS-1966:
----------------------------------------------
Jitendra, thanks for the review.
> 1. What is the reason for defining header classes inside the Op enum?
It is because the headers are operation related. There are other classes like {{PacketHeader}} which is nothing to do with operations.
> 2. I will recommend adding a factory to create right header object depending on the opcode. The factory could be useful at the receiving end.
We already have {{DataTransferProtocol.Receiver}}. I think it is the factory you mean.
> 3. Please add a few unit tests for serialization/de-serialization of the headers.
We have many tests for read, write, fault-inject tests, balancer, etc. These tests cover {{DataTransferProtocol}}. So adding new tests for the header seems redundant. Do you agree?
> Encapsulate individual DataTransferProtocol op header
> -----------------------------------------------------
>
> Key: HDFS-1966
> URL: https://issues.apache.org/jira/browse/HDFS-1966
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: data-node, hdfs client
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1966_20110519.patch, h1966_20110524.patch, h1966_20110526.patch, h1966_20110527b.patch
>
>
> It will make a clear distinction between the variables used in the protocol and the others.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira