You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Ivan Kelly (JIRA)" <ji...@apache.org> on 2012/08/16 17:51:38 UTC

[jira] [Created] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Ivan Kelly created HDFS-3809:
--------------------------------

             Summary: Make BKJM use protobufs for all serialization with ZK
                 Key: HDFS-3809
                 URL: https://issues.apache.org/jira/browse/HDFS-3809
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Ivan Kelly
            Assignee: Ivan Kelly
             Fix For: 3.0.0


HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated HDFS-3809:
--------------------------------------

    Fix Version/s: 3.0.0
     Hadoop Flags: Reviewed
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0
>
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449906#comment-13449906 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

Branch-2 merge is not trivial merge as some changes for BookKeeperJournalManager went only into trunk.
We may have to backport it separately.

@Ivan, do you want to take a stab at it?

Couple of JIRAs went into trunk, they are not in Branch-2 are HDFS-3573, HDFS-3695 etc.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Joseph Evans reopened HDFS-3809:
---------------------------------------


Branch-2 is failing with 
{noformat}
main:
     [exec] bkjournal.proto:30:12: "NamespaceInfoProto" is not defined.
{noformat}

after this was merged in.  Please either fix it or revert the change.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated HDFS-3809:
--------------------------------------

          Resolution: Fixed
    Target Version/s: 2.0.1-alpha, 3.0.0  (was: 3.0.0, 2.0.1-alpha)
              Status: Resolved  (was: Patch Available)

[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [1.784s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [1.483s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [1.712s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [0.520s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.204s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [2.090s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [0.957s]
[INFO] Apache Hadoop Common .............................. SUCCESS [47.699s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.083s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [1:03.694
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [8.764s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [5.288s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.137s]
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449890#comment-13449890 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Hdfs-trunk-Commit #2757 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2757/])
    Removed unnecessary .org files added in previous commit HDFS-3809 (Revision 1381700)
HDFS-3809. Make BKJM use protobufs for all serialization with ZK. Contributed by Ivan Kelly (Revision 1381699)

     Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381700
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig

umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381699
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/CurrentInprogress.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Status: Patch Available  (was: Open)
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487794#comment-13487794 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Mapreduce-trunk #1242 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1242/])
    Moved HDFS-3809 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403769)

     Result = FAILURE
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403769
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Attachment: 0004-HDFS-3809-for-branch-2.patch

Some offset changes.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487075#comment-13487075 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

Thanks Robert. I will check this for branch-2, as there were only offset changes from trunk patch I applied it directly. To unblock others I have just reverted it for now.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450636#comment-13450636 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Mapreduce-trunk #1189 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1189/])
    Removed unnecessary .org files added in previous commit HDFS-3809 (Revision 1381700)
HDFS-3809. Make BKJM use protobufs for all serialization with ZK. Contributed by Ivan Kelly (Revision 1381699)

     Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381700
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig

umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381699
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/CurrentInprogress.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0
>
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487233#comment-13487233 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-trunk-Commit #2944 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2944/])
    Moved HDFS-3809 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403769)

     Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403769
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444233#comment-13444233 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

@Ivan, Thanks a lot for the patch.
On quick it looks good to me. I will take a look in detail tomorrow. Also thanks a lot Vinay, for your review!
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated HDFS-3809:
--------------------------------------

          Resolution: Fixed
    Target Version/s: 2.0.1-alpha, 3.0.0  (was: 3.0.0, 2.0.1-alpha)
              Status: Resolved  (was: Patch Available)
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449814#comment-13449814 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

Ok, it is fine. I will commit it here, as you said this is used in next immediate patch.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487729#comment-13487729 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Hdfs-trunk #1212 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1212/])
    Moved HDFS-3809 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403769)

     Result = FAILURE
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403769
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450478#comment-13450478 ] 

Ivan Kelly commented on HDFS-3809:
----------------------------------

[~umamahesh] I think both these should be pushed to Branch-2 though. I'll ping todd, see if he had a particular reason for not pushing them there.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0
>
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Attachment: HDFS-3809.diff

Fixed findbugs (exclude protobuf from check)
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450590#comment-13450590 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Hdfs-trunk #1158 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1158/])
    Removed unnecessary .org files added in previous commit HDFS-3809 (Revision 1381700)
HDFS-3809. Make BKJM use protobufs for all serialization with ZK. Contributed by Ivan Kelly (Revision 1381699)

     Result = FAILURE
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381700
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig

umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381699
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/CurrentInprogress.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0
>
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486610#comment-13486610 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

I need to update in trunk CHANGES.txt to 2.0.3-alpha section. I will do this in some time today. Also need to do the same for HDFS-3573, HDFS-3695, HDFS-3789. So, leaving it as open util I do that update.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449710#comment-13449710 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

+1 for the patch. It looks good to me. Also thanks Vinay for your reviews part.
I will commit this in some time today. Thanks a lot, Ivan.

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449793#comment-13449793 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

one small nit:

{code}
private final NamespaceInfo nsInfo;
{code}
Why do we need this as field variable? we just used it in ctor only right?
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449801#comment-13449801 ] 

Ivan Kelly commented on HDFS-3809:
----------------------------------

We use it later in HDFS-3810. I can move it to that patch, but if it's only a small issue, i think it'd better to leave it here to push this patch.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Target Version/s: 2.0.1-alpha, 3.0.0  (was: 3.0.0, 2.0.1-alpha)
              Status: Patch Available  (was: Reopened)
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486607#comment-13486607 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

+1 for the ported patch. I have just committed this to branch-2.
 Committed revision 1403606.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449945#comment-13449945 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Mapreduce-trunk-Commit #2718 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2718/])
    Removed unnecessary .org files added in previous commit HDFS-3809 (Revision 1381700)
HDFS-3809. Make BKJM use protobufs for all serialization with ZK. Contributed by Ivan Kelly (Revision 1381699)

     Result = FAILURE
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381700
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig

umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381699
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/CurrentInprogress.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0
>
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Vinay (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449556#comment-13449556 ] 

Vinay commented on HDFS-3809:
-----------------------------

Thanks Ivan for clarifications.
+1. Patch looks good.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487673#comment-13487673 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Yarn-trunk #22 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/22/])
    Moved HDFS-3809 entry in CHANGES.txt from trunk to 2.0.3-alpha section (Revision 1403769)

     Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1403769
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487100#comment-13487100 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

Yes, Robert I just compared, they both looks almost identical. As you said there must be some issue in build scripts. let me/Ivan dig into it.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Attachment: HDFS-3809.diff

Patch makes BKJM use NamespaceInfo also.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Attachment: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487094#comment-13487094 ] 

Robert Joseph Evans commented on HDFS-3809:
-------------------------------------------

Thanks for doing that Uma. It looks like there is something about the build scripts that is causing it, because hdfs.proto, where NameSpaceInfoProto is defined, is more or less identical between trunk and branch-2.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449552#comment-13449552 ] 

Hadoop QA commented on HDFS-3809:
---------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12544022/HDFS-3809.diff
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 2 new or modified test files.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3154//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3154//console

This message is automatically generated.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487160#comment-13487160 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

ok, Adding '+package hadoop.hdfs;' fixed the problem. I will commit new patch.
Generally we will not keep changes.txt in patches. do you mind removing it. For me it is fine to commit above patch. Asking just to keep patches in JIRA consistently.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated HDFS-3809:
--------------------------------------

    Component/s: name-node
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486998#comment-13486998 ] 

Uma Maheswara Rao G commented on HDFS-3809:
-------------------------------------------

I have just updated CHANGES.txt for all. marking it as closed.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487137#comment-13487137 ] 

Ivan Kelly commented on HDFS-3809:
----------------------------------

HDFS-4121 caused this. New patch attached.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436065#comment-13436065 ] 

Hadoop QA commented on HDFS-3809:
---------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12541233/HDFS-3809.diff
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified test files.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    -1 findbugs.  The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3026//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3026//artifact/trunk/patchprocess/newPatchFindbugsWarningsbkjournal.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3026//console

This message is automatically generated.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483323#comment-13483323 ] 

Hadoop QA commented on HDFS-3809:
---------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12550638/0004-HDFS-3809-for-branch-2.patch
  against trunk revision .

    {color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3394//console

This message is automatically generated.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436128#comment-13436128 ] 

Hadoop QA commented on HDFS-3809:
---------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12541241/HDFS-3809.diff
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified test files.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3027//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3027//console

This message is automatically generated.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Attachment: HDFS-3809.diff

New patch addresses comments.

{quote}
One doubt, Do we need to handle existing BKJM layout data compatibility, while reading the existing ledgers..?
{quote}
The current layout is only in the -alpha releases, which are marked as such because APIs haven't been finalized. This can be considered part of this.

{quote}
CURRENT_INPROGRESS_LAYOUT_VERSION version check is removed from the CurrentInprogress.java, do you think this version check not required. In that case CURRENT_INPROGRESS_LAYOUT_VERSION
and also CONTENT_DELIMITER can be removed from CurrentInprogress.java
{quote}
Protobufs remove the need for an explicit inprogress layout version.

{quote}
In CurrentInprogressProto, why hostName is made optional.? is there any specific reason for it..? But i can see that previously always hostname was present in data.
{quote}
It's optional because it's not strictly necessary to function. It's just convenient for debugging. Protobuf guidelines suggest that you always make things optional unless they absolutely need to be there.

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449888#comment-13449888 ] 

Hudson commented on HDFS-3809:
------------------------------

Integrated in Hadoop-Common-trunk-Commit #2694 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2694/])
    Removed unnecessary .org files added in previous commit HDFS-3809 (Revision 1381700)
HDFS-3809. Make BKJM use protobufs for all serialization with ZK. Contributed by Ivan Kelly (Revision 1381699)

     Result = SUCCESS
umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381700
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig

umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381699
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/CurrentInprogress.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto.orig
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java

                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated HDFS-3809:
--------------------------------------

     Target Version/s: 3.0.0, 2.0.1-alpha
    Affects Version/s: 3.0.0
                       2.0.0-alpha
        Fix Version/s:     (was: 3.0.0)
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Fix Version/s: 2.0.3-alpha
    
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Vinay (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442325#comment-13442325 ] 

Vinay commented on HDFS-3809:
-----------------------------

Hi Ivan,
Patch looks nice. Following are the simple comments from my side.

{code}+      LOG.info("Reading " + path + " data: " + new String(data, UTF_8));{code}
Better to be in debug. Or logging only 'data' in debug also Ok.

Compilation Errors in TestBookKeeperJournalManager.java and TestBookKeeperConfiguration.java. Because of {{NamespaceInfo(int,String,String,int,int)}} contructor removal in HDFS-2686.

One doubt, Do we need to handle existing BKJM layout data compatibility, while reading the existing ledgers..?

CURRENT_INPROGRESS_LAYOUT_VERSION version check is removed from the CurrentInprogress.java, do you think this version check not required. In that case CURRENT_INPROGRESS_LAYOUT_VERSION 
and also CONTENT_DELIMITER can be removed from CurrentInprogress.java

In CurrentInprogressProto, why hostName is made optional.? is there any specific reason for it..? But i can see that previously always hostname was present in data.

TestCurrentInprogress.java has only one space change. I think, this can be removed from the patch.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3809) Make BKJM use protobufs for all serialization with ZK

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HDFS-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Kelly updated HDFS-3809:
-----------------------------

    Attachment: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch

ah, I had generated from what had been committed, so thats why it was there. removed now.
                
> Make BKJM use protobufs for all serialization with ZK
> -----------------------------------------------------
>
>                 Key: HDFS-3809
>                 URL: https://issues.apache.org/jira/browse/HDFS-3809
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 3.0.0, 2.0.3-alpha
>
>         Attachments: 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0001-HDFS-3809.-Make-BKJM-use-protobufs-for-all-serializa.patch, 0004-HDFS-3809-for-branch-2.patch, HDFS-3809.diff, HDFS-3809.diff, HDFS-3809.diff
>
>
> HDFS uses protobufs for serialization in many places. Protobufs allow fields to be added without breaking bc or requiring new parsing code to be written. For this reason, we should use them in BKJM also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira