You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Benjamin Reed (JIRA)" <ji...@apache.org> on 2008/07/25 22:27:31 UTC
[jira] Created: (ZOOKEEPER-102) Need to replace Jute with supported
code
Need to replace Jute with supported code
----------------------------------------
Key: ZOOKEEPER-102
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102
Project: Zookeeper
Issue Type: Improvement
Reporter: Benjamin Reed
ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.
Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with
supported code
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923521#action_12923521 ]
Patrick Hunt commented on ZOOKEEPER-102:
----------------------------------------
haha, doug and I crossed beams on comments. :-)
> Need to replace Jute with supported code
> ----------------------------------------
>
> Key: ZOOKEEPER-102
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102
> Project: Zookeeper
> Issue Type: Improvement
> Reporter: Benjamin Reed
>
> ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.
> Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with
supported code
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617226#action_12617226 ]
Hiram Chirino commented on ZOOKEEPER-102:
-----------------------------------------
[protobuf|http://code.google.com/p/protobuf/] might be an option.
> Need to replace Jute with supported code
> ----------------------------------------
>
> Key: ZOOKEEPER-102
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102
> Project: Zookeeper
> Issue Type: Improvement
> Reporter: Benjamin Reed
>
> ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.
> Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with
supported code
Posted by "Thomas Koch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923380#action_12923380 ]
Thomas Koch commented on ZOOKEEPER-102:
---------------------------------------
Another option would be avro. Does anybody know, whether avro is wire compatible with jute? We would need to support communication between older jute-only Clients/Servers and newer avro(+jute?) Clients/Servers.
Would this be a good Google Summer of Code task?
> Need to replace Jute with supported code
> ----------------------------------------
>
> Key: ZOOKEEPER-102
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102
> Project: Zookeeper
> Issue Type: Improvement
> Reporter: Benjamin Reed
>
> ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.
> Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with
supported code
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923516#action_12923516 ]
Doug Cutting commented on ZOOKEEPER-102:
----------------------------------------
Avro is not wire-compatible with Jute.
> Need to replace Jute with supported code
> ----------------------------------------
>
> Key: ZOOKEEPER-102
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102
> Project: Zookeeper
> Issue Type: Improvement
> Reporter: Benjamin Reed
>
> ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.
> Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with
supported code
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923519#action_12923519 ]
Patrick Hunt commented on ZOOKEEPER-102:
----------------------------------------
bq. avro is wire compatible with jute?
in general no, however Doug was wondering if there was some way to define the avro spec in such a way that it would work.
bq. Would this be a good Google Summer of Code task?
it would be an interesting project regardless. We've discussed it a few times, the general idea being that we should add the ability to configure a new client port using Avro, maintain b/w compatibility by leaving the old jute based port alone (although allow it to be turned off through config). Once all clients were migrated the zk ensemble admin could turn off the legacy jute port.
Btw, while fixing this we should also split the client and admin (4letterword) ports. This causes us all kinds of headache (today shared) and really the 4letterword feature should be expanded to allow for more sophisticated usage (for example long lived connection). Also I've heard complaints from admins that they don't want to expose the 4letter words to all users (it's r/o but they want to lock down that data). Again, having a separate admin more might allow for more features.
One other issue comes to mind. Last time I benchmarked our message definitions on Jute vs Avro the avro C client performance was terrible. We wouldn't want to switch over from jute to avro (default I'm saying) until this is addressed. I'm also not sure if the avro c api is locked down.
> Need to replace Jute with supported code
> ----------------------------------------
>
> Key: ZOOKEEPER-102
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102
> Project: Zookeeper
> Issue Type: Improvement
> Reporter: Benjamin Reed
>
> ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.
> Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.