[jira] [Comment Edited] (ZOOKEEPER-102) Need to replace Jute with supported code

["Jordan Zimmerman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976894#comment-16976894 ] 

Jordan Zimmerman edited comment on ZOOKEEPER-102 at 11/18/19 9:24 PM:
----------------------------------------------------------------------

I've written a proof of concept implementation of a ServerCnxnFactory that implements gRPC. The goal is to make it possible to easily write ZooKeeper clients in non-JVM languages. Using the proof of concept I was able to write a Golang client easily. I'd be willing to move this from proof of concept to production but I'll need help (1 or 2 co-developers).
 
If you want to try it, I've pushed the Golang client and some instructions here (let me know if you have any issues - I'm a go neophyte). Note: "zookeeper/test.go" is the interesting file:
 
[https://github.com/Randgalt/zkgrpc]
 
Here's the proof of concept on the ZK server side (the interesting files are RpcServerCnxn.java, RpcServerCnxnFactory.java, RpcZooKeeperServer.java and zookeeper.proto):
 
[https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc]
 
Issues:
 * Writing a client, even with gRPC, will require some work. Sessions have to be maintained, watchers have to be maintained, etc.
 * Currently, Jute is deeply embedded in ZooKeeper. The proof of concept has to emulate Jute byte buffers. Ideally, this will be abstracted so that only records could be used so that the gRPC connection doesn't have to keep marshalling/unmarshalling byte buffers
 * I don't know enough about the gRPC client/server implementations to know if it will meet the needs of ZooKeeper. Anyone have experience here?
 * I haven't completely thought through how much work it will take to write useful clients. As I've shown with the proof of concept simple ZK CRUD db operations work well. I need to spend time writing a recipe such as Leader Election to see how much work is required.
 * I'm not sure how things like SASL and reconfig would work with gRPC


was (Author: randgalt):
've written a proof of concept implementation of a ServerCnxnFactory that implements gRPC. The goal is to make it possible to easily write ZooKeeper clients in non-JVM languages. Using the proof of concept I was able to write a Golang client easily. I'd be willing to move this from proof of concept to production but I'll need help (1 or 2 co-developers).
 
If you want to try it, I've pushed the Golang client and some instructions here (let me know if you have any issues - I'm a go neophyte). Note: "zookeeper/test.go" is the interesting file:
 
[https://github.com/Randgalt/zkgrpc]
 
Here's the proof of concept on the ZK server side (the interesting files are RpcServerCnxn.java, RpcServerCnxnFactory.java, RpcZooKeeperServer.java and zookeeper.proto):
 
[https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc]
 
Issues:
 * Writing a client, even with gRPC, will require some work. Sessions have to be maintained, watchers have to be maintained, etc.
 * Currently, Jute is deeply embedded in ZooKeeper. The proof of concept has to emulate Jute byte buffers. Ideally, this will be abstracted so that only records could be used so that the gRPC connection doesn't have to keep marshalling/unmarshalling byte buffers
 * I don't know enough about the gRPC client/server implementations to know if it will meet the needs of ZooKeeper. Anyone have experience here?
 * I haven't completely thought through how much work it will take to write useful clients. As I've shown with the proof of concept simple ZK CRUD db operations work well. I need to spend time writing a recipe such as Leader Election to see how much work is required.
 * I'm not sure how things like SASL and reconfig would work with gRPC

> Need to replace Jute with supported code
> ----------------------------------------
>
>                 Key: ZOOKEEPER-102
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102
>             Project: ZooKeeper
>          Issue Type: Improvement
>            Reporter: Benjamin Reed
>            Assignee: Jordan Zimmerman
>            Priority: Major
>             Fix For: 4.0.0
>
>
> ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.
> Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)