Protobuf Proxy - your feedback is welcomed !

[Amit Mor <am...@hotmail.com>]

I've been thinking of creating a new Twemproxy/Thrift (https://github.com/twitter/twemproxy) like proxy for HBase that would be based on using (as the underlying transport protocol) the native protocol-buffer templates that come with HBase source. 
This could be viewed as a Thrift alternative, also, and CMIIW, I don't see any reason why the HBase community should support 2 protocol types, at least from 'singularity' version onward, where protobuf is in heavy use internally/externally.
 
I envision that the proxy would provide the following benefits:Async semantics - client would not need to implement anything new except wrapping their calls in something like futures and pass the calls to the proxy, using the same 'commands' as the Java native cmds (that are already defined as protos). Only (well, mostly...) the proxy would have to deal the concerns associated with contention, starvation, blocking etc.Batching/Pipelining - the service would be able to batch queries sent from several clients connected to the (local) proxy service and batch them The proxy service could act as Facade between the client and HBase (i.e. stabilize a protocol that withhold API changes in HBase client) Separation of concerns - no need for the application to 'tame' the HBase driver in terms of IO resources, memory and buffer size, connection pool size, etc.Naturally - Language agonisticTesting - the proxy could hold a mock connection to HBase or a mock HBase for easier testing
Your kind feedback on the idea, concept and usefulness is very welcomed
Thanks,
Amit Mor
  		 	   		  

Re: Protobuf Proxy - your feedback is welcomed !

[Stack <st...@duboce.net>]

On Wed, Aug 5, 2015 at 9:29 AM, Amit Mor <am...@hotmail.com> wrote:

>
>
>
>
> I've been thinking of creating a new Twemproxy/Thrift (
> https://github.com/twitter/twemproxy) like proxy for HBase that would be
> based on using (as the underlying transport protocol) the native
> protocol-buffer templates that come with HBase source.
>

So, what'd be the memcached/redis in this case? What is the problem that is
being solved? Thanks Amit,
St.Ack



> This could be viewed as a Thrift alternative, also, and CMIIW, I don't see
> any reason why the HBase community should support 2 protocol types, at
> least from 'singularity' version onward, where protobuf is in heavy use
> internally/externally.
>
> I envision that the proxy would provide the following benefits:Async
> semantics - client would not need to implement anything new except wrapping
> their calls in something like futures and pass the calls to the proxy,
> using the same 'commands' as the Java native cmds (that are already defined
> as protos). Only (well, mostly...) the proxy would have to deal the
> concerns associated with contention, starvation, blocking
> etc.Batching/Pipelining - the service would be able to batch queries sent
> from several clients connected to the (local) proxy service and batch them
> The proxy service could act as Facade between the client and HBase (i.e.
> stabilize a protocol that withhold API changes in HBase client) Separation
> of concerns - no need for the application to 'tame' the HBase driver in
> terms of IO resources, memory and buffer size, connection pool size,
> etc.Naturally - Language agonisticTesting - the proxy could hold a mock
> connection to HBase or a mock HBase for easier testing
> Your kind feedback on the idea, concept and usefulness is very welcomed
> Thanks,
> Amit Mor
>