You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Matan Amir <ma...@voxer.com> on 2012/12/20 05:05:24 UTC
HBase Wire Protocol
Hi,
I'm wondering if anyone has a spec for the wire protocol of HBase (v0.94.x
compatible)? I am interested in creating a new native client this would be
more efficient than reverse-engineering.
Much appreciated!
Re: HBase Wire Protocol
Posted by Matan Amir <ma...@voxer.com>.
Thanks again St.Ack. Will do.
I have to say that the comments alone in asynchbase are worth the price of
admission! Very entertaining :)
On Wed, Dec 19, 2012 at 11:31 PM, Stack <st...@duboce.net> wrote:
> On Wed, Dec 19, 2012 at 8:53 PM, Matan Amir <ma...@voxer.com> wrote:
>
> > Thanks St.Ack,
> >
> > Funny enough, since we use asynchbase internally, i am planning on using
> > that model (async, queue per region, time-based push rather than size,
> etc)
> > and I've already started reading through asynchbase inline comments -
> > specifically in HBaseRpc. I was just hoping there was something more
> > official. The VLong encoding is quite something btw > <
> >
> >
> Any recommendations for what we should use going forward? (Benoît on
> hadoops' vlong encoding makes for a good read indeed:
> https://github.com/OpenTSDB/asynchbase/blob/master/src/HBaseRpc.java#L826)
>
>
>
> > I did also notice the protobuf code in HBase trunk but i assume that is
> for
> > a future release (0.96+?)
> >
> >
> Yessir.
>
> All pb all the time except for passing of actual KeyValues. We'll pass
> DataBlocks instead. Clients will say what they can digest (compressions,
> etc.).
>
> Keep us in the loop. We might save you a bit of head banging when you are
> trying to figure why something is the way it is.
>
> St.Ack
>
Re: HBase Wire Protocol
Posted by Stack <st...@duboce.net>.
On Wed, Dec 19, 2012 at 8:53 PM, Matan Amir <ma...@voxer.com> wrote:
> Thanks St.Ack,
>
> Funny enough, since we use asynchbase internally, i am planning on using
> that model (async, queue per region, time-based push rather than size, etc)
> and I've already started reading through asynchbase inline comments -
> specifically in HBaseRpc. I was just hoping there was something more
> official. The VLong encoding is quite something btw > <
>
>
Any recommendations for what we should use going forward? (Benoît on
hadoops' vlong encoding makes for a good read indeed:
https://github.com/OpenTSDB/asynchbase/blob/master/src/HBaseRpc.java#L826)
> I did also notice the protobuf code in HBase trunk but i assume that is for
> a future release (0.96+?)
>
>
Yessir.
All pb all the time except for passing of actual KeyValues. We'll pass
DataBlocks instead. Clients will say what they can digest (compressions,
etc.).
Keep us in the loop. We might save you a bit of head banging when you are
trying to figure why something is the way it is.
St.Ack
Re: HBase Wire Protocol
Posted by Matan Amir <ma...@voxer.com>.
Thanks St.Ack,
Funny enough, since we use asynchbase internally, i am planning on using
that model (async, queue per region, time-based push rather than size, etc)
and I've already started reading through asynchbase inline comments -
specifically in HBaseRpc. I was just hoping there was something more
official. The VLong encoding is quite something btw > <
I did also notice the protobuf code in HBase trunk but i assume that is for
a future release (0.96+?)
On Wed, Dec 19, 2012 at 8:35 PM, Stack <st...@duboce.net> wrote:
> On Wed, Dec 19, 2012 at 8:05 PM, Matan Amir <ma...@voxer.com> wrote:
>
> > Hi,
> >
> > I'm wondering if anyone has a spec for the wire protocol of HBase
> (v0.94.x
> > compatible)? I am interested in creating a new native client this would
> be
> > more efficient than reverse-engineering.
> >
> > Much appreciated!
> >
>
>
> Best place to look as far as I know is asynchbase. It manages to ride over
> the vagaries and different hbase versions in spite of all we unknowingly
> have done to frustrate anyone being able to add a client beyond the one
> hbase ships with. Read the code comments (Hope bad language does not
> offend -- smile).
>
> 0.96 and beyond should be saner though still being defined (input welcome).
>
> Yours,
> St.Ack
>
Re: HBase Wire Protocol
Posted by Stack <st...@duboce.net>.
On Wed, Dec 19, 2012 at 8:05 PM, Matan Amir <ma...@voxer.com> wrote:
> Hi,
>
> I'm wondering if anyone has a spec for the wire protocol of HBase (v0.94.x
> compatible)? I am interested in creating a new native client this would be
> more efficient than reverse-engineering.
>
> Much appreciated!
>
Best place to look as far as I know is asynchbase. It manages to ride over
the vagaries and different hbase versions in spite of all we unknowingly
have done to frustrate anyone being able to add a client beyond the one
hbase ships with. Read the code comments (Hope bad language does not
offend -- smile).
0.96 and beyond should be saner though still being defined (input welcome).
Yours,
St.Ack