You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Mick Hittesdorf <mi...@gmail.com> on 2012/12/11 01:13:28 UTC
ClientService protobuf RPC service implementation?
Has any work been done to implement the ClientService protobuf server
defined in the following proto file?
http://svn.apache.org/viewvc/hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto?view=markup
I'd like to build an HBase client in C++ using the new protobuf-based
client protocol. Where should I start?
Thanks!
Re: ClientService protobuf RPC service implementation?
Posted by Elliott Clark <ec...@apache.org>.
The blocking interface is just used to tie the request type to a response
type. The actual non-blocking/blocking part is implemented in HBaseRPC
(it actually uses nio to do the io).
If you want to create your own client that's non-blocking you'll have to
create your own version of HBaseRPC and the serialization. Here's
something close to what you were talking about that I created. It can read
root and meta but not much more:
https://github.com/elliottneilclark/hbase/tree/webscale_client
Though currently in trunk lots of changes have been made for how HBase
sends KeyValues across the wire.
On Tue, Dec 11, 2012 at 9:08 PM, Ted Yu <yu...@gmail.com> wrote:
> protobuf is used in HBase for serialization.
>
> I searched the files under
> hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated but
> didn't see any non-blocking interface.
>
> A brief search on the web resulted in:
> https://groups.google.com/forum/?fromgroups=#!topic/protobuf/s7jAGvJKgmA
>
> But that thread was really old.
>
> Would do more searching.
>
> Cheers
>
> On Tue, Dec 11, 2012 at 6:05 PM, Mick Hittesdorf
> <mi...@gmail.com>wrote:
>
> > Thanks for the information. I was wondering why the ClientProtocol
> > specifically implements the BlockingInterface? I want to write a
> > non-blocking, event-driven, asynchronous, multi-threaded client similar
> to
> > asynchhbase (https://github.com/OpenTSDB/asynchbase) but in C++. Am I
> > going to need to 'hide' the blocking semantics implemented by HBase in my
> > client in order to achieve this?
> >
> > Thanks in advance! Mick
> >
> > On Mon, Dec 10, 2012 at 11:00 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > src/main/java/org/apache/hadoop/hbase/client/ClientProtocol.java
> > >
> >
>
Re: ClientService protobuf RPC service implementation?
Posted by Ted Yu <yu...@gmail.com>.
protobuf is used in HBase for serialization.
I searched the files under
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated but
didn't see any non-blocking interface.
A brief search on the web resulted in:
https://groups.google.com/forum/?fromgroups=#!topic/protobuf/s7jAGvJKgmA
But that thread was really old.
Would do more searching.
Cheers
On Tue, Dec 11, 2012 at 6:05 PM, Mick Hittesdorf
<mi...@gmail.com>wrote:
> Thanks for the information. I was wondering why the ClientProtocol
> specifically implements the BlockingInterface? I want to write a
> non-blocking, event-driven, asynchronous, multi-threaded client similar to
> asynchhbase (https://github.com/OpenTSDB/asynchbase) but in C++. Am I
> going to need to 'hide' the blocking semantics implemented by HBase in my
> client in order to achieve this?
>
> Thanks in advance! Mick
>
> On Mon, Dec 10, 2012 at 11:00 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > src/main/java/org/apache/hadoop/hbase/client/ClientProtocol.java
> >
>
Re: ClientService protobuf RPC service implementation?
Posted by Mick Hittesdorf <mi...@gmail.com>.
Thanks for the information. I was wondering why the ClientProtocol
specifically implements the BlockingInterface? I want to write a
non-blocking, event-driven, asynchronous, multi-threaded client similar to
asynchhbase (https://github.com/OpenTSDB/asynchbase) but in C++. Am I
going to need to 'hide' the blocking semantics implemented by HBase in my
client in order to achieve this?
Thanks in advance! Mick
On Mon, Dec 10, 2012 at 11:00 PM, Ted Yu <yu...@gmail.com> wrote:
> src/main/java/org/apache/hadoop/hbase/client/ClientProtocol.java
>
Re: ClientService protobuf RPC service implementation?
Posted by Ted Yu <yu...@gmail.com>.
Take a look at:
ClientService.BlockingInterface, VersionedProtocol {
in src/main/java/org/apache/hadoop/hbase/client/ClientProtocol.java
If you search for references to ClientProtocol, you would see, e.g. :
this.rpcServer = HBaseRPC.getServer(AdminProtocol.class, this,
new Class<?>[]{ClientProtocol.class,
in HRegionServer
Cheers
On Mon, Dec 10, 2012 at 4:13 PM, Mick Hittesdorf
<mi...@gmail.com>wrote:
> Has any work been done to implement the ClientService protobuf server
> defined in the following proto file?
>
>
> http://svn.apache.org/viewvc/hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto?view=markup
>
> I'd like to build an HBase client in C++ using the new protobuf-based
> client protocol. Where should I start?
>
> Thanks!
>
Re: ClientService protobuf RPC service implementation?
Posted by Stack <st...@duboce.net>.
On Mon, Dec 10, 2012 at 4:13 PM, Mick Hittesdorf
<mi...@gmail.com>wrote:
> Has any work been done to implement the ClientService protobuf server
> defined in the following proto file?
>
>
> http://svn.apache.org/viewvc/hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto?view=markup
>
> I'd like to build an HBase client in C++ using the new protobuf-based
> client protocol. Where should I start?
>
> Thanks!
>
That described service is currently what client and server use in trunk.
All communication is protobuf'd as of a commit today but we are working on
coming back some from the total protobuf'ing of all we pass, in particular,
the pb'ing of all KeyValues passed across the ipc chasm. We will do
something more 'raw' passing the bigdata. The issue we are discussing how
to pass big volumes of KeyValues is HBASE-7233. It would be interesting to
get your input coming from your c++ perspective.
Thanks,
St.Ack
*
*