[jira] [Updated] (HBASE-7905) Add passing of optional cell blocks over rpc

["stack (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-7905:
-------------------------

    Attachment: 7905v3.txt

Update patch.  Here is the new commit message, like the old but edited to match the patch:

{code}
Patch is a bunch of generated protobuf code so is larger than it actually is.

Still trying to get all tests to pass and still have to do perf testing.

This adds passing of encoded blocks of cells over rpc after the request header and request param
on request and then after the response header and response result on reply. Cell blocks
are optionally passed though I have the multi call doing it going both ways.

There is much not to like in this patch:

+ We need to ferry the Cells down from the client and into the rpc layer where they
are built ahead of sending. We do it by taking them on to a object to get them over
the proxy; given our protocols all implement protobuf Service, it tacks them on to
till now 'useless' rpccontroller (it is not a payloadcarryingrpccontroller). It
is symmetric in that we do the same server side both getting Cells and then replying.
We are constrained by protobuf Service in that it takes a single param Message only
and returns a single Message response. We could amend protoc and output methods
that take an extra arg, as garyh has suggested. We can do this later.

+ This is not the fault of this patch, but it does bring it out in starker
relief, when we get to the server-side, the first thing we do is we convert
protos to pojos.  There is an awful lot of work being done here. We should let
the protos go into the server and save a conversion. This we should do.

+ It is a little awkward trying to keep the pb object together with its
data which is being carried separately out in cell blocks. Hard to avoid but
ugly nonetheless.

Added a Cell proto. Have Result proto carry Cells rather than KeyValues
Renamed the Mutate proto to be MutationProto.

In rpc.proto, wrote out how the new wire protocol works. Amended
object types in accordance. New connectionheader, requestheader,
responseheader, etc., proto types used by rpc

Moved a ServerCallable out of ProtobufUtil which seemed like a strange
place for it and made it standalone in the client package. There is
alot going on in this class already setting up the multi calls.

Added FatalConnectionException in ipc. Used when Connection setup fails
fatally.

Rejigger of HBaseClient and HBaseServer to implement new rpc spec.
Renamed some methods to suit the spec and more what part of the
communication they are doing. A Call now has a CellScanner to send
or as its response. Stripped a bunch of unused code (// call)

The QoS stuff was internal to RegionServer. I kept it for now
(though we should do what Elliott suggests and just have client
say what its priority is – thats easier – and we should do it)
but moved it out to its own class. Removed the crazy bit where
we would deserialize the request just to figure its method name
so could figure a priority.

Added an IPCUtil in ipc package to do some common operation,
some of which are shared by server and client; e.g. cell block
building and reading.

Added PayloadCarryingRpcController class to carry CellScanners across
the proxy divide.

There were a few places where we were doing caching of class constructors
and methods. Moved them all out to new RelectionCache class and have
everyone use it.

RemoteWithExtrasException has been padded out to carry stuff needed
by RegionMovedException and DoNotRetryException.

RpcServer was crazy and still is. Needs cleanup badly.

In a bunch of places using TextFormat. Its nice in that it prints out
the protobuf; handly debugging... but it prints the whole thing.
Need to come along after and add a version that does a shortened version
but will go with this for now (need to get this toString facility up
in UI too... and into the slowToRespond type log messages....
really helps... its something that had to be hand done
when we were up on hand done Writables)

Added converiions to protobufutil and to ResponseConverter to handle CellScanner.

Adds a MessageCodec, a cell block encoder that passes Cells as pb.

Moved some fundamental classes back into hbase-common.  Odd having hbase-server
get classes from hbase-client.  Makes more sense getting them from hbase-common
when needed by both modules SCRATCH--UNDID THIS IN NAME OF MAKING PATCH SMALLER.
CAN DO IN ANOTHER PATCH

Moved ByteBufferOutputStream up into hbase-common.  Belongs in io package.

Made it so a null or empty row is illegal for Put, Get, etc. Can cause grief
if gets through to server-side and besides should just not be allowed.  Added
also a constructor that can take array, offset, length so can pass a Cell.

Added new WrongRowException thrown when you try to set a KV that has different
row from host object.

In HConnectionManager, made some logging trace level.  Moves the knotty multi callable
out to its own class, MultiServerCallable.

This patch changes Mutate pb object to instead be Mutation so some
ripple from this.
                
> Add passing of optional cell blocks over rpc
> --------------------------------------------
>
>                 Key: HBASE-7905
>                 URL: https://issues.apache.org/jira/browse/HBASE-7905
>             Project: HBase
>          Issue Type: Sub-task
>          Components: IPC/RPC
>            Reporter: stack
>             Fix For: 0.95.0
>
>         Attachments: 7905.txt, 7905v3.txt
>
>
> Make it so we can pass Cells/data w/o having to bury it all in protobuf to get it over the wire.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira