RPC Replication Compression

[Asaf Mesika <as...@gmail.com>]

Hi,

Just wanted to make sure if I read in the internet correctly: 0.96 will
support HBase RPC compression thus Replication between master and slave
will enjoy it as well (important since bandwidth between geographically
distant data centers is scarce and more expensive)

Re: RPC Replication Compression

[Stack <st...@duboce.net>]

On Tue, Jun 4, 2013 at 6:48 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:

> Replication doesn't need to know about compression at the RPC level so
> it won't refer to it and as far as I can tell you need to set
> compression only on the master cluster and the slave will figure it
> out.
>
> Looking at the code tho, I'm not sure it works the same way it used to
> work before everything went protobuf. I would give 2 internets to
> whoever tests 0.95.1 with RPC compression turned on and compares
> results with non-compressed RPC. See
> http://hbase.apache.org/book.html#rpc.configs


What are you looking for JD?  Faster replication or just less network used?

Looks like we have not had the ability to do compressed rpc before (We
almost did,  the original rpc compression attempt almost got committed to
trunk -- see HBASE-5355 and the referenced follow-on issue -- but was put
aside after the pb stuff went in).

St.Ack

Re: RPC Replication Compression

[Jean-Daniel Cryans <jd...@apache.org>]

Replication doesn't need to know about compression at the RPC level so
it won't refer to it and as far as I can tell you need to set
compression only on the master cluster and the slave will figure it
out.

Looking at the code tho, I'm not sure it works the same way it used to
work before everything went protobuf. I would give 2 internets to
whoever tests 0.95.1 with RPC compression turned on and compares
results with non-compressed RPC. See
http://hbase.apache.org/book.html#rpc.configs

J-D

On Tue, Jun 4, 2013 at 5:22 AM, Asaf Mesika <as...@gmail.com> wrote:
> If RPC has compression abilities, how come Replication, which also works in
> RPC does not get it automatically?
>
>
> On Tue, Jun 4, 2013 at 12:34 PM, Anoop John <an...@gmail.com> wrote:
>
>> > 0.96 will support HBase RPC compression
>> Yes
>>
>> > Replication between master and slave
>> will enjoy it as well (important since bandwidth between geographically
>> distant data centers is scarce and more expensive)
>>
>> But I can not see it is being utilized in replication. May be we can do
>> improvements in this area. I can see possibilities.
>>
>> -Anoop-
>>
>>
>> On Tue, Jun 4, 2013 at 1:51 PM, Asaf Mesika <as...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > Just wanted to make sure if I read in the internet correctly: 0.96 will
>> > support HBase RPC compression thus Replication between master and slave
>> > will enjoy it as well (important since bandwidth between geographically
>> > distant data centers is scarce and more expensive)
>> >
>>

Re: RPC Replication Compression

[Asaf Mesika <as...@gmail.com>]

If RPC has compression abilities, how come Replication, which also works in
RPC does not get it automatically?


On Tue, Jun 4, 2013 at 12:34 PM, Anoop John <an...@gmail.com> wrote:

> > 0.96 will support HBase RPC compression
> Yes
>
> > Replication between master and slave
> will enjoy it as well (important since bandwidth between geographically
> distant data centers is scarce and more expensive)
>
> But I can not see it is being utilized in replication. May be we can do
> improvements in this area. I can see possibilities.
>
> -Anoop-
>
>
> On Tue, Jun 4, 2013 at 1:51 PM, Asaf Mesika <as...@gmail.com> wrote:
>
> > Hi,
> >
> > Just wanted to make sure if I read in the internet correctly: 0.96 will
> > support HBase RPC compression thus Replication between master and slave
> > will enjoy it as well (important since bandwidth between geographically
> > distant data centers is scarce and more expensive)
> >
>

Re: RPC Replication Compression

[Anoop John <an...@gmail.com>]

> 0.96 will support HBase RPC compression
Yes

> Replication between master and slave
will enjoy it as well (important since bandwidth between geographically
distant data centers is scarce and more expensive)

But I can not see it is being utilized in replication. May be we can do
improvements in this area. I can see possibilities.

-Anoop-


On Tue, Jun 4, 2013 at 1:51 PM, Asaf Mesika <as...@gmail.com> wrote:

> Hi,
>
> Just wanted to make sure if I read in the internet correctly: 0.96 will
> support HBase RPC compression thus Replication between master and slave
> will enjoy it as well (important since bandwidth between geographically
> distant data centers is scarce and more expensive)
>