Replication - ports/hosts

[Patrick Schless <pa...@gmail.com>]

Working on setting up HBase replication across a VPN tunnel, and following
the docs here: [1] (and here: [2]).

Two questions, regarding firewall allowances required:
1) The docs say that the zookeeper clusters must be able to reach each
other. I don't see any docs on why this is (the high-level diagram doesn't
even show a ZK ensemble on the client side). Is it correct that the ZK
ensembles need to communicate with each other?

2) What ports are required to be open for inter-cluster Region Server
communication? Is it only the slave that needs to accept requests from the
master, or does the master also accept incoming requests from the slaves?

Thanks,
Patrick

[1] http://hbase.apache.org/replication.html
[2]
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements

Re: Replication - ports/hosts

[Stack <st...@duboce.net>]

(Thank you Michel)


On Wed, Jun 19, 2013 at 8:28 AM, Stack <st...@duboce.net> wrote:

> Pardon me.   I should have mentioned that the slave cluster will send the
> master cluster a response (success, faliure).
> St.Ack
>
>
> On Wed, Jun 19, 2013 at 2:49 AM, Michel Segel <mi...@hotmail.com>wrote:
>
>> What no ack from slave back to master???
>>
>> Sent from a remote device. Please excuse any typos...
>>
>> Mike Segel
>>
>> On Jun 19, 2013, at 12:41 AM, Stack <st...@duboce.net> wrote:
>>
>> > On Mon, Jun 17, 2013 at 12:06 PM, Patrick Schless <
>> patrick.schless@gmail.com
>> >> wrote:
>> >
>> >> Working on setting up HBase replication across a VPN tunnel, and
>> following
>> >> the docs here: [1] (and here: [2]).
>> >>
>> >> Two questions, regarding firewall allowances required:
>> >> 1) The docs say that the zookeeper clusters must be able to reach each
>> >> other. I don't see any docs on why this is (the high-level diagram
>> doesn't
>> >> even show a ZK ensemble on the client side). Is it correct that the ZK
>> >> ensembles need to communicate with each other?
>> > Yes.  ZK ensemble is the gateway to the cluster whether connecting
>> locally
>> > or remotely as replication does.
>> >
>> >
>> >
>> >> 2) What ports are required to be open for inter-cluster Region Server
>> >> communication? Is it only the slave that needs to accept requests from
>> the
>> >> master, or does the master also accept incoming requests from the
>> slaves?
>> >
>> >
>> > Do you mean master 'cluster' in the above?  If so, yes, the master
>> cluster
>> > sends data to the slave cluster (master RS to slave RSs).  The
>> > communication is one way; master to slave cluster.
>> >
>> > If you are asking if hbase masters need to communicate, the answer is
>> no.
>> >
>> > St.Ack
>> > P.S. Doug, we need to add above to doc
>>
>
>

Re: Replication - ports/hosts

[Stack <st...@duboce.net>]

Pardon me.   I should have mentioned that the slave cluster will send the
master cluster a response (success, faliure).
St.Ack


On Wed, Jun 19, 2013 at 2:49 AM, Michel Segel <mi...@hotmail.com>wrote:

> What no ack from slave back to master???
>
> Sent from a remote device. Please excuse any typos...
>
> Mike Segel
>
> On Jun 19, 2013, at 12:41 AM, Stack <st...@duboce.net> wrote:
>
> > On Mon, Jun 17, 2013 at 12:06 PM, Patrick Schless <
> patrick.schless@gmail.com
> >> wrote:
> >
> >> Working on setting up HBase replication across a VPN tunnel, and
> following
> >> the docs here: [1] (and here: [2]).
> >>
> >> Two questions, regarding firewall allowances required:
> >> 1) The docs say that the zookeeper clusters must be able to reach each
> >> other. I don't see any docs on why this is (the high-level diagram
> doesn't
> >> even show a ZK ensemble on the client side). Is it correct that the ZK
> >> ensembles need to communicate with each other?
> > Yes.  ZK ensemble is the gateway to the cluster whether connecting
> locally
> > or remotely as replication does.
> >
> >
> >
> >> 2) What ports are required to be open for inter-cluster Region Server
> >> communication? Is it only the slave that needs to accept requests from
> the
> >> master, or does the master also accept incoming requests from the
> slaves?
> >
> >
> > Do you mean master 'cluster' in the above?  If so, yes, the master
> cluster
> > sends data to the slave cluster (master RS to slave RSs).  The
> > communication is one way; master to slave cluster.
> >
> > If you are asking if hbase masters need to communicate, the answer is no.
> >
> > St.Ack
> > P.S. Doug, we need to add above to doc
>

Re: Replication - ports/hosts

[Michel Segel <mi...@hotmail.com>]

What no ack from slave back to master???

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 19, 2013, at 12:41 AM, Stack <st...@duboce.net> wrote:

> On Mon, Jun 17, 2013 at 12:06 PM, Patrick Schless <patrick.schless@gmail.com
>> wrote:
> 
>> Working on setting up HBase replication across a VPN tunnel, and following
>> the docs here: [1] (and here: [2]).
>> 
>> Two questions, regarding firewall allowances required:
>> 1) The docs say that the zookeeper clusters must be able to reach each
>> other. I don't see any docs on why this is (the high-level diagram doesn't
>> even show a ZK ensemble on the client side). Is it correct that the ZK
>> ensembles need to communicate with each other?
> Yes.  ZK ensemble is the gateway to the cluster whether connecting locally
> or remotely as replication does.
> 
> 
> 
>> 2) What ports are required to be open for inter-cluster Region Server
>> communication? Is it only the slave that needs to accept requests from the
>> master, or does the master also accept incoming requests from the slaves?
> 
> 
> Do you mean master 'cluster' in the above?  If so, yes, the master cluster
> sends data to the slave cluster (master RS to slave RSs).  The
> communication is one way; master to slave cluster.
> 
> If you are asking if hbase masters need to communicate, the answer is no.
> 
> St.Ack
> P.S. Doug, we need to add above to doc

Re: Replication - ports/hosts

[Shahab Yunus <sh...@gmail.com>]

Though experts can chime in and would know better but can't Hadoop/Hbase
authentication (Kerberos) be used for making sure that only the right
clusters (salve/master) combo are talking to each other? Or would we need
something more, outside of HBase even?

Regards,
Shahab


On Wed, Jun 19, 2013 at 11:57 AM, Patrick Schless <patrick.schless@gmail.com
> wrote:

> On Wed, Jun 19, 2013 at 12:41 AM, Stack <st...@duboce.net> wrote:
>
> > On Mon, Jun 17, 2013 at 12:06 PM, Patrick Schless <
> > patrick.schless@gmail.com
> > > wrote:
> >
> > > Working on setting up HBase replication across a VPN tunnel, and
> > following
> > > the docs here: [1] (and here: [2]).
> > >
> > > Two questions, regarding firewall allowances required:
> > > 1) The docs say that the zookeeper clusters must be able to reach each
> > > other. I don't see any docs on why this is (the high-level diagram
> > doesn't
> > > even show a ZK ensemble on the client side). Is it correct that the ZK
> > > ensembles need to communicate with each other?
> > >
> > >
> > Yes.  ZK ensemble is the gateway to the cluster whether connecting
> locally
> > or remotely as replication does.
>
>
> Cool, makes sense. Is there a way to restrict the foreign connections
> (connections from the other cluster, whether master or slave) to be
> read-only?
>
>
> >
> > > 2) What ports are required to be open for inter-cluster Region Server
> > > communication? Is it only the slave that needs to accept requests from
> > the
> > > master, or does the master also accept incoming requests from the
> slaves?
> > >
> >
> >
> > Do you mean master 'cluster' in the above?  If so, yes, the master
> cluster
> > sends data to the slave cluster (master RS to slave RSs).  The
> > communication is one way; master to slave cluster.
> >
> > If you are asking if hbase masters need to communicate, the answer is no.
> >
>
> I should have been more clear, but yes, I was using 'master' to refer to
> the cluster, not the HMaster. Sounds like I only need firewall allowances
> on the region servers in the slave cluster (and slave ZK ensemble). I'll
> give that a shot.
>
> Thanks for the info.
>
> - Patrick
>

Re: Replication - ports/hosts

[Patrick Schless <pa...@gmail.com>]

On Wed, Jun 19, 2013 at 12:41 AM, Stack <st...@duboce.net> wrote:

> On Mon, Jun 17, 2013 at 12:06 PM, Patrick Schless <
> patrick.schless@gmail.com
> > wrote:
>
> > Working on setting up HBase replication across a VPN tunnel, and
> following
> > the docs here: [1] (and here: [2]).
> >
> > Two questions, regarding firewall allowances required:
> > 1) The docs say that the zookeeper clusters must be able to reach each
> > other. I don't see any docs on why this is (the high-level diagram
> doesn't
> > even show a ZK ensemble on the client side). Is it correct that the ZK
> > ensembles need to communicate with each other?
> >
> >
> Yes.  ZK ensemble is the gateway to the cluster whether connecting locally
> or remotely as replication does.


Cool, makes sense. Is there a way to restrict the foreign connections
(connections from the other cluster, whether master or slave) to be
read-only?


>
> > 2) What ports are required to be open for inter-cluster Region Server
> > communication? Is it only the slave that needs to accept requests from
> the
> > master, or does the master also accept incoming requests from the slaves?
> >
>
>
> Do you mean master 'cluster' in the above?  If so, yes, the master cluster
> sends data to the slave cluster (master RS to slave RSs).  The
> communication is one way; master to slave cluster.
>
> If you are asking if hbase masters need to communicate, the answer is no.
>

I should have been more clear, but yes, I was using 'master' to refer to
the cluster, not the HMaster. Sounds like I only need firewall allowances
on the region servers in the slave cluster (and slave ZK ensemble). I'll
give that a shot.

Thanks for the info.

- Patrick

Re: Replication - ports/hosts

[Stack <st...@duboce.net>]

On Mon, Jun 17, 2013 at 12:06 PM, Patrick Schless <patrick.schless@gmail.com
> wrote:

> Working on setting up HBase replication across a VPN tunnel, and following
> the docs here: [1] (and here: [2]).
>
> Two questions, regarding firewall allowances required:
> 1) The docs say that the zookeeper clusters must be able to reach each
> other. I don't see any docs on why this is (the high-level diagram doesn't
> even show a ZK ensemble on the client side). Is it correct that the ZK
> ensembles need to communicate with each other?
>
>
Yes.  ZK ensemble is the gateway to the cluster whether connecting locally
or remotely as replication does.



> 2) What ports are required to be open for inter-cluster Region Server
> communication? Is it only the slave that needs to accept requests from the
> master, or does the master also accept incoming requests from the slaves?
>


Do you mean master 'cluster' in the above?  If so, yes, the master cluster
sends data to the slave cluster (master RS to slave RSs).  The
communication is one way; master to slave cluster.

If you are asking if hbase masters need to communicate, the answer is no.

St.Ack
P.S. Doug, we need to add above to doc