You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Keith Turner <ke...@deenlo.com> on 2013/12/18 05:32:14 UTC
Accumulo 1.6.0 HA namenode testing
Has anyone has done any HA namenode testing w/ 1.6.0?
I have started poking at viewfs:// w/ two regular namenodes and have been
finding issues. I want to eventually run the test suite against Accumulo
1.6.0 w/ multiple HA namenodes.
Re: Accumulo 1.6.0 HA namenode testing
Posted by Josh Elser <jo...@gmail.com>.
On 12/17/13, 11:52 PM, Keith Turner wrote:
> On Tue, Dec 17, 2013 at 11:39 PM, Josh Elser <jo...@gmail.com> wrote:
>
>> I've been running CI tests against 1.5.1 with HA (QJM) Namenode support
>> for a few weeks now. Earlier this week I started running the same against
>> 1.6.0.
>>
>> AFAICT, everything has been fine so far. This is with the
>> hdfs-agitation.pl (causing a failover using the haadmin command every ten
>> minutes) script that Sean provided a while back.
>
>
> Were you also killing datanodes and tservers? Did you run continuous
> ingest and verify?
Yes and yes.
>
>>
>>
>> On 12/17/13, 11:32 PM, Keith Turner wrote:
>>
>>> Has anyone has done any HA namenode testing w/ 1.6.0?
>>>
>>> I have started poking at viewfs:// w/ two regular namenodes and have been
>>> finding issues. I want to eventually run the test suite against Accumulo
>>> 1.6.0 w/ multiple HA namenodes.
>>>
>>>
>
Re: Accumulo 1.6.0 HA namenode testing
Posted by Keith Turner <ke...@deenlo.com>.
On Tue, Dec 17, 2013 at 11:39 PM, Josh Elser <jo...@gmail.com> wrote:
> I've been running CI tests against 1.5.1 with HA (QJM) Namenode support
> for a few weeks now. Earlier this week I started running the same against
> 1.6.0.
>
> AFAICT, everything has been fine so far. This is with the
> hdfs-agitation.pl (causing a failover using the haadmin command every ten
> minutes) script that Sean provided a while back.
Were you also killing datanodes and tservers? Did you run continuous
ingest and verify?
>
>
> On 12/17/13, 11:32 PM, Keith Turner wrote:
>
>> Has anyone has done any HA namenode testing w/ 1.6.0?
>>
>> I have started poking at viewfs:// w/ two regular namenodes and have been
>> finding issues. I want to eventually run the test suite against Accumulo
>> 1.6.0 w/ multiple HA namenodes.
>>
>>
Re: Accumulo 1.6.0 HA namenode testing
Posted by Ravi Mutyala <ra...@hortonworks.com>.
Do you mean a federated cluster? Different namenodes that share the same
datanodes.
On Tue, Dec 17, 2013 at 10:44 PM, Keith Turner <ke...@deenlo.com> wrote:
> On Tue, Dec 17, 2013 at 11:39 PM, Josh Elser <jo...@gmail.com> wrote:
>
> > I've been running CI tests against 1.5.1 with HA (QJM) Namenode support
> > for a few weeks now. Earlier this week I started running the same against
> > 1.6.0.
> >
> > AFAICT, everything has been fine so far. This is with the
> > hdfs-agitation.pl (causing a failover using the haadmin command every
> ten
> > minutes) script that Sean provided a while back.
> >
> >
> I was not using the correct terminology for what I was thinking, but I am
> not sure what the correct terminology is. I do not want to be limited by
> the memory of one machine for hdfs metadata and I want HA. So would it be
> correct to say that I want to test with multiple HA nameservices? For
> example test 1.6 w/ three nameservices, where each nameservice has two HA
> namenodes. This would allow hdfs metadata to spread across the memory of
> three machines.
>
>
> >
> > On 12/17/13, 11:32 PM, Keith Turner wrote:
> >
> >> Has anyone has done any HA namenode testing w/ 1.6.0?
> >>
> >> I have started poking at viewfs:// w/ two regular namenodes and have
> been
> >> finding issues. I want to eventually run the test suite against
> Accumulo
> >> 1.6.0 w/ multiple HA namenodes.
> >>
> >>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Accumulo 1.6.0 HA namenode testing
Posted by Sean Busbey <bu...@clouderagovt.com>.
On Dec 18, 2013 12:00 AM, "Keith Turner" <ke...@deenlo.com> wrote:
>
> On Tue, Dec 17, 2013 at 11:52 PM, Sean Busbey <busbey+ml@clouderagovt.com
>wrote:
>
> > On Tue, Dec 17, 2013 at 10:44 PM, Keith Turner <ke...@deenlo.com> wrote:
> >
> > > On Tue, Dec 17, 2013 at 11:39 PM, Josh Elser <jo...@gmail.com>
> > wrote:
> > >
> > >
> > > I was not using the correct terminology for what I was thinking, but
I am
> > > not sure what the correct terminology is. I do not want to be
limited by
> > > the memory of one machine for hdfs metadata and I want HA. So would
it
> > be
> > > correct to say that I want to test with multiple HA nameservices?
For
> > > example test 1.6 w/ three nameservices, where each nameservice has
two HA
> > > namenodes. This would allow hdfs metadata to spread across the
memory
> > of
> > > three machines.
> > >
> > >
> > These HDFS nameservices are independent HDFS instances using the new
> > span-HDFS-instances feature in Accumulo 1.6? Or they are nameservices
using
> > HDFS federation across HA pairs?
> >
>
> HDFS federation across HA pairs, describes what I am thinking. I did
want
> the HA pairs to share datanodes as Ravi mentioned. So I would like to
test
> w/ 2 or more federated HA pairs.
>
>
Yeah, that's a supported configuration of HDFS federation with HA. HDFS
Federation always shares datanodes.
I've done some testing on earlier versions with HA as Josh described.
Nothing with Federation yet.
Re: Accumulo 1.6.0 HA namenode testing
Posted by Keith Turner <ke...@deenlo.com>.
On Tue, Dec 17, 2013 at 11:52 PM, Sean Busbey <bu...@clouderagovt.com>wrote:
> On Tue, Dec 17, 2013 at 10:44 PM, Keith Turner <ke...@deenlo.com> wrote:
>
> > On Tue, Dec 17, 2013 at 11:39 PM, Josh Elser <jo...@gmail.com>
> wrote:
> >
> >
> > I was not using the correct terminology for what I was thinking, but I am
> > not sure what the correct terminology is. I do not want to be limited by
> > the memory of one machine for hdfs metadata and I want HA. So would it
> be
> > correct to say that I want to test with multiple HA nameservices? For
> > example test 1.6 w/ three nameservices, where each nameservice has two HA
> > namenodes. This would allow hdfs metadata to spread across the memory
> of
> > three machines.
> >
> >
> These HDFS nameservices are independent HDFS instances using the new
> span-HDFS-instances feature in Accumulo 1.6? Or they are nameservices using
> HDFS federation across HA pairs?
>
HDFS federation across HA pairs, describes what I am thinking. I did want
the HA pairs to share datanodes as Ravi mentioned. So I would like to test
w/ 2 or more federated HA pairs.
>
> If the former, I think the terminology would be "spanning HDFS clusters."
>
> If the latter, it's HDFS federation[1].
>
> [1]:
>
> http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-hdfs/Federation.html
>
>
> --
> Sean
>
Re: Accumulo 1.6.0 HA namenode testing
Posted by Sean Busbey <bu...@clouderagovt.com>.
On Tue, Dec 17, 2013 at 10:44 PM, Keith Turner <ke...@deenlo.com> wrote:
> On Tue, Dec 17, 2013 at 11:39 PM, Josh Elser <jo...@gmail.com> wrote:
>
>
> I was not using the correct terminology for what I was thinking, but I am
> not sure what the correct terminology is. I do not want to be limited by
> the memory of one machine for hdfs metadata and I want HA. So would it be
> correct to say that I want to test with multiple HA nameservices? For
> example test 1.6 w/ three nameservices, where each nameservice has two HA
> namenodes. This would allow hdfs metadata to spread across the memory of
> three machines.
>
>
These HDFS nameservices are independent HDFS instances using the new
span-HDFS-instances feature in Accumulo 1.6? Or they are nameservices using
HDFS federation across HA pairs?
If the former, I think the terminology would be "spanning HDFS clusters."
If the latter, it's HDFS federation[1].
[1]:
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-hdfs/Federation.html
--
Sean
Re: Accumulo 1.6.0 HA namenode testing
Posted by Keith Turner <ke...@deenlo.com>.
On Tue, Dec 17, 2013 at 11:39 PM, Josh Elser <jo...@gmail.com> wrote:
> I've been running CI tests against 1.5.1 with HA (QJM) Namenode support
> for a few weeks now. Earlier this week I started running the same against
> 1.6.0.
>
> AFAICT, everything has been fine so far. This is with the
> hdfs-agitation.pl (causing a failover using the haadmin command every ten
> minutes) script that Sean provided a while back.
>
>
I was not using the correct terminology for what I was thinking, but I am
not sure what the correct terminology is. I do not want to be limited by
the memory of one machine for hdfs metadata and I want HA. So would it be
correct to say that I want to test with multiple HA nameservices? For
example test 1.6 w/ three nameservices, where each nameservice has two HA
namenodes. This would allow hdfs metadata to spread across the memory of
three machines.
>
> On 12/17/13, 11:32 PM, Keith Turner wrote:
>
>> Has anyone has done any HA namenode testing w/ 1.6.0?
>>
>> I have started poking at viewfs:// w/ two regular namenodes and have been
>> finding issues. I want to eventually run the test suite against Accumulo
>> 1.6.0 w/ multiple HA namenodes.
>>
>>
Re: Accumulo 1.6.0 HA namenode testing
Posted by Josh Elser <jo...@gmail.com>.
I've been running CI tests against 1.5.1 with HA (QJM) Namenode support
for a few weeks now. Earlier this week I started running the same
against 1.6.0.
AFAICT, everything has been fine so far. This is with the
hdfs-agitation.pl (causing a failover using the haadmin command every
ten minutes) script that Sean provided a while back.
On 12/17/13, 11:32 PM, Keith Turner wrote:
> Has anyone has done any HA namenode testing w/ 1.6.0?
>
> I have started poking at viewfs:// w/ two regular namenodes and have been
> finding issues. I want to eventually run the test suite against Accumulo
> 1.6.0 w/ multiple HA namenodes.
>