You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Akmal Abbasov <ak...@icloud.com> on 2015/07/07 10:23:10 UTC
Large number of application attempts
Hi all,
I have HBase cluster.
I run a cron job each hour to create a snapshot of tables in HBase and export these snapshots to another HBase cluster for backup.
But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
What are these znodes?
Are they failed yarn applications?
How I need to handle them?
Thank you.
Akmal
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
bq. Can I manually remove this znodes, or do I need to do this through
yarn command line?
Which version of hadoop you are using ? If the version is 2.6 or after
2.6, you could do yarn resourcemanager -format-state-store which will
format all the rmstore (basically delete path
ha/testhbase1/rmstore/ZKRMStateRoot and all the subdir/files)
Thanks
Xuan Gong
On 7/8/15, 5:45 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>Can I manually remove this znodes, or do I need to do this through yarn
>command line?
>
>Thank you.
>
>> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com>
>>wrote:
>>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>>export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
bq. Can I manually remove this znodes, or do I need to do this through
yarn command line?
Which version of hadoop you are using ? If the version is 2.6 or after
2.6, you could do yarn resourcemanager -format-state-store which will
format all the rmstore (basically delete path
ha/testhbase1/rmstore/ZKRMStateRoot and all the subdir/files)
Thanks
Xuan Gong
On 7/8/15, 5:45 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>Can I manually remove this znodes, or do I need to do this through yarn
>command line?
>
>Thank you.
>
>> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com>
>>wrote:
>>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>>export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
bq. Can I manually remove this znodes, or do I need to do this through
yarn command line?
Which version of hadoop you are using ? If the version is 2.6 or after
2.6, you could do yarn resourcemanager -format-state-store which will
format all the rmstore (basically delete path
ha/testhbase1/rmstore/ZKRMStateRoot and all the subdir/files)
Thanks
Xuan Gong
On 7/8/15, 5:45 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>Can I manually remove this znodes, or do I need to do this through yarn
>command line?
>
>Thank you.
>
>> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com>
>>wrote:
>>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>>export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
bq. Can I manually remove this znodes, or do I need to do this through
yarn command line?
Which version of hadoop you are using ? If the version is 2.6 or after
2.6, you could do yarn resourcemanager -format-state-store which will
format all the rmstore (basically delete path
ha/testhbase1/rmstore/ZKRMStateRoot and all the subdir/files)
Thanks
Xuan Gong
On 7/8/15, 5:45 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>Can I manually remove this znodes, or do I need to do this through yarn
>command line?
>
>Thank you.
>
>> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com>
>>wrote:
>>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>>export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
Can I manually remove this znodes, or do I need to do this through yarn command line?
Thank you.
> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com> wrote:
>
> Hi all,
> I have HBase cluster.
> I run a cron job each hour to create a snapshot of tables in HBase and export these snapshots to another HBase cluster for backup.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
> What are these znodes?
> Are they failed yarn applications?
> How I need to handle them?
>
> Thank you.
> Akmal
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
Can I manually remove this znodes, or do I need to do this through yarn command line?
Thank you.
> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com> wrote:
>
> Hi all,
> I have HBase cluster.
> I run a cron job each hour to create a snapshot of tables in HBase and export these snapshots to another HBase cluster for backup.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
> What are these znodes?
> Are they failed yarn applications?
> How I need to handle them?
>
> Thank you.
> Akmal
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi,
> Are you running a lots of apps ?
I have 5 tables in HBase, and I run a cron job each hour which will create a snapshot of each table,
and tries to export it to a remote HBase cluster.
> Or could you, maybe, share some of file names of those znodes, please ?
they are all have the name format as follows ‘application_xxxxxxxxxxxxx_xxxx’
Thank you.
> On 07 Jul 2015, at 19:00, Xuan Gong <xg...@hortonworks.com> wrote:
>
> Hey, Akmal:
> Are you running a lots of apps ? Or could you, maybe, share some of
> file names of those znodes, please ?
>
>
> Thanks
>
> Xuan Gong
> On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>> export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>> /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi,
> Are you running a lots of apps ?
I have 5 tables in HBase, and I run a cron job each hour which will create a snapshot of each table,
and tries to export it to a remote HBase cluster.
> Or could you, maybe, share some of file names of those znodes, please ?
they are all have the name format as follows ‘application_xxxxxxxxxxxxx_xxxx’
Thank you.
> On 07 Jul 2015, at 19:00, Xuan Gong <xg...@hortonworks.com> wrote:
>
> Hey, Akmal:
> Are you running a lots of apps ? Or could you, maybe, share some of
> file names of those znodes, please ?
>
>
> Thanks
>
> Xuan Gong
> On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>> export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>> /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi,
> Are you running a lots of apps ?
I have 5 tables in HBase, and I run a cron job each hour which will create a snapshot of each table,
and tries to export it to a remote HBase cluster.
> Or could you, maybe, share some of file names of those znodes, please ?
they are all have the name format as follows ‘application_xxxxxxxxxxxxx_xxxx’
Thank you.
> On 07 Jul 2015, at 19:00, Xuan Gong <xg...@hortonworks.com> wrote:
>
> Hey, Akmal:
> Are you running a lots of apps ? Or could you, maybe, share some of
> file names of those znodes, please ?
>
>
> Thanks
>
> Xuan Gong
> On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>> export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>> /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
Hi,
> Are you running a lots of apps ?
I have 5 tables in HBase, and I run a cron job each hour which will create a snapshot of each table,
and tries to export it to a remote HBase cluster.
> Or could you, maybe, share some of file names of those znodes, please ?
they are all have the name format as follows ‘application_xxxxxxxxxxxxx_xxxx’
Thank you.
> On 07 Jul 2015, at 19:00, Xuan Gong <xg...@hortonworks.com> wrote:
>
> Hey, Akmal:
> Are you running a lots of apps ? Or could you, maybe, share some of
> file names of those znodes, please ?
>
>
> Thanks
>
> Xuan Gong
> On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>
>> Hi all,
>> I have HBase cluster.
>> I run a cron job each hour to create a snapshot of tables in HBase and
>> export these snapshots to another HBase cluster for backup.
>> But I¹ve noticed that there are ~4000 znodes in
>> /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>> What are these znodes?
>> Are they failed yarn applications?
>> How I need to handle them?
>>
>> Thank you.
>> Akmal
>
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Akmal:
Are you running a lots of apps ? Or could you, maybe, share some of
file names of those znodes, please ?
Thanks
Xuan Gong
On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>Hi all,
>I have HBase cluster.
>I run a cron job each hour to create a snapshot of tables in HBase and
>export these snapshots to another HBase cluster for backup.
>But I¹ve noticed that there are ~4000 znodes in
>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>What are these znodes?
>Are they failed yarn applications?
>How I need to handle them?
>
>Thank you.
>Akmal
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Akmal:
Are you running a lots of apps ? Or could you, maybe, share some of
file names of those znodes, please ?
Thanks
Xuan Gong
On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>Hi all,
>I have HBase cluster.
>I run a cron job each hour to create a snapshot of tables in HBase and
>export these snapshots to another HBase cluster for backup.
>But I¹ve noticed that there are ~4000 znodes in
>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>What are these znodes?
>Are they failed yarn applications?
>How I need to handle them?
>
>Thank you.
>Akmal
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Akmal:
Are you running a lots of apps ? Or could you, maybe, share some of
file names of those znodes, please ?
Thanks
Xuan Gong
On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>Hi all,
>I have HBase cluster.
>I run a cron job each hour to create a snapshot of tables in HBase and
>export these snapshots to another HBase cluster for backup.
>But I¹ve noticed that there are ~4000 znodes in
>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>What are these znodes?
>Are they failed yarn applications?
>How I need to handle them?
>
>Thank you.
>Akmal
Re: Large number of application attempts
Posted by Xuan Gong <xg...@hortonworks.com>.
Hey, Akmal:
Are you running a lots of apps ? Or could you, maybe, share some of
file names of those znodes, please ?
Thanks
Xuan Gong
On 7/7/15, 1:23 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>Hi all,
>I have HBase cluster.
>I run a cron job each hour to create a snapshot of tables in HBase and
>export these snapshots to another HBase cluster for backup.
>But I¹ve noticed that there are ~4000 znodes in
>/hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
>What are these znodes?
>Are they failed yarn applications?
>How I need to handle them?
>
>Thank you.
>Akmal
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
Can I manually remove this znodes, or do I need to do this through yarn command line?
Thank you.
> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com> wrote:
>
> Hi all,
> I have HBase cluster.
> I run a cron job each hour to create a snapshot of tables in HBase and export these snapshots to another HBase cluster for backup.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
> What are these znodes?
> Are they failed yarn applications?
> How I need to handle them?
>
> Thank you.
> Akmal
Re: Large number of application attempts
Posted by Akmal Abbasov <ak...@icloud.com>.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
Can I manually remove this znodes, or do I need to do this through yarn command line?
Thank you.
> On 07 Jul 2015, at 10:23, Akmal Abbasov <ak...@icloud.com> wrote:
>
> Hi all,
> I have HBase cluster.
> I run a cron job each hour to create a snapshot of tables in HBase and export these snapshots to another HBase cluster for backup.
> But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper.
> What are these znodes?
> Are they failed yarn applications?
> How I need to handle them?
>
> Thank you.
> Akmal