You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Henry Hung <YT...@winbond.com> on 2014/06/12 04:49:29 UTC
hadoop 2.2.0 HA: standby namenode generate a long list of loading
edits
Hi All,
I'm using QJM with 2 namenodes, in the active namenode, the main page's loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@ Suresh,
Can you please explain how you arrived at this?
The question just pop up in my head, because I don’t familiar with the purpose of loading edit and assume that when namenode becomes active, it will scan all the edits before activated.
I think I need to do more study about HA QJM mechanic in the future.
BTW, thank you for your assurance that nothing wrong with the long list. I will try to replicate this condition in testing environment and test it.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:32 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com>> wrote:
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Can you please explain how you arrived at this?
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Other than showing a long list of loaded edits, there is nothing to be concerned here. I agree that this is confusing and we could change this where we print only last set of loaded edits instead of entire list.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com<ma...@hortonworks.com>]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@ Suresh,
Can you please explain how you arrived at this?
The question just pop up in my head, because I don’t familiar with the purpose of loading edit and assume that when namenode becomes active, it will scan all the edits before activated.
I think I need to do more study about HA QJM mechanic in the future.
BTW, thank you for your assurance that nothing wrong with the long list. I will try to replicate this condition in testing environment and test it.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:32 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com>> wrote:
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Can you please explain how you arrived at this?
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Other than showing a long list of loaded edits, there is nothing to be concerned here. I agree that this is confusing and we could change this where we print only last set of loaded edits instead of entire list.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com<ma...@hortonworks.com>]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@ Suresh,
Can you please explain how you arrived at this?
The question just pop up in my head, because I don’t familiar with the purpose of loading edit and assume that when namenode becomes active, it will scan all the edits before activated.
I think I need to do more study about HA QJM mechanic in the future.
BTW, thank you for your assurance that nothing wrong with the long list. I will try to replicate this condition in testing environment and test it.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:32 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com>> wrote:
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Can you please explain how you arrived at this?
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Other than showing a long list of loaded edits, there is nothing to be concerned here. I agree that this is confusing and we could change this where we print only last set of loaded edits instead of entire list.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com<ma...@hortonworks.com>]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@ Suresh,
Can you please explain how you arrived at this?
The question just pop up in my head, because I don’t familiar with the purpose of loading edit and assume that when namenode becomes active, it will scan all the edits before activated.
I think I need to do more study about HA QJM mechanic in the future.
BTW, thank you for your assurance that nothing wrong with the long list. I will try to replicate this condition in testing environment and test it.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:32 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com>> wrote:
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Can you please explain how you arrived at this?
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Other than showing a long list of loaded edits, there is nothing to be concerned here. I agree that this is confusing and we could change this where we print only last set of loaded edits instead of entire list.
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com<ma...@hortonworks.com>]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com> wrote:
> @Suresh,
>
>
>
> Q1: But is this kind of behavior can cause some problem in fail over
> event? I’m afraid that standby namenode will took a long time to be active.
>
Can you please explain how you arrived at this?
>
>
> Q2: Is there a way to purge the loading edit records? Should I do restart
> on standby namenode?
>
>
>
Other than showing a long list of loaded edits, there is nothing to be
concerned here. I agree that this is confusing and we could change this
where we print only last set of loaded edits instead of entire list.
> Best regards,
>
> Henry
>
>
>
> *From:* Suresh Srinivas [mailto:suresh@hortonworks.com]
> *Sent:* Thursday, June 12, 2014 11:23 AM
> *To:* hdfs-user@hadoop.apache.org
> *Subject:* Re: hadoop 2.2.0 HA: standby namenode generate a long list of
> loading edits
>
>
>
> Henry,
>
>
>
> I suspect this is what is happening. On active namenode, oncethe existing
> set of editlogs during startup are loaded, it becomes active and from then
> it has no need to load any more edits. It only generates edits. On the
> other hand, standby namenode not only loads the edits during startup, it
> also continuously loads the edits being generated by the active. Hence the
> difference.
>
>
>
> Regards,
>
> Suresh
>
>
>
> On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
>
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
>
> ------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
>
>
>
>
> --
> http://hortonworks.com/download/
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com> wrote:
> @Suresh,
>
>
>
> Q1: But is this kind of behavior can cause some problem in fail over
> event? I’m afraid that standby namenode will took a long time to be active.
>
Can you please explain how you arrived at this?
>
>
> Q2: Is there a way to purge the loading edit records? Should I do restart
> on standby namenode?
>
>
>
Other than showing a long list of loaded edits, there is nothing to be
concerned here. I agree that this is confusing and we could change this
where we print only last set of loaded edits instead of entire list.
> Best regards,
>
> Henry
>
>
>
> *From:* Suresh Srinivas [mailto:suresh@hortonworks.com]
> *Sent:* Thursday, June 12, 2014 11:23 AM
> *To:* hdfs-user@hadoop.apache.org
> *Subject:* Re: hadoop 2.2.0 HA: standby namenode generate a long list of
> loading edits
>
>
>
> Henry,
>
>
>
> I suspect this is what is happening. On active namenode, oncethe existing
> set of editlogs during startup are loaded, it becomes active and from then
> it has no need to load any more edits. It only generates edits. On the
> other hand, standby namenode not only loads the edits during startup, it
> also continuously loads the edits being generated by the active. Hence the
> difference.
>
>
>
> Regards,
>
> Suresh
>
>
>
> On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
>
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
>
> ------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
>
>
>
>
> --
> http://hortonworks.com/download/
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com> wrote:
> @Suresh,
>
>
>
> Q1: But is this kind of behavior can cause some problem in fail over
> event? I’m afraid that standby namenode will took a long time to be active.
>
Can you please explain how you arrived at this?
>
>
> Q2: Is there a way to purge the loading edit records? Should I do restart
> on standby namenode?
>
>
>
Other than showing a long list of loaded edits, there is nothing to be
concerned here. I agree that this is confusing and we could change this
where we print only last set of loaded edits instead of entire list.
> Best regards,
>
> Henry
>
>
>
> *From:* Suresh Srinivas [mailto:suresh@hortonworks.com]
> *Sent:* Thursday, June 12, 2014 11:23 AM
> *To:* hdfs-user@hadoop.apache.org
> *Subject:* Re: hadoop 2.2.0 HA: standby namenode generate a long list of
> loading edits
>
>
>
> Henry,
>
>
>
> I suspect this is what is happening. On active namenode, oncethe existing
> set of editlogs during startup are loaded, it becomes active and from then
> it has no need to load any more edits. It only generates edits. On the
> other hand, standby namenode not only loads the edits during startup, it
> also continuously loads the edits being generated by the active. Hence the
> difference.
>
>
>
> Regards,
>
> Suresh
>
>
>
> On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
>
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
>
> ------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
>
>
>
>
> --
> http://hortonworks.com/download/
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
On Wed, Jun 11, 2014 at 8:27 PM, Henry Hung <YT...@winbond.com> wrote:
> @Suresh,
>
>
>
> Q1: But is this kind of behavior can cause some problem in fail over
> event? I’m afraid that standby namenode will took a long time to be active.
>
Can you please explain how you arrived at this?
>
>
> Q2: Is there a way to purge the loading edit records? Should I do restart
> on standby namenode?
>
>
>
Other than showing a long list of loaded edits, there is nothing to be
concerned here. I agree that this is confusing and we could change this
where we print only last set of loaded edits instead of entire list.
> Best regards,
>
> Henry
>
>
>
> *From:* Suresh Srinivas [mailto:suresh@hortonworks.com]
> *Sent:* Thursday, June 12, 2014 11:23 AM
> *To:* hdfs-user@hadoop.apache.org
> *Subject:* Re: hadoop 2.2.0 HA: standby namenode generate a long list of
> loading edits
>
>
>
> Henry,
>
>
>
> I suspect this is what is happening. On active namenode, oncethe existing
> set of editlogs during startup are loaded, it becomes active and from then
> it has no need to load any more edits. It only generates edits. On the
> other hand, standby namenode not only loads the edits during startup, it
> also continuously loads the edits being generated by the active. Hence the
> difference.
>
>
>
> Regards,
>
> Suresh
>
>
>
> On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
>
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
>
> ------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
>
>
>
>
> --
> http://hortonworks.com/download/
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: hadoop 2.2.0 HA: standby namenode generate a long list of
loading edits
Posted by Henry Hung <YT...@winbond.com>.
@Suresh,
Q1: But is this kind of behavior can cause some problem in fail over event? I’m afraid that standby namenode will took a long time to be active.
Q2: Is there a way to purge the loading edit records? Should I do restart on standby namenode?
Best regards,
Henry
From: Suresh Srinivas [mailto:suresh@hortonworks.com]
Sent: Thursday, June 12, 2014 11:23 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Henry,
I suspect this is what is happening. On active namenode, oncethe existing set of editlogs during startup are loaded, it becomes active and from then it has no need to load any more edits. It only generates edits. On the other hand, standby namenode not only loads the edits during startup, it also continuously loads the edits being generated by the active. Hence the difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com>> wrote:
Hi All,
I’m using QJM with 2 namenodes, in the active namenode, the main page’s loading edits panel only show 10 records, but in standby namenode, the loading edits panel show a lot more records, never count it, but I think it has > 100 records.
Is this a problem?
Here I provide some of the data:
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e (0/0)
100.00%
0sec
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
http://hortonworks.com/download/
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
Henry,
I suspect this is what is happening. On active namenode, oncethe existing
set of editlogs during startup are loaded, it becomes active and from then
it has no need to load any more edits. It only generates edits. On the
other hand, standby namenode not only loads the edits during startup, it
also continuously loads the edits being generated by the active. Hence the
difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
Henry,
I suspect this is what is happening. On active namenode, oncethe existing
set of editlogs during startup are loaded, it becomes active and from then
it has no need to load any more edits. It only generates edits. On the
other hand, standby namenode not only loads the edits during startup, it
also continuously loads the edits being generated by the active. Hence the
difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
Henry,
I suspect this is what is happening. On active namenode, oncethe existing
set of editlogs during startup are loaded, it becomes active and from then
it has no need to load any more edits. It only generates edits. On the
other hand, standby namenode not only loads the edits during startup, it
also continuously loads the edits being generated by the active. Hence the
difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hadoop 2.2.0 HA: standby namenode generate a long list of loading edits
Posted by Suresh Srinivas <su...@hortonworks.com>.
Henry,
I suspect this is what is happening. On active namenode, oncethe existing
set of editlogs during startup are loaded, it becomes active and from then
it has no need to load any more edits. It only generates edits. On the
other hand, standby namenode not only loads the edits during startup, it
also continuously loads the edits being generated by the active. Hence the
difference.
Regards,
Suresh
On Wed, Jun 11, 2014 at 7:49 PM, Henry Hung <YT...@winbond.com> wrote:
> Hi All,
>
>
>
> I’m using QJM with 2 namenodes, in the active namenode, the main page’s
> loading edits panel only show 10 records, but in standby namenode, the
> loading edits panel show a lot more records, never count it, but I think it
> has > 100 records.
>
> Is this a problem?
>
>
>
> Here I provide some of the data:
>
>
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000080&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000361&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10000830&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000140&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10001638&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002099&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10002359&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000332&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000421&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005210&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005529&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=1000577&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005831&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10005951&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006089&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006154&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006291&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
> http://fchdgw1.ctfab.com:8480/getJournal?jid=hadoop_prod&segmentTxId=10006482&storageInfo=-47%3A1313059004%3A1395811267413%3ACID-9e1b67b3-8190-4652-b34a-210212a50a9e
> (0/0)
>
> 100.00%
>
> 0sec
>
>
>
>
>
> Best regards,
>
> Henry
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.