You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Henry Hung <YT...@winbond.com> on 2013/04/17 08:02:00 UTC
Adding new name node location
Hi Everyone,
I'm using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
答复: Adding new name node location
Posted by jiangchaocai <ji...@haiyisoft.com>.
NOT authoritative:
The “new name node” will get the full copy of the name node metadata when you restart the dfs service.
During the hdfs running , same changes will be made on the name node and the backup folder.
In fact, it only contains 2 files: FSImage, EditLog
Thanks,
John
发件人: 李洪忠 [mailto:lhztop@hotmail.com]
发送时间: 2013年4月17日 15:18
收件人: user@hadoop.apache.org
抄送: Henry Hung
主题: Re: Adding new name node location
modify conf file and restart name node is the best way. you needn't restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
Hi Everyone,
I’m using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
答复: Adding new name node location
Posted by jiangchaocai <ji...@haiyisoft.com>.
NOT authoritative:
The “new name node” will get the full copy of the name node metadata when you restart the dfs service.
During the hdfs running , same changes will be made on the name node and the backup folder.
In fact, it only contains 2 files: FSImage, EditLog
Thanks,
John
发件人: 李洪忠 [mailto:lhztop@hotmail.com]
发送时间: 2013年4月17日 15:18
收件人: user@hadoop.apache.org
抄送: Henry Hung
主题: Re: Adding new name node location
modify conf file and restart name node is the best way. you needn't restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
Hi Everyone,
I’m using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
答复: Adding new name node location
Posted by jiangchaocai <ji...@haiyisoft.com>.
NOT authoritative:
The “new name node” will get the full copy of the name node metadata when you restart the dfs service.
During the hdfs running , same changes will be made on the name node and the backup folder.
In fact, it only contains 2 files: FSImage, EditLog
Thanks,
John
发件人: 李洪忠 [mailto:lhztop@hotmail.com]
发送时间: 2013年4月17日 15:18
收件人: user@hadoop.apache.org
抄送: Henry Hung
主题: Re: Adding new name node location
modify conf file and restart name node is the best way. you needn't restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
Hi Everyone,
I’m using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
答复: Adding new name node location
Posted by jiangchaocai <ji...@haiyisoft.com>.
NOT authoritative:
The “new name node” will get the full copy of the name node metadata when you restart the dfs service.
During the hdfs running , same changes will be made on the name node and the backup folder.
In fact, it only contains 2 files: FSImage, EditLog
Thanks,
John
发件人: 李洪忠 [mailto:lhztop@hotmail.com]
发送时间: 2013年4月17日 15:18
收件人: user@hadoop.apache.org
抄送: Henry Hung
主题: Re: Adding new name node location
modify conf file and restart name node is the best way. you needn't restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
Hi Everyone,
I’m using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
_____
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
Re: Adding new name node location
Posted by 李洪忠 <lh...@hotmail.com>.
modify conf file and restart name node is the best way. you needn't
restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
>
> Hi Varun Kumar,
>
> Could you be more elaborate about how the new changes being made to
> new name node?
>
> The scenario in my mind is:
>
> Suppose old name node metadata contains 100 hdfs files.
>
> Then I restart by using stop-dfs, change config and start-dfs.
>
> Hadoop will automatically create new name node directory in /backup.
>
> Now I have one old name node that have metadata of 100 hdfs files and
> new name node that have none metadata.
>
> I put a new file into hdfs, the old name node will have 101 and new
> name node will have 1.
>
> Best regards,
>
> Henry
>
> *From:*varun kumar [mailto:varun.uid@gmail.com]
> *Sent:* Wednesday, April 17, 2013 2:18 PM
> *To:* user
> *Cc:* MA11 YTHung1
> *Subject:* Re: Adding new name node location
>
> Hi Henry,
>
> As per your mail Point number 1 is correct.
>
> After doing these changes metadata will be written in the new partition.
>
> Regards,
>
> Varun Kumar.P
>
> On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YTHung1@winbond.com
> <ma...@winbond.com>> wrote:
>
> Hi Everyone,
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Now I want to protect my name node files by changing the configuration to:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Where /backup is another mount point. This /backup can be another disk
> or from another NFS server.
>
> My question are:
>
> 1.Is my procedure correct: do stop-dfs.sh then modify conf, and last
> start-dfs.sh?
>
> 2.If answer to no 1 is no, then could you provide the correct procedure?
>
> 3.Would the new name node files will auto copy the original name node
> files?
>
> Best regards,
>
> Henry
>
> ------------------------------------------------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
>
>
>
> --
>
> Regards,
>
> Varun Kumar.P
>
>
> ------------------------------------------------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
Re: Adding new name node location
Posted by 李洪忠 <lh...@hotmail.com>.
modify conf file and restart name node is the best way. you needn't
restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
>
> Hi Varun Kumar,
>
> Could you be more elaborate about how the new changes being made to
> new name node?
>
> The scenario in my mind is:
>
> Suppose old name node metadata contains 100 hdfs files.
>
> Then I restart by using stop-dfs, change config and start-dfs.
>
> Hadoop will automatically create new name node directory in /backup.
>
> Now I have one old name node that have metadata of 100 hdfs files and
> new name node that have none metadata.
>
> I put a new file into hdfs, the old name node will have 101 and new
> name node will have 1.
>
> Best regards,
>
> Henry
>
> *From:*varun kumar [mailto:varun.uid@gmail.com]
> *Sent:* Wednesday, April 17, 2013 2:18 PM
> *To:* user
> *Cc:* MA11 YTHung1
> *Subject:* Re: Adding new name node location
>
> Hi Henry,
>
> As per your mail Point number 1 is correct.
>
> After doing these changes metadata will be written in the new partition.
>
> Regards,
>
> Varun Kumar.P
>
> On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YTHung1@winbond.com
> <ma...@winbond.com>> wrote:
>
> Hi Everyone,
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Now I want to protect my name node files by changing the configuration to:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Where /backup is another mount point. This /backup can be another disk
> or from another NFS server.
>
> My question are:
>
> 1.Is my procedure correct: do stop-dfs.sh then modify conf, and last
> start-dfs.sh?
>
> 2.If answer to no 1 is no, then could you provide the correct procedure?
>
> 3.Would the new name node files will auto copy the original name node
> files?
>
> Best regards,
>
> Henry
>
> ------------------------------------------------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
>
>
>
> --
>
> Regards,
>
> Varun Kumar.P
>
>
> ------------------------------------------------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
Re: Adding new name node location
Posted by 李洪忠 <lh...@hotmail.com>.
modify conf file and restart name node is the best way. you needn't
restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
>
> Hi Varun Kumar,
>
> Could you be more elaborate about how the new changes being made to
> new name node?
>
> The scenario in my mind is:
>
> Suppose old name node metadata contains 100 hdfs files.
>
> Then I restart by using stop-dfs, change config and start-dfs.
>
> Hadoop will automatically create new name node directory in /backup.
>
> Now I have one old name node that have metadata of 100 hdfs files and
> new name node that have none metadata.
>
> I put a new file into hdfs, the old name node will have 101 and new
> name node will have 1.
>
> Best regards,
>
> Henry
>
> *From:*varun kumar [mailto:varun.uid@gmail.com]
> *Sent:* Wednesday, April 17, 2013 2:18 PM
> *To:* user
> *Cc:* MA11 YTHung1
> *Subject:* Re: Adding new name node location
>
> Hi Henry,
>
> As per your mail Point number 1 is correct.
>
> After doing these changes metadata will be written in the new partition.
>
> Regards,
>
> Varun Kumar.P
>
> On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YTHung1@winbond.com
> <ma...@winbond.com>> wrote:
>
> Hi Everyone,
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Now I want to protect my name node files by changing the configuration to:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Where /backup is another mount point. This /backup can be another disk
> or from another NFS server.
>
> My question are:
>
> 1.Is my procedure correct: do stop-dfs.sh then modify conf, and last
> start-dfs.sh?
>
> 2.If answer to no 1 is no, then could you provide the correct procedure?
>
> 3.Would the new name node files will auto copy the original name node
> files?
>
> Best regards,
>
> Henry
>
> ------------------------------------------------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
>
>
>
> --
>
> Regards,
>
> Varun Kumar.P
>
>
> ------------------------------------------------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
Re: Adding new name node location
Posted by 李洪忠 <lh...@hotmail.com>.
modify conf file and restart name node is the best way. you needn't
restart the cluster DFS.
the files in /backup and /home are the same.
于 2013/4/17 14:38, Henry Hung 写道:
>
> Hi Varun Kumar,
>
> Could you be more elaborate about how the new changes being made to
> new name node?
>
> The scenario in my mind is:
>
> Suppose old name node metadata contains 100 hdfs files.
>
> Then I restart by using stop-dfs, change config and start-dfs.
>
> Hadoop will automatically create new name node directory in /backup.
>
> Now I have one old name node that have metadata of 100 hdfs files and
> new name node that have none metadata.
>
> I put a new file into hdfs, the old name node will have 101 and new
> name node will have 1.
>
> Best regards,
>
> Henry
>
> *From:*varun kumar [mailto:varun.uid@gmail.com]
> *Sent:* Wednesday, April 17, 2013 2:18 PM
> *To:* user
> *Cc:* MA11 YTHung1
> *Subject:* Re: Adding new name node location
>
> Hi Henry,
>
> As per your mail Point number 1 is correct.
>
> After doing these changes metadata will be written in the new partition.
>
> Regards,
>
> Varun Kumar.P
>
> On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YTHung1@winbond.com
> <ma...@winbond.com>> wrote:
>
> Hi Everyone,
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Now I want to protect my name node files by changing the configuration to:
>
> <property>
>
> <name>dfs.name.dir</name>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
>
> </property>
>
> Where /backup is another mount point. This /backup can be another disk
> or from another NFS server.
>
> My question are:
>
> 1.Is my procedure correct: do stop-dfs.sh then modify conf, and last
> start-dfs.sh?
>
> 2.If answer to no 1 is no, then could you provide the correct procedure?
>
> 3.Would the new name node files will auto copy the original name node
> files?
>
> Best regards,
>
> Henry
>
> ------------------------------------------------------------------------
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
>
>
>
> --
>
> Regards,
>
> Varun Kumar.P
>
>
> ------------------------------------------------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original
> sender of this email. If you are not the addressee indicated in this
> email or are not responsible for delivery of the email to such a
> person, please kindly reply to the sender indicating this fact and
> delete all copies of it from your computer and network server
> immediately. Your cooperation is highly appreciated. It is advised
> that any unauthorized use of confidential information of Winbond is
> strictly prohibited; and any information in this email irrelevant to
> the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
RE: Adding new name node location
Posted by Henry Hung <YT...@winbond.com>.
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com>> wrote:
Hi Everyone,
I'm using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: Adding new name node location
Posted by Henry Hung <YT...@winbond.com>.
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com>> wrote:
Hi Everyone,
I'm using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: Adding new name node location
Posted by Henry Hung <YT...@winbond.com>.
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com>> wrote:
Hi Everyone,
I'm using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
RE: Adding new name node location
Posted by Henry Hung <YT...@winbond.com>.
Hi Varun Kumar,
Could you be more elaborate about how the new changes being made to new name node?
The scenario in my mind is:
Suppose old name node metadata contains 100 hdfs files.
Then I restart by using stop-dfs, change config and start-dfs.
Hadoop will automatically create new name node directory in /backup.
Now I have one old name node that have metadata of 100 hdfs files and new name node that have none metadata.
I put a new file into hdfs, the old name node will have 101 and new name node will have 1.
Best regards,
Henry
From: varun kumar [mailto:varun.uid@gmail.com]
Sent: Wednesday, April 17, 2013 2:18 PM
To: user
Cc: MA11 YTHung1
Subject: Re: Adding new name node location
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com>> wrote:
Hi Everyone,
I'm using Hadoop 1.0.4 and only define 1 location for name node files, like this:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode</value>
</property>
Now I want to protect my name node files by changing the configuration to:
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
</property>
Where /backup is another mount point. This /backup can be another disk or from another NFS server.
My question are:
1. Is my procedure correct: do stop-dfs.sh then modify conf, and last start-dfs.sh?
2. If answer to no 1 is no, then could you provide the correct procedure?
3. Would the new name node files will auto copy the original name node files?
Best regards,
Henry
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
--
Regards,
Varun Kumar.P
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
Re: Adding new name node location
Posted by varun kumar <va...@gmail.com>.
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
> Hi Everyone,****
>
> ** **
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
> <value>/home/hadoop/hadoop-data/namenode</value>****
>
> </property>****
>
> ** **
>
> Now I want to protect my name node files by changing the configuration to:
> ****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
> ****
>
> </property>****
>
> ** **
>
> Where /backup is another mount point. This /backup can be another disk or
> from another NFS server.****
>
> ** **
>
> My question are:****
>
> **1. **Is my procedure correct: do stop-dfs.sh then modify conf,
> and last start-dfs.sh?****
>
> **2. **If answer to no 1 is no, then could you provide the correct
> procedure?****
>
> **3. **Would the new name node files will auto copy the original
> name node files?****
>
> ** **
>
> Best regards,****
>
> Henry****
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
Regards,
Varun Kumar.P
Re: Adding new name node location
Posted by varun kumar <va...@gmail.com>.
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
> Hi Everyone,****
>
> ** **
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
> <value>/home/hadoop/hadoop-data/namenode</value>****
>
> </property>****
>
> ** **
>
> Now I want to protect my name node files by changing the configuration to:
> ****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
> ****
>
> </property>****
>
> ** **
>
> Where /backup is another mount point. This /backup can be another disk or
> from another NFS server.****
>
> ** **
>
> My question are:****
>
> **1. **Is my procedure correct: do stop-dfs.sh then modify conf,
> and last start-dfs.sh?****
>
> **2. **If answer to no 1 is no, then could you provide the correct
> procedure?****
>
> **3. **Would the new name node files will auto copy the original
> name node files?****
>
> ** **
>
> Best regards,****
>
> Henry****
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
Regards,
Varun Kumar.P
Re: Adding new name node location
Posted by varun kumar <va...@gmail.com>.
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
> Hi Everyone,****
>
> ** **
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
> <value>/home/hadoop/hadoop-data/namenode</value>****
>
> </property>****
>
> ** **
>
> Now I want to protect my name node files by changing the configuration to:
> ****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
> ****
>
> </property>****
>
> ** **
>
> Where /backup is another mount point. This /backup can be another disk or
> from another NFS server.****
>
> ** **
>
> My question are:****
>
> **1. **Is my procedure correct: do stop-dfs.sh then modify conf,
> and last start-dfs.sh?****
>
> **2. **If answer to no 1 is no, then could you provide the correct
> procedure?****
>
> **3. **Would the new name node files will auto copy the original
> name node files?****
>
> ** **
>
> Best regards,****
>
> Henry****
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
Regards,
Varun Kumar.P
Re: Adding new name node location
Posted by varun kumar <va...@gmail.com>.
Hi Henry,
As per your mail Point number 1 is correct.
After doing these changes metadata will be written in the new partition.
Regards,
Varun Kumar.P
On Wed, Apr 17, 2013 at 11:32 AM, Henry Hung <YT...@winbond.com> wrote:
> Hi Everyone,****
>
> ** **
>
> I’m using Hadoop 1.0.4 and only define 1 location for name node files,
> like this:****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
> <value>/home/hadoop/hadoop-data/namenode</value>****
>
> </property>****
>
> ** **
>
> Now I want to protect my name node files by changing the configuration to:
> ****
>
> <property>****
>
> <name>dfs.name.dir</name>****
>
>
> <value>/home/hadoop/hadoop-data/namenode,/backup/hadoop/hadoop-data/namenode</value>
> ****
>
> </property>****
>
> ** **
>
> Where /backup is another mount point. This /backup can be another disk or
> from another NFS server.****
>
> ** **
>
> My question are:****
>
> **1. **Is my procedure correct: do stop-dfs.sh then modify conf,
> and last start-dfs.sh?****
>
> **2. **If answer to no 1 is no, then could you provide the correct
> procedure?****
>
> **3. **Would the new name node files will auto copy the original
> name node files?****
>
> ** **
>
> Best regards,****
>
> Henry****
>
> ------------------------------
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>
--
Regards,
Varun Kumar.P