You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Geelong Yao <ge...@gmail.com> on 2013/04/24 04:17:47 UTC
How to make the setting changes works
Hi
Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
I have met a issue after I change the content of hdfs-site.xml to add
another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
value, /sda is the new one
<property>
<name>data.dfs.dir</name>
<value>/usr/hadoop/tmp/dfs/data,/sda</value>
</property>
The permission of /sda has been changed.
[image: 内嵌图片 1]
First I stop the cluster with stop-all.sh and replace the old hdfs-site.xml
with new ones,then I restart the cluster with start-all.sh,but I found that
datanode did not start
--
>From Good To Great
Re: How to make the setting changes works
Posted by Harsh J <ha...@cloudera.com>.
Did you end up reformatting the NameNode? Thats one reason why DNs will
refuse to work. You need to clear out the dfs.data.dir directories for it
to work again. Note that you shouldn't reformat the NN ever, unless it is
your intention to wipe away all data and begin all over again.
On Wed, Apr 24, 2013 at 7:47 AM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: 内嵌图片 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
--
Harsh J
Re: How to make the setting changes works
Posted by Harsh J <ha...@cloudera.com>.
Did you end up reformatting the NameNode? Thats one reason why DNs will
refuse to work. You need to clear out the dfs.data.dir directories for it
to work again. Note that you shouldn't reformat the NN ever, unless it is
your intention to wipe away all data and begin all over again.
On Wed, Apr 24, 2013 at 7:47 AM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: 内嵌图片 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
--
Harsh J
Re: How to make the setting changes works
Posted by 姚吉龙 <ge...@gmail.com>.
I have already give full permission for that dir.I think this maybe caused by reformatting the NN.this will lead to different Id of NN and DN
—
Sent from Mailbox for iPhone
On Thu, Apr 25, 2013 at 12:41 AM, Robert Molina <rm...@hortonworks.com>
wrote:
> Hi Geelong,
> What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
> recall correctly, I believe it should be hdfs:hadoop, where the owner has
> full permissions and the group can read and execute. Does /var/log for the
> datanode show any errors?
> Regards,
> Robert
> *
> Robert Molina
> Hortonworks, Inc.
> Support Engineer
> Phone: 408.645.7062
> <http://hortonworks.com/download/>
> *
> On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
>> Hi
>>
>> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
>> I have met a issue after I change the content of hdfs-site.xml to add
>> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
>> value, /sda is the new one
>>
>> <property>
>> <name>data.dfs.dir</name>
>> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
>> </property>
>> The permission of /sda has been changed.
>>
>> [image: 内嵌图片 1]
>>
>> First I stop the cluster with stop-all.sh and replace the old
>> hdfs-site.xml with new ones,then I restart the cluster with
>> start-all.sh,but I found that datanode did not start
>>
>> --
>> From Good To Great
>>
Re: How to make the setting changes works
Posted by 姚吉龙 <ge...@gmail.com>.
I have already give full permission for that dir.I think this maybe caused by reformatting the NN.this will lead to different Id of NN and DN
—
Sent from Mailbox for iPhone
On Thu, Apr 25, 2013 at 12:41 AM, Robert Molina <rm...@hortonworks.com>
wrote:
> Hi Geelong,
> What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
> recall correctly, I believe it should be hdfs:hadoop, where the owner has
> full permissions and the group can read and execute. Does /var/log for the
> datanode show any errors?
> Regards,
> Robert
> *
> Robert Molina
> Hortonworks, Inc.
> Support Engineer
> Phone: 408.645.7062
> <http://hortonworks.com/download/>
> *
> On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
>> Hi
>>
>> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
>> I have met a issue after I change the content of hdfs-site.xml to add
>> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
>> value, /sda is the new one
>>
>> <property>
>> <name>data.dfs.dir</name>
>> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
>> </property>
>> The permission of /sda has been changed.
>>
>> [image: 内嵌图片 1]
>>
>> First I stop the cluster with stop-all.sh and replace the old
>> hdfs-site.xml with new ones,then I restart the cluster with
>> start-all.sh,but I found that datanode did not start
>>
>> --
>> From Good To Great
>>
Re: How to make the setting changes works
Posted by 姚吉龙 <ge...@gmail.com>.
I have already give full permission for that dir.I think this maybe caused by reformatting the NN.this will lead to different Id of NN and DN
—
Sent from Mailbox for iPhone
On Thu, Apr 25, 2013 at 12:41 AM, Robert Molina <rm...@hortonworks.com>
wrote:
> Hi Geelong,
> What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
> recall correctly, I believe it should be hdfs:hadoop, where the owner has
> full permissions and the group can read and execute. Does /var/log for the
> datanode show any errors?
> Regards,
> Robert
> *
> Robert Molina
> Hortonworks, Inc.
> Support Engineer
> Phone: 408.645.7062
> <http://hortonworks.com/download/>
> *
> On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
>> Hi
>>
>> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
>> I have met a issue after I change the content of hdfs-site.xml to add
>> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
>> value, /sda is the new one
>>
>> <property>
>> <name>data.dfs.dir</name>
>> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
>> </property>
>> The permission of /sda has been changed.
>>
>> [image: 内嵌图片 1]
>>
>> First I stop the cluster with stop-all.sh and replace the old
>> hdfs-site.xml with new ones,then I restart the cluster with
>> start-all.sh,but I found that datanode did not start
>>
>> --
>> From Good To Great
>>
Re: How to make the setting changes works
Posted by 姚吉龙 <ge...@gmail.com>.
I have already give full permission for that dir.I think this maybe caused by reformatting the NN.this will lead to different Id of NN and DN
—
Sent from Mailbox for iPhone
On Thu, Apr 25, 2013 at 12:41 AM, Robert Molina <rm...@hortonworks.com>
wrote:
> Hi Geelong,
> What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
> recall correctly, I believe it should be hdfs:hadoop, where the owner has
> full permissions and the group can read and execute. Does /var/log for the
> datanode show any errors?
> Regards,
> Robert
> *
> Robert Molina
> Hortonworks, Inc.
> Support Engineer
> Phone: 408.645.7062
> <http://hortonworks.com/download/>
> *
> On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
>> Hi
>>
>> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
>> I have met a issue after I change the content of hdfs-site.xml to add
>> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
>> value, /sda is the new one
>>
>> <property>
>> <name>data.dfs.dir</name>
>> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
>> </property>
>> The permission of /sda has been changed.
>>
>> [image: 内嵌图片 1]
>>
>> First I stop the cluster with stop-all.sh and replace the old
>> hdfs-site.xml with new ones,then I restart the cluster with
>> start-all.sh,but I found that datanode did not start
>>
>> --
>> From Good To Great
>>
Re: How to make the setting changes works
Posted by Robert Molina <rm...@hortonworks.com>.
Hi Geelong,
What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
recall correctly, I believe it should be hdfs:hadoop, where the owner has
full permissions and the group can read and execute. Does /var/log for the
datanode show any errors?
Regards,
Robert
*
Robert Molina
Hortonworks, Inc.
Support Engineer
Phone: 408.645.7062
<http://hortonworks.com/download/>
*
On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: 内嵌图片 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
Re: How to make the setting changes works
Posted by Harsh J <ha...@cloudera.com>.
Did you end up reformatting the NameNode? Thats one reason why DNs will
refuse to work. You need to clear out the dfs.data.dir directories for it
to work again. Note that you shouldn't reformat the NN ever, unless it is
your intention to wipe away all data and begin all over again.
On Wed, Apr 24, 2013 at 7:47 AM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: ��ǶͼƬ 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
--
Harsh J
Re: How to make the setting changes works
Posted by Robert Molina <rm...@hortonworks.com>.
Hi Geelong,
What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
recall correctly, I believe it should be hdfs:hadoop, where the owner has
full permissions and the group can read and execute. Does /var/log for the
datanode show any errors?
Regards,
Robert
*
Robert Molina
Hortonworks, Inc.
Support Engineer
Phone: 408.645.7062
<http://hortonworks.com/download/>
*
On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: ��ǶͼƬ 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
Re: How to make the setting changes works
Posted by Robert Molina <rm...@hortonworks.com>.
Hi Geelong,
What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
recall correctly, I believe it should be hdfs:hadoop, where the owner has
full permissions and the group can read and execute. Does /var/log for the
datanode show any errors?
Regards,
Robert
*
Robert Molina
Hortonworks, Inc.
Support Engineer
Phone: 408.645.7062
<http://hortonworks.com/download/>
*
On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: ��ǶͼƬ 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
Re: How to make the setting changes works
Posted by Robert Molina <rm...@hortonworks.com>.
Hi Geelong,
What's the ownership and permissions on /usr/hadoop/tmp/dfs/data? If I
recall correctly, I believe it should be hdfs:hadoop, where the owner has
full permissions and the group can read and execute. Does /var/log for the
datanode show any errors?
Regards,
Robert
*
Robert Molina
Hortonworks, Inc.
Support Engineer
Phone: 408.645.7062
<http://hortonworks.com/download/>
*
On Tue, Apr 23, 2013 at 7:17 PM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: 内嵌图片 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
Re: How to make the setting changes works
Posted by Harsh J <ha...@cloudera.com>.
Did you end up reformatting the NameNode? Thats one reason why DNs will
refuse to work. You need to clear out the dfs.data.dir directories for it
to work again. Note that you shouldn't reformat the NN ever, unless it is
your intention to wipe away all data and begin all over again.
On Wed, Apr 24, 2013 at 7:47 AM, Geelong Yao <ge...@gmail.com> wrote:
> Hi
>
> Sorry to interrupt you.But nobody answer my question in Hadoop maillist.
> I have met a issue after I change the content of hdfs-site.xml to add
> another dfs.data.dir in my cluster. /usr/hadoop/tmp/dfs/data is the default
> value, /sda is the new one
>
> <property>
> <name>data.dfs.dir</name>
> <value>/usr/hadoop/tmp/dfs/data,/sda</value>
> </property>
> The permission of /sda has been changed.
>
> [image: ��ǶͼƬ 1]
>
> First I stop the cluster with stop-all.sh and replace the old
> hdfs-site.xml with new ones,then I restart the cluster with
> start-all.sh,but I found that datanode did not start
>
> --
> From Good To Great
>
--
Harsh J