You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "Arthur.hk.chan@gmail.com" <ar...@gmail.com> on 2014/08/02 08:59:08 UTC
Re: Hadoop 2.4.0 How to change "Configured Capacity"
Hi,
Both ”dfs.name.data.dir” and “dfs.datanode.data.dir” are not set in my cluster. By the way I have searched around about these two parameters, I cannot find them in Hadoop Default page.
http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
Can you please advise where to set them and how to set them? in hdfs-site.xml or in core-site.xml or another configuration file?
Many thanks
Arthur
On 29 Jul, 2014, at 1:27 am, hadoop hive <ha...@gmail.com> wrote:
> You need to add each disk inside "dfs.name.data.dir" parameter.
>
> On Jul 28, 2014 5:14 AM, "Arthur.hk.chan@gmail.com" <ar...@gmail.com> wrote:
> Hi,
>
> I have installed Hadoop 2.4.0 with 5 nodes, each node physically has 4T hard disk, when checking the configured capacity, I found it is about 49.22 GB per node, can anyone advise how to set bigger “configured capacity” e.g. 2T or more per node?
>
> Name node
> Configured Capacity: 264223436800 (246.08 GB)
>
> Each Datanode
> Configured Capacity: 52844687360 (49.22 GB)
>
> regards
> Arthur
Re: Hadoop 2.4.0 How to change "Configured Capacity"
Posted by Harsh J <ha...@cloudera.com>.
You will need to set them up in the hdfs-site.xml.
P.s. Their default is present in the hdfs-default.xml you linked to:
http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml#dfs.datanode.data.dir
On Sat, Aug 2, 2014 at 12:29 PM, Arthur.hk.chan@gmail.com
<ar...@gmail.com> wrote:
> Hi,
>
> Both ”dfs.name.data.dir” and “dfs.datanode.data.dir” are not set in my
> cluster. By the way I have searched around about these two parameters, I
> cannot find them in Hadoop Default page.
> http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
>
> Can you please advise where to set them and how to set them? in
> hdfs-site.xml or in core-site.xml or another configuration file?
>
> Many thanks
> Arthur
>
> On 29 Jul, 2014, at 1:27 am, hadoop hive <ha...@gmail.com> wrote:
>
> You need to add each disk inside "dfs.name.data.dir" parameter.
>
> On Jul 28, 2014 5:14 AM, "Arthur.hk.chan@gmail.com"
> <ar...@gmail.com> wrote:
>>
>> Hi,
>>
>> I have installed Hadoop 2.4.0 with 5 nodes, each node physically has 4T
>> hard disk, when checking the configured capacity, I found it is about 49.22
>> GB per node, can anyone advise how to set bigger “configured capacity” e.g.
>> 2T or more per node?
>>
>> Name node
>> Configured Capacity: 264223436800 (246.08 GB)
>>
>> Each Datanode
>> Configured Capacity: 52844687360 (49.22 GB)
>>
>> regards
>> Arthur
>
>
--
Harsh J
Re: Hadoop 2.4.0 How to change "Configured Capacity"
Posted by Harsh J <ha...@cloudera.com>.
You will need to set them up in the hdfs-site.xml.
P.s. Their default is present in the hdfs-default.xml you linked to:
http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml#dfs.datanode.data.dir
On Sat, Aug 2, 2014 at 12:29 PM, Arthur.hk.chan@gmail.com
<ar...@gmail.com> wrote:
> Hi,
>
> Both ”dfs.name.data.dir” and “dfs.datanode.data.dir” are not set in my
> cluster. By the way I have searched around about these two parameters, I
> cannot find them in Hadoop Default page.
> http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
>
> Can you please advise where to set them and how to set them? in
> hdfs-site.xml or in core-site.xml or another configuration file?
>
> Many thanks
> Arthur
>
> On 29 Jul, 2014, at 1:27 am, hadoop hive <ha...@gmail.com> wrote:
>
> You need to add each disk inside "dfs.name.data.dir" parameter.
>
> On Jul 28, 2014 5:14 AM, "Arthur.hk.chan@gmail.com"
> <ar...@gmail.com> wrote:
>>
>> Hi,
>>
>> I have installed Hadoop 2.4.0 with 5 nodes, each node physically has 4T
>> hard disk, when checking the configured capacity, I found it is about 49.22
>> GB per node, can anyone advise how to set bigger “configured capacity” e.g.
>> 2T or more per node?
>>
>> Name node
>> Configured Capacity: 264223436800 (246.08 GB)
>>
>> Each Datanode
>> Configured Capacity: 52844687360 (49.22 GB)
>>
>> regards
>> Arthur
>
>
--
Harsh J
Re: Hadoop 2.4.0 How to change "Configured Capacity"
Posted by Harsh J <ha...@cloudera.com>.
You will need to set them up in the hdfs-site.xml.
P.s. Their default is present in the hdfs-default.xml you linked to:
http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml#dfs.datanode.data.dir
On Sat, Aug 2, 2014 at 12:29 PM, Arthur.hk.chan@gmail.com
<ar...@gmail.com> wrote:
> Hi,
>
> Both ”dfs.name.data.dir” and “dfs.datanode.data.dir” are not set in my
> cluster. By the way I have searched around about these two parameters, I
> cannot find them in Hadoop Default page.
> http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
>
> Can you please advise where to set them and how to set them? in
> hdfs-site.xml or in core-site.xml or another configuration file?
>
> Many thanks
> Arthur
>
> On 29 Jul, 2014, at 1:27 am, hadoop hive <ha...@gmail.com> wrote:
>
> You need to add each disk inside "dfs.name.data.dir" parameter.
>
> On Jul 28, 2014 5:14 AM, "Arthur.hk.chan@gmail.com"
> <ar...@gmail.com> wrote:
>>
>> Hi,
>>
>> I have installed Hadoop 2.4.0 with 5 nodes, each node physically has 4T
>> hard disk, when checking the configured capacity, I found it is about 49.22
>> GB per node, can anyone advise how to set bigger “configured capacity” e.g.
>> 2T or more per node?
>>
>> Name node
>> Configured Capacity: 264223436800 (246.08 GB)
>>
>> Each Datanode
>> Configured Capacity: 52844687360 (49.22 GB)
>>
>> regards
>> Arthur
>
>
--
Harsh J
Re: Hadoop 2.4.0 How to change "Configured Capacity"
Posted by Harsh J <ha...@cloudera.com>.
You will need to set them up in the hdfs-site.xml.
P.s. Their default is present in the hdfs-default.xml you linked to:
http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml#dfs.datanode.data.dir
On Sat, Aug 2, 2014 at 12:29 PM, Arthur.hk.chan@gmail.com
<ar...@gmail.com> wrote:
> Hi,
>
> Both ”dfs.name.data.dir” and “dfs.datanode.data.dir” are not set in my
> cluster. By the way I have searched around about these two parameters, I
> cannot find them in Hadoop Default page.
> http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
>
> Can you please advise where to set them and how to set them? in
> hdfs-site.xml or in core-site.xml or another configuration file?
>
> Many thanks
> Arthur
>
> On 29 Jul, 2014, at 1:27 am, hadoop hive <ha...@gmail.com> wrote:
>
> You need to add each disk inside "dfs.name.data.dir" parameter.
>
> On Jul 28, 2014 5:14 AM, "Arthur.hk.chan@gmail.com"
> <ar...@gmail.com> wrote:
>>
>> Hi,
>>
>> I have installed Hadoop 2.4.0 with 5 nodes, each node physically has 4T
>> hard disk, when checking the configured capacity, I found it is about 49.22
>> GB per node, can anyone advise how to set bigger “configured capacity” e.g.
>> 2T or more per node?
>>
>> Name node
>> Configured Capacity: 264223436800 (246.08 GB)
>>
>> Each Datanode
>> Configured Capacity: 52844687360 (49.22 GB)
>>
>> regards
>> Arthur
>
>
--
Harsh J