You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Iv...@emc.com on 2011/09/22 23:43:14 UTC
formatting hdfs without user interaction
Hello,
I am trying to automate formatting an HDFS volume. Is there any way to do this without the interaction (and using expect)?
Cheers,
Ivan
Re: formatting hdfs without user interaction
Posted by Iv...@emc.com.
On 9/23/11 9:01 AM, "Edward Capriolo" <ed...@gmail.com> wrote:
>On Fri, Sep 23, 2011 at 11:52 AM, <Iv...@emc.com> wrote:
>
>> Hi Harsh,
>>
>> On 9/22/11 8:48 PM, "Harsh J" <ha...@cloudera.com> wrote:
>>
>> >Ivan,
>> >
>> >Writing your own program was overkill.
>> >
>> >The 'yes' coreutil is pretty silly, but nifty at the same time. It
>> >accepts an argument, which it would repeat infinitely.
>> >
>> >So:
>> >
>> >$ yes Y | hadoop namenode -format
>> >
>> >Would do it for you.
>>
>> Nice! I read the man page for yes too quickly and did not see that
>> option. Thanks!
>>
>>...... SNIP ...............
>>
>You might want to try expect scripting. You open a stream to the process
>and
>then can wait for prompts and print replies. expect also has this feature
>autoexpect which is a shell that records the streams and turns your
>teminal
>interaction into an expect script.
Yeah, I have used expect via a python library numerous times, but expect
is a bit brittle and error prone, so I try to avoid it whenever possible.
Cheers,
Ivan
Re: formatting hdfs without user interaction
Posted by Edward Capriolo <ed...@gmail.com>.
On Fri, Sep 23, 2011 at 11:52 AM, <Iv...@emc.com> wrote:
> Hi Harsh,
>
> On 9/22/11 8:48 PM, "Harsh J" <ha...@cloudera.com> wrote:
>
> >Ivan,
> >
> >Writing your own program was overkill.
> >
> >The 'yes' coreutil is pretty silly, but nifty at the same time. It
> >accepts an argument, which it would repeat infinitely.
> >
> >So:
> >
> >$ yes Y | hadoop namenode -format
> >
> >Would do it for you.
>
> Nice! I read the man page for yes too quickly and did not see that
> option. Thanks!
>
>
> >(Note that in the future release, saner answers will be acceptable,
> >i.e. y instead of strictly Y, etc.)
>
> Y/y/yes/YES would all seem like good things to accept :)
>
>
> >Also, two other things:
> >
> >- What do you mean by 'Yeah I have a secondary namenode as well so 2
> >directories.'? A secondary namenode uses different directories than
> >dfs.name.dir.
>
> Which parameter are you referring to? I am planning on using 2 directories
> in dfs.name.dir, one is local and the other is an NFS mount of a 2nd
> machine running the secondary namenode.
>
>
> >- The prompt only appears when it detects a 'reformat' being happening
> >- which is very dangerous to do non-interactively. If you do the
> >-format the first time, on clean dfs.name.dir setups, you will never
> >receive a prompt.
>
> Yeah I am creating some automation, so it needs to be able to wipe out an
> existing filesystem and start over......
>
>
> Cheers,
> Ivan
>
>
>
> >
>
>
You might want to try expect scripting. You open a stream to the process and
then can wait for prompts and print replies. expect also has this feature
autoexpect which is a shell that records the streams and turns your teminal
interaction into an expect script.
Re: formatting hdfs without user interaction
Posted by Iv...@emc.com.
On 9/23/11 9:46 AM, "Harsh J" <ha...@cloudera.com> wrote:
>Ivan,
>
>On Fri, Sep 23, 2011 at 9:22 PM, <Iv...@emc.com> wrote:
>[snip]
>> Which parameter are you referring to? I am planning on using 2
>>directories
>> in dfs.name.dir, one is local and the other is an NFS mount of a 2nd
>> machine running the secondary namenode.
>
>I'm slightly confused here. Do you mean to say a 'SecondaryNameNode',
>which is a form of checkpointing daemon available as part of Hadoop,
>or do you mean to say, loosely, 'secondary namenode', which would
>imply a machine that can step in as a NameNode once you start it (in
>case the primary fails)?
>
>For the former, the properties are the ones that begin with
>"fs.checkpoint.*" in 0.20.x versions, which you can checkout.
I mean both... A 2nd node which can step in as a name node and also that
node will run the checkpointing daemon. I have 2 directories in
dfs.name.dir one is local and one is remote on the 2nd machine so we have
a copy of the data if we lose the first machine.
Cheers,
Ivan
Re: formatting hdfs without user interaction
Posted by Harsh J <ha...@cloudera.com>.
Ivan,
On Fri, Sep 23, 2011 at 9:22 PM, <Iv...@emc.com> wrote:
[snip]
> Which parameter are you referring to? I am planning on using 2 directories
> in dfs.name.dir, one is local and the other is an NFS mount of a 2nd
> machine running the secondary namenode.
I'm slightly confused here. Do you mean to say a 'SecondaryNameNode',
which is a form of checkpointing daemon available as part of Hadoop,
or do you mean to say, loosely, 'secondary namenode', which would
imply a machine that can step in as a NameNode once you start it (in
case the primary fails)?
For the former, the properties are the ones that begin with
"fs.checkpoint.*" in 0.20.x versions, which you can checkout.
--
Harsh J
Re: formatting hdfs without user interaction
Posted by Iv...@emc.com.
Hi Harsh,
On 9/22/11 8:48 PM, "Harsh J" <ha...@cloudera.com> wrote:
>Ivan,
>
>Writing your own program was overkill.
>
>The 'yes' coreutil is pretty silly, but nifty at the same time. It
>accepts an argument, which it would repeat infinitely.
>
>So:
>
>$ yes Y | hadoop namenode -format
>
>Would do it for you.
Nice! I read the man page for yes too quickly and did not see that
option. Thanks!
>(Note that in the future release, saner answers will be acceptable,
>i.e. y instead of strictly Y, etc.)
Y/y/yes/YES would all seem like good things to accept :)
>Also, two other things:
>
>- What do you mean by 'Yeah I have a secondary namenode as well so 2
>directories.'? A secondary namenode uses different directories than
>dfs.name.dir.
Which parameter are you referring to? I am planning on using 2 directories
in dfs.name.dir, one is local and the other is an NFS mount of a 2nd
machine running the secondary namenode.
>- The prompt only appears when it detects a 'reformat' being happening
>- which is very dangerous to do non-interactively. If you do the
>-format the first time, on clean dfs.name.dir setups, you will never
>receive a prompt.
Yeah I am creating some automation, so it needs to be able to wipe out an
existing filesystem and start over......
Cheers,
Ivan
>
Re: formatting hdfs without user interaction
Posted by Harsh J <ha...@cloudera.com>.
Ivan,
Writing your own program was overkill.
The 'yes' coreutil is pretty silly, but nifty at the same time. It
accepts an argument, which it would repeat infinitely.
So:
$ yes Y | hadoop namenode -format
Would do it for you.
(Note that in the future release, saner answers will be acceptable,
i.e. y instead of strictly Y, etc.)
Also, two other things:
- What do you mean by 'Yeah I have a secondary namenode as well so 2
directories.'? A secondary namenode uses different directories than
dfs.name.dir.
- The prompt only appears when it detects a 'reformat' being happening
- which is very dangerous to do non-interactively. If you do the
-format the first time, on clean dfs.name.dir setups, you will never
receive a prompt.
On Fri, Sep 23, 2011 at 3:53 AM, <Iv...@emc.com> wrote:
> Hi Adam,
>
> Well the "yes" program prints lower case y's and apparently only captial Y
> is accepted.
>
> But by creating my out "Yes" program that spews Y's to stdout it works :)
>
> Cheers,
> Ivan
>
> On 9/22/11 3:02 PM, "Adam Shook" <as...@clearedgeit.com> wrote:
>
>>yes | hadoop namenode -format
>>
>>The "yes" program simply outputs 'y' a bunch of times. "echo yes" will
>>just print "yes" to stdout.
>>
>>-- Adam
>>
>>-----Original Message-----
>>From: Ivan.Novick@emc.com [mailto:Ivan.Novick@emc.com]
>>Sent: Thursday, September 22, 2011 6:00 PM
>>To: common-user@hadoop.apache.org; rajvish@yahoo.com
>>Subject: Re: formatting hdfs without user interaction
>>
>>Hi Raj and Arpit,
>>
>>I think the fact that there are 2 confirmations is causing it not to work:
>>
>>echo "Y" | hadoop namenode -format
>>11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
>>/************************************************************
>>STARTUP_MSG: Starting NameNode
>>STARTUP_MSG: host = hdm1/172.28.9.250
>>STARTUP_MSG: args = [-format]
>>STARTUP_MSG: version = 0.20.204.0
>>STARTUP_MSG: build = git://hrt8n35.cc1.ygridcore.net/ on branch
>>branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
>>compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
>>************************************************************/
>>Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
>>filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
>>/data/remote/hdfs/name
>>11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
>>/************************************************************
>>SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
>>************************************************************/
>>Heap
>> par new generation total 2831168K, used 150996K [0x00002aaaae210000,
>>0x00002aab6e210000, 0x00002aab6e210000)
>> eden space 2516608K, 6% used [0x00002aaaae210000, 0x00002aaab75852d0,
>>0x00002aab47bb0000)
>> from space 314560K, 0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
>>0x00002aab5aee0000)
>> to space 314560K, 0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
>>0x00002aab6e210000)
>> concurrent mark-sweep generation total 30408704K, used 0K
>>[0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
>> concurrent-mark-sweep perm gen total 21248K, used 7333K
>>[0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
>>
>>
>>Cheers,
>>Ivan
>>
>>On 9/22/11 2:52 PM, "Raj V" <ra...@yahoo.com> wrote:
>>
>>>Ian
>>>"echo 'Y' | hadoop namenode -format"
>>>
>>>should work.,
>>>
>>>Raj
>>>
>>>
>>>>________________________________
>>>>From: "Ivan.Novick@emc.com" <Iv...@emc.com>
>>>>To: common-user@hadoop.apache.org
>>>>Sent: Thursday, September 22, 2011 2:43 PM
>>>>Subject: formatting hdfs without user interaction
>>>>
>>>>Hello,
>>>>
>>>>I am trying to automate formatting an HDFS volume. Is there any way to
>>>>do this without the interaction (and using expect)?
>>>>
>>>>Cheers,
>>>>Ivan
>>>>
>>>>
>>
>>
>>-----
>>No virus found in this message.
>>Checked by AVG - www.avg.com
>>Version: 10.0.1410 / Virus Database: 1520/3912 - Release Date: 09/22/11
>>
>
>
--
Harsh J
Re: formatting hdfs without user interaction
Posted by Iv...@emc.com.
Hi Adam,
Well the "yes" program prints lower case y's and apparently only captial Y
is accepted.
But by creating my out "Yes" program that spews Y's to stdout it works :)
Cheers,
Ivan
On 9/22/11 3:02 PM, "Adam Shook" <as...@clearedgeit.com> wrote:
>yes | hadoop namenode -format
>
>The "yes" program simply outputs 'y' a bunch of times. "echo yes" will
>just print "yes" to stdout.
>
>-- Adam
>
>-----Original Message-----
>From: Ivan.Novick@emc.com [mailto:Ivan.Novick@emc.com]
>Sent: Thursday, September 22, 2011 6:00 PM
>To: common-user@hadoop.apache.org; rajvish@yahoo.com
>Subject: Re: formatting hdfs without user interaction
>
>Hi Raj and Arpit,
>
>I think the fact that there are 2 confirmations is causing it not to work:
>
>echo "Y" | hadoop namenode -format
>11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
>/************************************************************
>STARTUP_MSG: Starting NameNode
>STARTUP_MSG: host = hdm1/172.28.9.250
>STARTUP_MSG: args = [-format]
>STARTUP_MSG: version = 0.20.204.0
>STARTUP_MSG: build = git://hrt8n35.cc1.ygridcore.net/ on branch
>branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
>compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
>************************************************************/
>Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
>filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
>/data/remote/hdfs/name
>11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
>/************************************************************
>SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
>************************************************************/
>Heap
> par new generation total 2831168K, used 150996K [0x00002aaaae210000,
>0x00002aab6e210000, 0x00002aab6e210000)
> eden space 2516608K, 6% used [0x00002aaaae210000, 0x00002aaab75852d0,
>0x00002aab47bb0000)
> from space 314560K, 0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
>0x00002aab5aee0000)
> to space 314560K, 0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
>0x00002aab6e210000)
> concurrent mark-sweep generation total 30408704K, used 0K
>[0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
> concurrent-mark-sweep perm gen total 21248K, used 7333K
>[0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
>
>
>Cheers,
>Ivan
>
>On 9/22/11 2:52 PM, "Raj V" <ra...@yahoo.com> wrote:
>
>>Ian
>>"echo 'Y' | hadoop namenode -format"
>>
>>should work.,
>>
>>Raj
>>
>>
>>>________________________________
>>>From: "Ivan.Novick@emc.com" <Iv...@emc.com>
>>>To: common-user@hadoop.apache.org
>>>Sent: Thursday, September 22, 2011 2:43 PM
>>>Subject: formatting hdfs without user interaction
>>>
>>>Hello,
>>>
>>>I am trying to automate formatting an HDFS volume. Is there any way to
>>>do this without the interaction (and using expect)?
>>>
>>>Cheers,
>>>Ivan
>>>
>>>
>
>
>-----
>No virus found in this message.
>Checked by AVG - www.avg.com
>Version: 10.0.1410 / Virus Database: 1520/3912 - Release Date: 09/22/11
>
RE: formatting hdfs without user interaction
Posted by Adam Shook <as...@clearedgeit.com>.
yes | hadoop namenode -format
The "yes" program simply outputs 'y' a bunch of times. "echo yes" will just print "yes" to stdout.
-- Adam
-----Original Message-----
From: Ivan.Novick@emc.com [mailto:Ivan.Novick@emc.com]
Sent: Thursday, September 22, 2011 6:00 PM
To: common-user@hadoop.apache.org; rajvish@yahoo.com
Subject: Re: formatting hdfs without user interaction
Hi Raj and Arpit,
I think the fact that there are 2 confirmations is causing it not to work:
echo "Y" | hadoop namenode -format
11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = hdm1/172.28.9.250
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.20.204.0
STARTUP_MSG: build = git://hrt8n35.cc1.ygridcore.net/ on branch
branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
************************************************************/
Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
/data/remote/hdfs/name
11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
************************************************************/
Heap
par new generation total 2831168K, used 150996K [0x00002aaaae210000,
0x00002aab6e210000, 0x00002aab6e210000)
eden space 2516608K, 6% used [0x00002aaaae210000, 0x00002aaab75852d0,
0x00002aab47bb0000)
from space 314560K, 0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
0x00002aab5aee0000)
to space 314560K, 0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
0x00002aab6e210000)
concurrent mark-sweep generation total 30408704K, used 0K
[0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
concurrent-mark-sweep perm gen total 21248K, used 7333K
[0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
Cheers,
Ivan
On 9/22/11 2:52 PM, "Raj V" <ra...@yahoo.com> wrote:
>Ian
>"echo 'Y' | hadoop namenode -format"
>
>should work.,
>
>Raj
>
>
>>________________________________
>>From: "Ivan.Novick@emc.com" <Iv...@emc.com>
>>To: common-user@hadoop.apache.org
>>Sent: Thursday, September 22, 2011 2:43 PM
>>Subject: formatting hdfs without user interaction
>>
>>Hello,
>>
>>I am trying to automate formatting an HDFS volume. Is there any way to
>>do this without the interaction (and using expect)?
>>
>>Cheers,
>>Ivan
>>
>>
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 10.0.1410 / Virus Database: 1520/3912 - Release Date: 09/22/11
Re: formatting hdfs without user interaction
Posted by Iv...@emc.com.
Yeah I have a secondary namenode as well so 2 directories.
I was trying to avoid expect if possible..... But this is always an option.
Cheers,
Ivan
On 9/22/11 3:17 PM, "Arpit Gupta" <ar...@hortonworks.com> wrote:
>The reason you are getting multiple prompts is that you have multiple
>dir's defined in the dfs.name.dir.
>
>A simple expect script would take care of this.
>
>#!/usr/bin/expect -f
>spawn /bin/hadoop namenode -format
>expect "Re-format filesystem in"
>send Y\n
>expect "Re-format filesystem in"
>send Y\n
>interact
>
>
>You might have to put the above in a loop based on how many dirs are
>defined in the dfs.name.dir
>
>--
>Arpit
>arpit@hortonworks.com
>
>
>
>
>On Sep 22, 2011, at 2:59 PM, <Iv...@emc.com> wrote:
>
>> Hi Raj and Arpit,
>>
>> I think the fact that there are 2 confirmations is causing it not to
>>work:
>>
>> echo "Y" | hadoop namenode -format
>> 11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
>> /************************************************************
>> STARTUP_MSG: Starting NameNode
>> STARTUP_MSG: host = hdm1/172.28.9.250
>> STARTUP_MSG: args = [-format]
>> STARTUP_MSG: version = 0.20.204.0
>> STARTUP_MSG: build = git://hrt8n35.cc1.ygridcore.net/ on branch
>> branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
>> compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
>> ************************************************************/
>> Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
>> filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
>> /data/remote/hdfs/name
>> 11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
>> ************************************************************/
>> Heap
>> par new generation total 2831168K, used 150996K [0x00002aaaae210000,
>> 0x00002aab6e210000, 0x00002aab6e210000)
>> eden space 2516608K, 6% used [0x00002aaaae210000, 0x00002aaab75852d0,
>> 0x00002aab47bb0000)
>> from space 314560K, 0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
>> 0x00002aab5aee0000)
>> to space 314560K, 0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
>> 0x00002aab6e210000)
>> concurrent mark-sweep generation total 30408704K, used 0K
>> [0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
>> concurrent-mark-sweep perm gen total 21248K, used 7333K
>> [0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
>>
>>
>> Cheers,
>> Ivan
>>
>> On 9/22/11 2:52 PM, "Raj V" <ra...@yahoo.com> wrote:
>>
>>> Ian
>>> "echo 'Y' | hadoop namenode -format"
>>>
>>> should work.,
>>>
>>> Raj
>>>
>>>
>>>> ________________________________
>>>> From: "Ivan.Novick@emc.com" <Iv...@emc.com>
>>>> To: common-user@hadoop.apache.org
>>>> Sent: Thursday, September 22, 2011 2:43 PM
>>>> Subject: formatting hdfs without user interaction
>>>>
>>>> Hello,
>>>>
>>>> I am trying to automate formatting an HDFS volume. Is there any way
>>>>to
>>>> do this without the interaction (and using expect)?
>>>>
>>>> Cheers,
>>>> Ivan
>>>>
>>>>
>>
>
>
Re: formatting hdfs without user interaction
Posted by Arpit Gupta <ar...@hortonworks.com>.
The reason you are getting multiple prompts is that you have multiple dir's defined in the dfs.name.dir.
A simple expect script would take care of this.
#!/usr/bin/expect -f
spawn /bin/hadoop namenode -format
expect "Re-format filesystem in"
send Y\n
expect "Re-format filesystem in"
send Y\n
interact
You might have to put the above in a loop based on how many dirs are defined in the dfs.name.dir
--
Arpit
arpit@hortonworks.com
On Sep 22, 2011, at 2:59 PM, <Iv...@emc.com> wrote:
> Hi Raj and Arpit,
>
> I think the fact that there are 2 confirmations is causing it not to work:
>
> echo "Y" | hadoop namenode -format
> 11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG: host = hdm1/172.28.9.250
> STARTUP_MSG: args = [-format]
> STARTUP_MSG: version = 0.20.204.0
> STARTUP_MSG: build = git://hrt8n35.cc1.ygridcore.net/ on branch
> branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
> compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
> ************************************************************/
> Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
> filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
> /data/remote/hdfs/name
> 11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
> ************************************************************/
> Heap
> par new generation total 2831168K, used 150996K [0x00002aaaae210000,
> 0x00002aab6e210000, 0x00002aab6e210000)
> eden space 2516608K, 6% used [0x00002aaaae210000, 0x00002aaab75852d0,
> 0x00002aab47bb0000)
> from space 314560K, 0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
> 0x00002aab5aee0000)
> to space 314560K, 0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
> 0x00002aab6e210000)
> concurrent mark-sweep generation total 30408704K, used 0K
> [0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
> concurrent-mark-sweep perm gen total 21248K, used 7333K
> [0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
>
>
> Cheers,
> Ivan
>
> On 9/22/11 2:52 PM, "Raj V" <ra...@yahoo.com> wrote:
>
>> Ian
>> "echo 'Y' | hadoop namenode -format"
>>
>> should work.,
>>
>> Raj
>>
>>
>>> ________________________________
>>> From: "Ivan.Novick@emc.com" <Iv...@emc.com>
>>> To: common-user@hadoop.apache.org
>>> Sent: Thursday, September 22, 2011 2:43 PM
>>> Subject: formatting hdfs without user interaction
>>>
>>> Hello,
>>>
>>> I am trying to automate formatting an HDFS volume. Is there any way to
>>> do this without the interaction (and using expect)?
>>>
>>> Cheers,
>>> Ivan
>>>
>>>
>
Re: formatting hdfs without user interaction
Posted by Iv...@emc.com.
Hi Raj and Arpit,
I think the fact that there are 2 confirmations is causing it not to work:
echo "Y" | hadoop namenode -format
11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = hdm1/172.28.9.250
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.20.204.0
STARTUP_MSG: build = git://hrt8n35.cc1.ygridcore.net/ on branch
branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
************************************************************/
Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
/data/remote/hdfs/name
11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
************************************************************/
Heap
par new generation total 2831168K, used 150996K [0x00002aaaae210000,
0x00002aab6e210000, 0x00002aab6e210000)
eden space 2516608K, 6% used [0x00002aaaae210000, 0x00002aaab75852d0,
0x00002aab47bb0000)
from space 314560K, 0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
0x00002aab5aee0000)
to space 314560K, 0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
0x00002aab6e210000)
concurrent mark-sweep generation total 30408704K, used 0K
[0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
concurrent-mark-sweep perm gen total 21248K, used 7333K
[0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
Cheers,
Ivan
On 9/22/11 2:52 PM, "Raj V" <ra...@yahoo.com> wrote:
>Ian
>"echo 'Y' | hadoop namenode -format"
>
>should work.,
>
>Raj
>
>
>>________________________________
>>From: "Ivan.Novick@emc.com" <Iv...@emc.com>
>>To: common-user@hadoop.apache.org
>>Sent: Thursday, September 22, 2011 2:43 PM
>>Subject: formatting hdfs without user interaction
>>
>>Hello,
>>
>>I am trying to automate formatting an HDFS volume. Is there any way to
>>do this without the interaction (and using expect)?
>>
>>Cheers,
>>Ivan
>>
>>
Re: formatting hdfs without user interaction
Posted by Raj V <ra...@yahoo.com>.
Ian
"echo 'Y' | hadoop namenode -format"
should work.,
Raj
>________________________________
>From: "Ivan.Novick@emc.com" <Iv...@emc.com>
>To: common-user@hadoop.apache.org
>Sent: Thursday, September 22, 2011 2:43 PM
>Subject: formatting hdfs without user interaction
>
>Hello,
>
>I am trying to automate formatting an HDFS volume. Is there any way to do this without the interaction (and using expect)?
>
>Cheers,
>Ivan
>
>
>
Re: formatting hdfs without user interaction
Posted by Arpit Gupta <ar...@hortonworks.com>.
you could try
echo yes | bin/hadoop namenode -format
--
Arpit
arpit@hortonworks.com
On Sep 22, 2011, at 2:43 PM, <Iv...@emc.com> wrote:
> Hello,
>
> I am trying to automate formatting an HDFS volume. Is there any way to do this without the interaction (and using expect)?
>
> Cheers,
> Ivan