You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Glen Mazza <gm...@talend.com> on 2013/01/17 14:54:58 UTC
DFS filesystem used for Standalone and Pseudo-Distributed operation
Hi, I'm following the Hadoop 1.1.1 tutorial
(http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
these questions:
1.) The Standalone Operation
(http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
just to confirm, can run without any DFS filesystem? (We're not being
asked to run "bin/hadoop namenode -format" for it.)
2.) For Pseudo-Distributed Operation
(http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
where we are instructed to run "bin/hadoop namenode -format", question:
how do I undo whatever that command does? I.e., how do I delete that
distributed file system created as well as remove from Hadoop any
knowledge of its prior existence -- do I have to manually delete files
with OS commands (what do I remove?) or is there some type of
"bin/hadoop namenode -delete" command that undoes the "-format" command?
Thanks,
Glen
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Vikas Jadhav <vi...@gmail.com>.
You can control whr to store hdfs data set path of following property in
core-site.xml
also u can refer to Yahoo tutorial
http://developer.yahoo.com/hadoop/tutorial/module2.html
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://*your.server.name.com*:9000</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/*username*/hdfs/data</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/*username*/hdfs/name</value>
</property>
</configuration>
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
--
*
*
*
Thanx and Regards*
* Vikas Jadhav*
unsubscribe
Posted by pl...@ups.com.
unsubscribe
Posted by pl...@ups.com.
unsubscribe
Posted by pl...@ups.com.
unsubscribe
Posted by pl...@ups.com.
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
That is the default behavior. I is advisable to include "hadoop.tmp.dir"
property in your core-site.xml file and "dfs.name.dir" & "dfs.data.dir"
properties in your hdfs-site.xml in order to avoid any problem when you
reboot the machine. Looks like you have already achieved the goal. But,
if you still need any config related help you can visit this
link<http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UPgTlZs-vl8>.
I have tries
to explain the procedure there along with the reasons behind adding the
config properties.
HTH
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Vikas Jadhav <vi...@gmail.com>.
You can control whr to store hdfs data set path of following property in
core-site.xml
also u can refer to Yahoo tutorial
http://developer.yahoo.com/hadoop/tutorial/module2.html
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://*your.server.name.com*:9000</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/*username*/hdfs/data</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/*username*/hdfs/name</value>
</property>
</configuration>
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
--
*
*
*
Thanx and Regards*
* Vikas Jadhav*
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Vikas Jadhav <vi...@gmail.com>.
You can control whr to store hdfs data set path of following property in
core-site.xml
also u can refer to Yahoo tutorial
http://developer.yahoo.com/hadoop/tutorial/module2.html
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://*your.server.name.com*:9000</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/*username*/hdfs/data</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/*username*/hdfs/name</value>
</property>
</configuration>
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
--
*
*
*
Thanx and Regards*
* Vikas Jadhav*
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
That is the default behavior. I is advisable to include "hadoop.tmp.dir"
property in your core-site.xml file and "dfs.name.dir" & "dfs.data.dir"
properties in your hdfs-site.xml in order to avoid any problem when you
reboot the machine. Looks like you have already achieved the goal. But,
if you still need any config related help you can visit this
link<http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UPgTlZs-vl8>.
I have tries
to explain the procedure there along with the reasons behind adding the
config properties.
HTH
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Vikas Jadhav <vi...@gmail.com>.
You can control whr to store hdfs data set path of following property in
core-site.xml
also u can refer to Yahoo tutorial
http://developer.yahoo.com/hadoop/tutorial/module2.html
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://*your.server.name.com*:9000</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/*username*/hdfs/data</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/*username*/hdfs/name</value>
</property>
</configuration>
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
--
*
*
*
Thanx and Regards*
* Vikas Jadhav*
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
That is the default behavior. I is advisable to include "hadoop.tmp.dir"
property in your core-site.xml file and "dfs.name.dir" & "dfs.data.dir"
properties in your hdfs-site.xml in order to avoid any problem when you
reboot the machine. Looks like you have already achieved the goal. But,
if you still need any config related help you can visit this
link<http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UPgTlZs-vl8>.
I have tries
to explain the procedure there along with the reasons behind adding the
config properties.
HTH
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
That is the default behavior. I is advisable to include "hadoop.tmp.dir"
property in your core-site.xml file and "dfs.name.dir" & "dfs.data.dir"
properties in your hdfs-site.xml in order to avoid any problem when you
reboot the machine. Looks like you have already achieved the goal. But,
if you still need any config related help you can visit this
link<http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UPgTlZs-vl8>.
I have tries
to explain the procedure there along with the reasons behind adding the
config properties.
HTH
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <gm...@talend.com> wrote:
> On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem? (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
> >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp? It's subject
> that way to getting erased each time I reboot my computer. Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem? (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does? I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Glen Mazza <gm...@talend.com>.
On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do I
> delete that distributed file system created as well as remove from
> Hadoop any knowledge of its prior existence -- do I have to manually
> delete files with OS commands (what do I remove?) or is there some
> type of "bin/hadoop namenode -delete" command that undoes the
> "-format" command?
> >>You can reformat the NameNode or manually delete the directories holding
> the data and the metadata.
Thanks, Tariq, looking at the output of the namenode -format command it
looks like the only folder I have to delete is /tmp/hadoop-gmazza:
13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO common.Storage: Storage directory
/tmp/hadoop-gmazza/dfs/name has been successfully formatted.
Question: Why does Hadoop place the dfs folder under /tmp? It's subject
that way to getting erased each time I reboot my computer. Looking at
the instructions for namenode -format:
http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
doesn't appear we have an option to specify where we would like the dfs
folder to be created.
Thanks,
Glen
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com <http://cloudfront.blogspot.com>
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gmazza@talend.com
> <ma...@talend.com>> wrote:
>
> Hi, I'm following the Hadoop 1.1.1 tutorial
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and
> I have these questions:
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not
> being asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do
> I delete that distributed file system created as well as remove
> from Hadoop any knowledge of its prior existence -- do I have to
> manually delete files with OS commands (what do I remove?) or is
> there some type of "bin/hadoop namenode -delete" command that
> undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com <http://coders.talend.com>
> blog: www.jroller.com/gmazza <http://www.jroller.com/gmazza>
>
>
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Glen Mazza <gm...@talend.com>.
On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do I
> delete that distributed file system created as well as remove from
> Hadoop any knowledge of its prior existence -- do I have to manually
> delete files with OS commands (what do I remove?) or is there some
> type of "bin/hadoop namenode -delete" command that undoes the
> "-format" command?
> >>You can reformat the NameNode or manually delete the directories holding
> the data and the metadata.
Thanks, Tariq, looking at the output of the namenode -format command it
looks like the only folder I have to delete is /tmp/hadoop-gmazza:
13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO common.Storage: Storage directory
/tmp/hadoop-gmazza/dfs/name has been successfully formatted.
Question: Why does Hadoop place the dfs folder under /tmp? It's subject
that way to getting erased each time I reboot my computer. Looking at
the instructions for namenode -format:
http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
doesn't appear we have an option to specify where we would like the dfs
folder to be created.
Thanks,
Glen
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com <http://cloudfront.blogspot.com>
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gmazza@talend.com
> <ma...@talend.com>> wrote:
>
> Hi, I'm following the Hadoop 1.1.1 tutorial
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and
> I have these questions:
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not
> being asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do
> I delete that distributed file system created as well as remove
> from Hadoop any knowledge of its prior existence -- do I have to
> manually delete files with OS commands (what do I remove?) or is
> there some type of "bin/hadoop namenode -delete" command that
> undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com <http://coders.talend.com>
> blog: www.jroller.com/gmazza <http://www.jroller.com/gmazza>
>
>
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Glen Mazza <gm...@talend.com>.
On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do I
> delete that distributed file system created as well as remove from
> Hadoop any knowledge of its prior existence -- do I have to manually
> delete files with OS commands (what do I remove?) or is there some
> type of "bin/hadoop namenode -delete" command that undoes the
> "-format" command?
> >>You can reformat the NameNode or manually delete the directories holding
> the data and the metadata.
Thanks, Tariq, looking at the output of the namenode -format command it
looks like the only folder I have to delete is /tmp/hadoop-gmazza:
13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO common.Storage: Storage directory
/tmp/hadoop-gmazza/dfs/name has been successfully formatted.
Question: Why does Hadoop place the dfs folder under /tmp? It's subject
that way to getting erased each time I reboot my computer. Looking at
the instructions for namenode -format:
http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
doesn't appear we have an option to specify where we would like the dfs
folder to be created.
Thanks,
Glen
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com <http://cloudfront.blogspot.com>
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gmazza@talend.com
> <ma...@talend.com>> wrote:
>
> Hi, I'm following the Hadoop 1.1.1 tutorial
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and
> I have these questions:
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not
> being asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do
> I delete that distributed file system created as well as remove
> from Hadoop any knowledge of its prior existence -- do I have to
> manually delete files with OS commands (what do I remove?) or is
> there some type of "bin/hadoop namenode -delete" command that
> undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com <http://coders.talend.com>
> blog: www.jroller.com/gmazza <http://www.jroller.com/gmazza>
>
>
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Glen Mazza <gm...@talend.com>.
On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
> Hello Glen,
>
> Pl find my comments embedded below :
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do I
> delete that distributed file system created as well as remove from
> Hadoop any knowledge of its prior existence -- do I have to manually
> delete files with OS commands (what do I remove?) or is there some
> type of "bin/hadoop namenode -delete" command that undoes the
> "-format" command?
> >>You can reformat the NameNode or manually delete the directories holding
> the data and the metadata.
Thanks, Tariq, looking at the output of the namenode -format command it
looks like the only folder I have to delete is /tmp/hadoop-gmazza:
13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO common.Storage: Storage directory
/tmp/hadoop-gmazza/dfs/name has been successfully formatted.
Question: Why does Hadoop place the dfs folder under /tmp? It's subject
that way to getting erased each time I reboot my computer. Looking at
the instructions for namenode -format:
http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
doesn't appear we have an option to specify where we would like the dfs
folder to be created.
Thanks,
Glen
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com <http://cloudfront.blogspot.com>
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gmazza@talend.com
> <ma...@talend.com>> wrote:
>
> Hi, I'm following the Hadoop 1.1.1 tutorial
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and
> I have these questions:
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem? (We're not
> being asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does? I.e., how do
> I delete that distributed file system created as well as remove
> from Hadoop any knowledge of its prior existence -- do I have to
> manually delete files with OS commands (what do I remove?) or is
> there some type of "bin/hadoop namenode -delete" command that
> undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com <http://coders.talend.com>
> blog: www.jroller.com/gmazza <http://www.jroller.com/gmazza>
>
>
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
Pl find my comments embedded below :
1.) The Standalone Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
just to confirm, can run without any DFS filesystem? (We're not being
asked to run "bin/hadoop namenode -format" for it.)
>>Yes. You don't need HDFS for this.
2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
where we are instructed to run "bin/hadoop namenode -format", question: how
do I undo whatever that command does? I.e., how do I delete that
distributed file system created as well as remove from Hadoop any knowledge
of its prior existence -- do I have to manually delete files with OS
commands (what do I remove?) or is there some type of "bin/hadoop namenode
-delete" command that undoes the "-format" command?
>>You can reformat the NameNode or manually delete the directories holding
the data and the metadata.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
> Hi, I'm following the Hadoop 1.1.1 tutorial (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html>)
> and I have these questions:
>
> 1.) The Standalone Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
Pl find my comments embedded below :
1.) The Standalone Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
just to confirm, can run without any DFS filesystem? (We're not being
asked to run "bin/hadoop namenode -format" for it.)
>>Yes. You don't need HDFS for this.
2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
where we are instructed to run "bin/hadoop namenode -format", question: how
do I undo whatever that command does? I.e., how do I delete that
distributed file system created as well as remove from Hadoop any knowledge
of its prior existence -- do I have to manually delete files with OS
commands (what do I remove?) or is there some type of "bin/hadoop namenode
-delete" command that undoes the "-format" command?
>>You can reformat the NameNode or manually delete the directories holding
the data and the metadata.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
> Hi, I'm following the Hadoop 1.1.1 tutorial (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html>)
> and I have these questions:
>
> 1.) The Standalone Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
Pl find my comments embedded below :
1.) The Standalone Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
just to confirm, can run without any DFS filesystem? (We're not being
asked to run "bin/hadoop namenode -format" for it.)
>>Yes. You don't need HDFS for this.
2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
where we are instructed to run "bin/hadoop namenode -format", question: how
do I undo whatever that command does? I.e., how do I delete that
distributed file system created as well as remove from Hadoop any knowledge
of its prior existence -- do I have to manually delete files with OS
commands (what do I remove?) or is there some type of "bin/hadoop namenode
-delete" command that undoes the "-format" command?
>>You can reformat the NameNode or manually delete the directories holding
the data and the metadata.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
> Hi, I'm following the Hadoop 1.1.1 tutorial (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html>)
> and I have these questions:
>
> 1.) The Standalone Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Glen,
Pl find my comments embedded below :
1.) The Standalone Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
just to confirm, can run without any DFS filesystem? (We're not being
asked to run "bin/hadoop namenode -format" for it.)
>>Yes. You don't need HDFS for this.
2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
where we are instructed to run "bin/hadoop namenode -format", question: how
do I undo whatever that command does? I.e., how do I delete that
distributed file system created as well as remove from Hadoop any knowledge
of its prior existence -- do I have to manually delete files with OS
commands (what do I remove?) or is there some type of "bin/hadoop namenode
-delete" command that undoes the "-format" command?
>>You can reformat the NameNode or manually delete the directories holding
the data and the metadata.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gm...@talend.com> wrote:
> Hi, I'm following the Hadoop 1.1.1 tutorial (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html>)
> and I have these questions:
>
> 1.) The Standalone Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#Local<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local>),
> just to confirm, can run without any DFS filesystem? (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
>
> 2.) For Pseudo-Distributed Operation (http://hadoop.apache.org/**
> docs/r1.1.1/single_node_setup.**html#PseudoDistributed<http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed>),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does? I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
>
> Thanks,
> Glen
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>