You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Aditya exalter <ad...@gmail.com> on 2013/05/14 10:51:38 UTC
How do secondary name node get files from name node
can i get complete details of how sec name node gets fsimage and edits file
from name node through http get and http post with addressing from memory.
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
SNN does not fetch data of NN from memory. It copies off of the stored
edits and fsimage files at the NN at the time of checkpoint request.
The link I pointed to you has the current stable source, you should be
able to read it for more details. Note that none of this is public
API.
On Tue, May 14, 2013 at 2:58 PM, Aditya exalter <ad...@gmail.com> wrote:
> As the secondary name node copies the data from name node for every 1 hour
> through http get and copies from main memory of name node so i want to use
> the same case as i want to get the data from the main memory as the
> secondary name node gets.I want the procedure and how to access it.
>
>
> On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Aditya,
>>
>> What do you mean by "addressing from memory"? Note that its difficult
>> for us to answer your questions perfectly if you don't also specify
>> exactly what you are looking to do and some of why.
>>
>> In any case, assuming you're use the maintenance 1.x release of HDFS
>> that does not have HA, you can look at the sources if you're
>> interested in the nitty gritty details:
>>
>> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
>> Sources are also available in the tarball of your Hadoop version.
>>
>> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
>> wrote:
>> > can i get complete details of how sec name node gets fsimage and edits
>> > file
>> > from name node through http get and http post with addressing from
>> > memory.
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
SNN does not fetch data of NN from memory. It copies off of the stored
edits and fsimage files at the NN at the time of checkpoint request.
The link I pointed to you has the current stable source, you should be
able to read it for more details. Note that none of this is public
API.
On Tue, May 14, 2013 at 2:58 PM, Aditya exalter <ad...@gmail.com> wrote:
> As the secondary name node copies the data from name node for every 1 hour
> through http get and copies from main memory of name node so i want to use
> the same case as i want to get the data from the main memory as the
> secondary name node gets.I want the procedure and how to access it.
>
>
> On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Aditya,
>>
>> What do you mean by "addressing from memory"? Note that its difficult
>> for us to answer your questions perfectly if you don't also specify
>> exactly what you are looking to do and some of why.
>>
>> In any case, assuming you're use the maintenance 1.x release of HDFS
>> that does not have HA, you can look at the sources if you're
>> interested in the nitty gritty details:
>>
>> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
>> Sources are also available in the tarball of your Hadoop version.
>>
>> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
>> wrote:
>> > can i get complete details of how sec name node gets fsimage and edits
>> > file
>> > from name node through http get and http post with addressing from
>> > memory.
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
SNN does not fetch data of NN from memory. It copies off of the stored
edits and fsimage files at the NN at the time of checkpoint request.
The link I pointed to you has the current stable source, you should be
able to read it for more details. Note that none of this is public
API.
On Tue, May 14, 2013 at 2:58 PM, Aditya exalter <ad...@gmail.com> wrote:
> As the secondary name node copies the data from name node for every 1 hour
> through http get and copies from main memory of name node so i want to use
> the same case as i want to get the data from the main memory as the
> secondary name node gets.I want the procedure and how to access it.
>
>
> On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Aditya,
>>
>> What do you mean by "addressing from memory"? Note that its difficult
>> for us to answer your questions perfectly if you don't also specify
>> exactly what you are looking to do and some of why.
>>
>> In any case, assuming you're use the maintenance 1.x release of HDFS
>> that does not have HA, you can look at the sources if you're
>> interested in the nitty gritty details:
>>
>> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
>> Sources are also available in the tarball of your Hadoop version.
>>
>> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
>> wrote:
>> > can i get complete details of how sec name node gets fsimage and edits
>> > file
>> > from name node through http get and http post with addressing from
>> > memory.
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
SNN does not fetch data of NN from memory. It copies off of the stored
edits and fsimage files at the NN at the time of checkpoint request.
The link I pointed to you has the current stable source, you should be
able to read it for more details. Note that none of this is public
API.
On Tue, May 14, 2013 at 2:58 PM, Aditya exalter <ad...@gmail.com> wrote:
> As the secondary name node copies the data from name node for every 1 hour
> through http get and copies from main memory of name node so i want to use
> the same case as i want to get the data from the main memory as the
> secondary name node gets.I want the procedure and how to access it.
>
>
> On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Hi Aditya,
>>
>> What do you mean by "addressing from memory"? Note that its difficult
>> for us to answer your questions perfectly if you don't also specify
>> exactly what you are looking to do and some of why.
>>
>> In any case, assuming you're use the maintenance 1.x release of HDFS
>> that does not have HA, you can look at the sources if you're
>> interested in the nitty gritty details:
>>
>> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
>> Sources are also available in the tarball of your Hadoop version.
>>
>> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
>> wrote:
>> > can i get complete details of how sec name node gets fsimage and edits
>> > file
>> > from name node through http get and http post with addressing from
>> > memory.
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: How do secondary name node get files from name node
Posted by Aditya exalter <ad...@gmail.com>.
As the secondary name node copies the data from name node for every 1 hour
through http get and copies from main memory of name node so i want to use
the same case as i want to get the data from the main memory as the
secondary name node gets.I want the procedure and how to access it.
On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi Aditya,
>
> What do you mean by "addressing from memory"? Note that its difficult
> for us to answer your questions perfectly if you don't also specify
> exactly what you are looking to do and some of why.
>
> In any case, assuming you're use the maintenance 1.x release of HDFS
> that does not have HA, you can look at the sources if you're
> interested in the nitty gritty details:
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
> .
> Sources are also available in the tarball of your Hadoop version.
>
> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
> wrote:
> > can i get complete details of how sec name node gets fsimage and edits
> file
> > from name node through http get and http post with addressing from
> memory.
>
>
>
> --
> Harsh J
>
Re: How do secondary name node get files from name node
Posted by Aditya exalter <ad...@gmail.com>.
As the secondary name node copies the data from name node for every 1 hour
through http get and copies from main memory of name node so i want to use
the same case as i want to get the data from the main memory as the
secondary name node gets.I want the procedure and how to access it.
On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi Aditya,
>
> What do you mean by "addressing from memory"? Note that its difficult
> for us to answer your questions perfectly if you don't also specify
> exactly what you are looking to do and some of why.
>
> In any case, assuming you're use the maintenance 1.x release of HDFS
> that does not have HA, you can look at the sources if you're
> interested in the nitty gritty details:
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
> .
> Sources are also available in the tarball of your Hadoop version.
>
> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
> wrote:
> > can i get complete details of how sec name node gets fsimage and edits
> file
> > from name node through http get and http post with addressing from
> memory.
>
>
>
> --
> Harsh J
>
Re: How do secondary name node get files from name node
Posted by Aditya exalter <ad...@gmail.com>.
As the secondary name node copies the data from name node for every 1 hour
through http get and copies from main memory of name node so i want to use
the same case as i want to get the data from the main memory as the
secondary name node gets.I want the procedure and how to access it.
On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi Aditya,
>
> What do you mean by "addressing from memory"? Note that its difficult
> for us to answer your questions perfectly if you don't also specify
> exactly what you are looking to do and some of why.
>
> In any case, assuming you're use the maintenance 1.x release of HDFS
> that does not have HA, you can look at the sources if you're
> interested in the nitty gritty details:
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
> .
> Sources are also available in the tarball of your Hadoop version.
>
> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
> wrote:
> > can i get complete details of how sec name node gets fsimage and edits
> file
> > from name node through http get and http post with addressing from
> memory.
>
>
>
> --
> Harsh J
>
Re: How do secondary name node get files from name node
Posted by Aditya exalter <ad...@gmail.com>.
As the secondary name node copies the data from name node for every 1 hour
through http get and copies from main memory of name node so i want to use
the same case as i want to get the data from the main memory as the
secondary name node gets.I want the procedure and how to access it.
On Tue, May 14, 2013 at 2:47 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi Aditya,
>
> What do you mean by "addressing from memory"? Note that its difficult
> for us to answer your questions perfectly if you don't also specify
> exactly what you are looking to do and some of why.
>
> In any case, assuming you're use the maintenance 1.x release of HDFS
> that does not have HA, you can look at the sources if you're
> interested in the nitty gritty details:
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
> .
> Sources are also available in the tarball of your Hadoop version.
>
> On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com>
> wrote:
> > can i get complete details of how sec name node gets fsimage and edits
> file
> > from name node through http get and http post with addressing from
> memory.
>
>
>
> --
> Harsh J
>
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
Hi Aditya,
What do you mean by "addressing from memory"? Note that its difficult
for us to answer your questions perfectly if you don't also specify
exactly what you are looking to do and some of why.
In any case, assuming you're use the maintenance 1.x release of HDFS
that does not have HA, you can look at the sources if you're
interested in the nitty gritty details:
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
Sources are also available in the tarball of your Hadoop version.
On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com> wrote:
> can i get complete details of how sec name node gets fsimage and edits file
> from name node through http get and http post with addressing from memory.
--
Harsh J
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
Hi Aditya,
What do you mean by "addressing from memory"? Note that its difficult
for us to answer your questions perfectly if you don't also specify
exactly what you are looking to do and some of why.
In any case, assuming you're use the maintenance 1.x release of HDFS
that does not have HA, you can look at the sources if you're
interested in the nitty gritty details:
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
Sources are also available in the tarball of your Hadoop version.
On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com> wrote:
> can i get complete details of how sec name node gets fsimage and edits file
> from name node through http get and http post with addressing from memory.
--
Harsh J
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
Hi Aditya,
What do you mean by "addressing from memory"? Note that its difficult
for us to answer your questions perfectly if you don't also specify
exactly what you are looking to do and some of why.
In any case, assuming you're use the maintenance 1.x release of HDFS
that does not have HA, you can look at the sources if you're
interested in the nitty gritty details:
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
Sources are also available in the tarball of your Hadoop version.
On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com> wrote:
> can i get complete details of how sec name node gets fsimage and edits file
> from name node through http get and http post with addressing from memory.
--
Harsh J
Re: How do secondary name node get files from name node
Posted by Harsh J <ha...@cloudera.com>.
Hi Aditya,
What do you mean by "addressing from memory"? Note that its difficult
for us to answer your questions perfectly if you don't also specify
exactly what you are looking to do and some of why.
In any case, assuming you're use the maintenance 1.x release of HDFS
that does not have HA, you can look at the sources if you're
interested in the nitty gritty details:
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/hdfs/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java.
Sources are also available in the tarball of your Hadoop version.
On Tue, May 14, 2013 at 2:21 PM, Aditya exalter <ad...@gmail.com> wrote:
> can i get complete details of how sec name node gets fsimage and edits file
> from name node through http get and http post with addressing from memory.
--
Harsh J