You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Karim Awara <ka...@kaust.edu.sa> on 2013/10/01 15:55:09 UTC
modify HDFS
Hi,
I want to modify the source code of HDFS to my usage, but I can't get any
handy sources for development of hdfs on eclipse. (I tried the hdfs
developer mailing list, but they are unresponsive). May you guide me?
--
Best Regards,
Karim Ahmed Awara
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
I checked out hadoop 2.1.0 beta through this command "svn checkout
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta/hadoop-common-2.1.0-beta"
and i built all the subprojects using maven and imported them to eclipse.
Now if I modified the HDFS code, how shall i test the hdfs subproject?
(knowing that the project is missing the configuration files to actually be
deployed on the cluster).
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 10:31 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> Hadoop 3.0 corresponds to trunk currently. I would recommend you to use
> branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance
> mode now. You can get all the branches from
> https://wiki.apache.org/hadoop/GitAndHadoop
>
> git clone git://git.apache.org/hadoop-common.git
>
> Please peruse https://hadoop.apache.org/docs/stable/cluster_setup.html to
> configure a distributed cluster. The block files should be in dfs.data.dir
> on the data nodes. You may also be able to use the Offline Image viewer to
> find all blocks and their locations from the fsimage file.
>
> HTH
> Ravi
>
>
> ------------------------------
> *From:* Pradeep Gollakota <pr...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Wednesday, October 2, 2013 9:29 AM
> *Subject:* Re: modify HDFS
>
> Since hadoop 3.0 is 2 major versions higher, it will be significantly
> different than working with hadoop 1.1.2. The hadoop-1.1 branch is
> available on SVN at
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
>
>
>
>
> On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
I checked out hadoop 2.1.0 beta through this command "svn checkout
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta/hadoop-common-2.1.0-beta"
and i built all the subprojects using maven and imported them to eclipse.
Now if I modified the HDFS code, how shall i test the hdfs subproject?
(knowing that the project is missing the configuration files to actually be
deployed on the cluster).
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 10:31 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> Hadoop 3.0 corresponds to trunk currently. I would recommend you to use
> branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance
> mode now. You can get all the branches from
> https://wiki.apache.org/hadoop/GitAndHadoop
>
> git clone git://git.apache.org/hadoop-common.git
>
> Please peruse https://hadoop.apache.org/docs/stable/cluster_setup.html to
> configure a distributed cluster. The block files should be in dfs.data.dir
> on the data nodes. You may also be able to use the Offline Image viewer to
> find all blocks and their locations from the fsimage file.
>
> HTH
> Ravi
>
>
> ------------------------------
> *From:* Pradeep Gollakota <pr...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Wednesday, October 2, 2013 9:29 AM
> *Subject:* Re: modify HDFS
>
> Since hadoop 3.0 is 2 major versions higher, it will be significantly
> different than working with hadoop 1.1.2. The hadoop-1.1 branch is
> available on SVN at
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
>
>
>
>
> On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
I checked out hadoop 2.1.0 beta through this command "svn checkout
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta/hadoop-common-2.1.0-beta"
and i built all the subprojects using maven and imported them to eclipse.
Now if I modified the HDFS code, how shall i test the hdfs subproject?
(knowing that the project is missing the configuration files to actually be
deployed on the cluster).
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 10:31 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> Hadoop 3.0 corresponds to trunk currently. I would recommend you to use
> branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance
> mode now. You can get all the branches from
> https://wiki.apache.org/hadoop/GitAndHadoop
>
> git clone git://git.apache.org/hadoop-common.git
>
> Please peruse https://hadoop.apache.org/docs/stable/cluster_setup.html to
> configure a distributed cluster. The block files should be in dfs.data.dir
> on the data nodes. You may also be able to use the Offline Image viewer to
> find all blocks and their locations from the fsimage file.
>
> HTH
> Ravi
>
>
> ------------------------------
> *From:* Pradeep Gollakota <pr...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Wednesday, October 2, 2013 9:29 AM
> *Subject:* Re: modify HDFS
>
> Since hadoop 3.0 is 2 major versions higher, it will be significantly
> different than working with hadoop 1.1.2. The hadoop-1.1 branch is
> available on SVN at
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
>
>
>
>
> On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
I checked out hadoop 2.1.0 beta through this command "svn checkout
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta/hadoop-common-2.1.0-beta"
and i built all the subprojects using maven and imported them to eclipse.
Now if I modified the HDFS code, how shall i test the hdfs subproject?
(knowing that the project is missing the configuration files to actually be
deployed on the cluster).
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 10:31 PM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> Hadoop 3.0 corresponds to trunk currently. I would recommend you to use
> branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance
> mode now. You can get all the branches from
> https://wiki.apache.org/hadoop/GitAndHadoop
>
> git clone git://git.apache.org/hadoop-common.git
>
> Please peruse https://hadoop.apache.org/docs/stable/cluster_setup.html to
> configure a distributed cluster. The block files should be in dfs.data.dir
> on the data nodes. You may also be able to use the Offline Image viewer to
> find all blocks and their locations from the fsimage file.
>
> HTH
> Ravi
>
>
> ------------------------------
> *From:* Pradeep Gollakota <pr...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Wednesday, October 2, 2013 9:29 AM
> *Subject:* Re: modify HDFS
>
> Since hadoop 3.0 is 2 major versions higher, it will be significantly
> different than working with hadoop 1.1.2. The hadoop-1.1 branch is
> available on SVN at
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
>
>
>
>
> On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
Hadoop 3.0 corresponds to trunk currently. I would recommend you to use branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance mode now. You can get all the branches from https://wiki.apache.org/hadoop/GitAndHadoop
git clone git://git.apache.org/hadoop-common.git
Please perusehttps://hadoop.apache.org/docs/stable/cluster_setup.html to configure a distributed cluster. The block files should be in dfs.data.dir on the data nodes. You may also be able to use the Offline Image viewer to find all blocks and their locations from the fsimage file.
HTH
Ravi
________________________________
From: Pradeep Gollakota <pr...@gmail.com>
To: user@hadoop.apache.org
Sent: Wednesday, October 2, 2013 9:29 AM
Subject: Re: modify HDFS
Since hadoop 3.0 is 2 major versions higher, it will be significantly different than working with hadoop 1.1.2. The hadoop-1.1 branch is available on SVN at http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa> wrote:
Hi all,
>
>
>My previous web surfing led me to such steps that I executed successfully. However, my issue is that what version of hadoop this is? (I believe it is hadoop 3.0 since it supports maven build). However, I want to modify a stable release (hadoop 1.1.2, which does not have the maven build support). Will working with the hadoop 3.0 make a lot of difference to me?
>
>
>Secondly, my goal is to modify the block placement strategy at HDFS in a distributed environment, and test such changes myself. Now, assuming I was successful in modifying the HDFS code, how to test such modifications (since the hadoop version is actually missing configuration files and so on that make it work in distributed environment)?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Karim!
>>
>>
>>You should read BUILDING.txt . I usually generate the eclipse files using
>>
>>
>>
>>mvn eclipse:eclipse
>>
>>
>>Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
>>mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>________________________________
>> From: Jagat Singh <ja...@gmail.com>
>>To: user@hadoop.apache.org
>>Sent: Tuesday, October 1, 2013 3:44 PM
>>Subject: Re: modify HDFS
>>
>>
>>
>>Hi,
>>What issue you having.
>>I Wrote it about here , might help you
>>Import it into eclipse as maven project
>>http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>>Thanks
>>On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>>Hi,
>>>
>>>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>>>
>>>
>>>
>>>--
>>>Best Regards,
>>>Karim Ahmed Awara
>>>
>>>
>>>________________________________
>>>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
>>
>>
>
>
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
Hadoop 3.0 corresponds to trunk currently. I would recommend you to use branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance mode now. You can get all the branches from https://wiki.apache.org/hadoop/GitAndHadoop
git clone git://git.apache.org/hadoop-common.git
Please perusehttps://hadoop.apache.org/docs/stable/cluster_setup.html to configure a distributed cluster. The block files should be in dfs.data.dir on the data nodes. You may also be able to use the Offline Image viewer to find all blocks and their locations from the fsimage file.
HTH
Ravi
________________________________
From: Pradeep Gollakota <pr...@gmail.com>
To: user@hadoop.apache.org
Sent: Wednesday, October 2, 2013 9:29 AM
Subject: Re: modify HDFS
Since hadoop 3.0 is 2 major versions higher, it will be significantly different than working with hadoop 1.1.2. The hadoop-1.1 branch is available on SVN at http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa> wrote:
Hi all,
>
>
>My previous web surfing led me to such steps that I executed successfully. However, my issue is that what version of hadoop this is? (I believe it is hadoop 3.0 since it supports maven build). However, I want to modify a stable release (hadoop 1.1.2, which does not have the maven build support). Will working with the hadoop 3.0 make a lot of difference to me?
>
>
>Secondly, my goal is to modify the block placement strategy at HDFS in a distributed environment, and test such changes myself. Now, assuming I was successful in modifying the HDFS code, how to test such modifications (since the hadoop version is actually missing configuration files and so on that make it work in distributed environment)?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Karim!
>>
>>
>>You should read BUILDING.txt . I usually generate the eclipse files using
>>
>>
>>
>>mvn eclipse:eclipse
>>
>>
>>Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
>>mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>________________________________
>> From: Jagat Singh <ja...@gmail.com>
>>To: user@hadoop.apache.org
>>Sent: Tuesday, October 1, 2013 3:44 PM
>>Subject: Re: modify HDFS
>>
>>
>>
>>Hi,
>>What issue you having.
>>I Wrote it about here , might help you
>>Import it into eclipse as maven project
>>http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>>Thanks
>>On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>>Hi,
>>>
>>>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>>>
>>>
>>>
>>>--
>>>Best Regards,
>>>Karim Ahmed Awara
>>>
>>>
>>>________________________________
>>>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
>>
>>
>
>
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
Hadoop 3.0 corresponds to trunk currently. I would recommend you to use branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance mode now. You can get all the branches from https://wiki.apache.org/hadoop/GitAndHadoop
git clone git://git.apache.org/hadoop-common.git
Please perusehttps://hadoop.apache.org/docs/stable/cluster_setup.html to configure a distributed cluster. The block files should be in dfs.data.dir on the data nodes. You may also be able to use the Offline Image viewer to find all blocks and their locations from the fsimage file.
HTH
Ravi
________________________________
From: Pradeep Gollakota <pr...@gmail.com>
To: user@hadoop.apache.org
Sent: Wednesday, October 2, 2013 9:29 AM
Subject: Re: modify HDFS
Since hadoop 3.0 is 2 major versions higher, it will be significantly different than working with hadoop 1.1.2. The hadoop-1.1 branch is available on SVN at http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa> wrote:
Hi all,
>
>
>My previous web surfing led me to such steps that I executed successfully. However, my issue is that what version of hadoop this is? (I believe it is hadoop 3.0 since it supports maven build). However, I want to modify a stable release (hadoop 1.1.2, which does not have the maven build support). Will working with the hadoop 3.0 make a lot of difference to me?
>
>
>Secondly, my goal is to modify the block placement strategy at HDFS in a distributed environment, and test such changes myself. Now, assuming I was successful in modifying the HDFS code, how to test such modifications (since the hadoop version is actually missing configuration files and so on that make it work in distributed environment)?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Karim!
>>
>>
>>You should read BUILDING.txt . I usually generate the eclipse files using
>>
>>
>>
>>mvn eclipse:eclipse
>>
>>
>>Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
>>mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>________________________________
>> From: Jagat Singh <ja...@gmail.com>
>>To: user@hadoop.apache.org
>>Sent: Tuesday, October 1, 2013 3:44 PM
>>Subject: Re: modify HDFS
>>
>>
>>
>>Hi,
>>What issue you having.
>>I Wrote it about here , might help you
>>Import it into eclipse as maven project
>>http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>>Thanks
>>On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>>Hi,
>>>
>>>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>>>
>>>
>>>
>>>--
>>>Best Regards,
>>>Karim Ahmed Awara
>>>
>>>
>>>________________________________
>>>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
>>
>>
>
>
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
Hadoop 3.0 corresponds to trunk currently. I would recommend you to use branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance mode now. You can get all the branches from https://wiki.apache.org/hadoop/GitAndHadoop
git clone git://git.apache.org/hadoop-common.git
Please perusehttps://hadoop.apache.org/docs/stable/cluster_setup.html to configure a distributed cluster. The block files should be in dfs.data.dir on the data nodes. You may also be able to use the Offline Image viewer to find all blocks and their locations from the fsimage file.
HTH
Ravi
________________________________
From: Pradeep Gollakota <pr...@gmail.com>
To: user@hadoop.apache.org
Sent: Wednesday, October 2, 2013 9:29 AM
Subject: Re: modify HDFS
Since hadoop 3.0 is 2 major versions higher, it will be significantly different than working with hadoop 1.1.2. The hadoop-1.1 branch is available on SVN at http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa> wrote:
Hi all,
>
>
>My previous web surfing led me to such steps that I executed successfully. However, my issue is that what version of hadoop this is? (I believe it is hadoop 3.0 since it supports maven build). However, I want to modify a stable release (hadoop 1.1.2, which does not have the maven build support). Will working with the hadoop 3.0 make a lot of difference to me?
>
>
>Secondly, my goal is to modify the block placement strategy at HDFS in a distributed environment, and test such changes myself. Now, assuming I was successful in modifying the HDFS code, how to test such modifications (since the hadoop version is actually missing configuration files and so on that make it work in distributed environment)?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>Karim!
>>
>>
>>You should read BUILDING.txt . I usually generate the eclipse files using
>>
>>
>>
>>mvn eclipse:eclipse
>>
>>
>>Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
>>mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>________________________________
>> From: Jagat Singh <ja...@gmail.com>
>>To: user@hadoop.apache.org
>>Sent: Tuesday, October 1, 2013 3:44 PM
>>Subject: Re: modify HDFS
>>
>>
>>
>>Hi,
>>What issue you having.
>>I Wrote it about here , might help you
>>Import it into eclipse as maven project
>>http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>>Thanks
>>On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>>Hi,
>>>
>>>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>>>
>>>
>>>
>>>--
>>>Best Regards,
>>>Karim Ahmed Awara
>>>
>>>
>>>________________________________
>>>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
>>
>>
>
>
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Pradeep Gollakota <pr...@gmail.com>.
Since hadoop 3.0 is 2 major versions higher, it will be significantly
different than working with hadoop 1.1.2. The hadoop-1.1 branch is
available on SVN at
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>> Karim!
>>
>> You should read BUILDING.txt . I usually generate the eclipse files using
>>
>> mvn eclipse:eclipse
>>
>> Then I can import all the projects into eclipse as eclipse projects. This
>> is useful for code navigation and completion etc. however I still build
>> using command line:
>> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>> HTH
>> Ravi
>>
>> ------------------------------
>> *From:* Jagat Singh <ja...@gmail.com>
>> *To:* user@hadoop.apache.org
>> *Sent:* Tuesday, October 1, 2013 3:44 PM
>> *Subject:* Re: modify HDFS
>>
>> Hi,
>> What issue you having.
>> I Wrote it about here , might help you
>> Import it into eclipse as maven project
>>
>> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>> Thanks
>> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>> Hi,
>>
>> I want to modify the source code of HDFS to my usage, but I can't get any
>> handy sources for development of hdfs on eclipse. (I tried the hdfs
>> developer mailing list, but they are unresponsive). May you guide me?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>>
>>
>>
>>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
Re: modify HDFS
Posted by Pradeep Gollakota <pr...@gmail.com>.
Since hadoop 3.0 is 2 major versions higher, it will be significantly
different than working with hadoop 1.1.2. The hadoop-1.1 branch is
available on SVN at
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>> Karim!
>>
>> You should read BUILDING.txt . I usually generate the eclipse files using
>>
>> mvn eclipse:eclipse
>>
>> Then I can import all the projects into eclipse as eclipse projects. This
>> is useful for code navigation and completion etc. however I still build
>> using command line:
>> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>> HTH
>> Ravi
>>
>> ------------------------------
>> *From:* Jagat Singh <ja...@gmail.com>
>> *To:* user@hadoop.apache.org
>> *Sent:* Tuesday, October 1, 2013 3:44 PM
>> *Subject:* Re: modify HDFS
>>
>> Hi,
>> What issue you having.
>> I Wrote it about here , might help you
>> Import it into eclipse as maven project
>>
>> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>> Thanks
>> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>> Hi,
>>
>> I want to modify the source code of HDFS to my usage, but I can't get any
>> handy sources for development of hdfs on eclipse. (I tried the hdfs
>> developer mailing list, but they are unresponsive). May you guide me?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>>
>>
>>
>>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
Re: modify HDFS
Posted by Pradeep Gollakota <pr...@gmail.com>.
Since hadoop 3.0 is 2 major versions higher, it will be significantly
different than working with hadoop 1.1.2. The hadoop-1.1 branch is
available on SVN at
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>> Karim!
>>
>> You should read BUILDING.txt . I usually generate the eclipse files using
>>
>> mvn eclipse:eclipse
>>
>> Then I can import all the projects into eclipse as eclipse projects. This
>> is useful for code navigation and completion etc. however I still build
>> using command line:
>> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>> HTH
>> Ravi
>>
>> ------------------------------
>> *From:* Jagat Singh <ja...@gmail.com>
>> *To:* user@hadoop.apache.org
>> *Sent:* Tuesday, October 1, 2013 3:44 PM
>> *Subject:* Re: modify HDFS
>>
>> Hi,
>> What issue you having.
>> I Wrote it about here , might help you
>> Import it into eclipse as maven project
>>
>> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>> Thanks
>> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>> Hi,
>>
>> I want to modify the source code of HDFS to my usage, but I can't get any
>> handy sources for development of hdfs on eclipse. (I tried the hdfs
>> developer mailing list, but they are unresponsive). May you guide me?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>>
>>
>>
>>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
Re: modify HDFS
Posted by Pradeep Gollakota <pr...@gmail.com>.
Since hadoop 3.0 is 2 major versions higher, it will be significantly
different than working with hadoop 1.1.2. The hadoop-1.1 branch is
available on SVN at
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support). Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
>
>> Karim!
>>
>> You should read BUILDING.txt . I usually generate the eclipse files using
>>
>> mvn eclipse:eclipse
>>
>> Then I can import all the projects into eclipse as eclipse projects. This
>> is useful for code navigation and completion etc. however I still build
>> using command line:
>> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>> HTH
>> Ravi
>>
>> ------------------------------
>> *From:* Jagat Singh <ja...@gmail.com>
>> *To:* user@hadoop.apache.org
>> *Sent:* Tuesday, October 1, 2013 3:44 PM
>> *Subject:* Re: modify HDFS
>>
>> Hi,
>> What issue you having.
>> I Wrote it about here , might help you
>> Import it into eclipse as maven project
>>
>> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>> Thanks
>> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>>
>> Hi,
>>
>> I want to modify the source code of HDFS to my usage, but I can't get any
>> handy sources for development of hdfs on eclipse. (I tried the hdfs
>> developer mailing list, but they are unresponsive). May you guide me?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>>
>>
>>
>>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
My previous web surfing led me to such steps that I executed successfully.
However, my issue is that what version of hadoop this is? (I believe it is
hadoop 3.0 since it supports maven build). However, I want to modify a
stable release (hadoop 1.1.2, which does not have the maven build
support). Will working with the hadoop 3.0 make a lot of difference to me?
Secondly, my goal is to modify the block placement strategy at HDFS in a
distributed environment, and test such changes myself. Now, assuming I was
successful in modifying the HDFS code, how to test such modifications
(since the hadoop version is actually missing configuration files and so on
that make it work in distributed environment)?
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
My previous web surfing led me to such steps that I executed successfully.
However, my issue is that what version of hadoop this is? (I believe it is
hadoop 3.0 since it supports maven build). However, I want to modify a
stable release (hadoop 1.1.2, which does not have the maven build
support). Will working with the hadoop 3.0 make a lot of difference to me?
Secondly, my goal is to modify the block placement strategy at HDFS in a
distributed environment, and test such changes myself. Now, assuming I was
successful in modifying the HDFS code, how to test such modifications
(since the hadoop version is actually missing configuration files and so on
that make it work in distributed environment)?
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
My previous web surfing led me to such steps that I executed successfully.
However, my issue is that what version of hadoop this is? (I believe it is
hadoop 3.0 since it supports maven build). However, I want to modify a
stable release (hadoop 1.1.2, which does not have the maven build
support). Will working with the hadoop 3.0 make a lot of difference to me?
Secondly, my goal is to modify the block placement strategy at HDFS in a
distributed environment, and test such changes myself. Now, assuming I was
successful in modifying the HDFS code, how to test such modifications
(since the hadoop version is actually missing configuration files and so on
that make it work in distributed environment)?
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Karim Awara <ka...@kaust.edu.sa>.
Hi all,
My previous web surfing led me to such steps that I executed successfully.
However, my issue is that what version of hadoop this is? (I believe it is
hadoop 3.0 since it supports maven build). However, I want to modify a
stable release (hadoop 1.1.2, which does not have the maven build
support). Will working with the hadoop 3.0 make a lot of difference to me?
Secondly, my goal is to modify the block placement strategy at HDFS in a
distributed environment, and test such changes myself. Now, assuming I was
successful in modifying the HDFS code, how to test such modifications
(since the hadoop version is actually missing configuration files and so on
that make it work in distributed environment)?
--
Best Regards,
Karim Ahmed Awara
On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ra...@ymail.com> wrote:
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
> ------------------------------
> *From:* Jagat Singh <ja...@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
You should read BUILDING.txt . I usually generate the eclipse files using
mvn eclipse:eclipse
Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
HTH
Ravi
________________________________
From: Jagat Singh <ja...@gmail.com>
To: user@hadoop.apache.org
Sent: Tuesday, October 1, 2013 3:44 PM
Subject: Re: modify HDFS
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
Hi,
>
>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
You should read BUILDING.txt . I usually generate the eclipse files using
mvn eclipse:eclipse
Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
HTH
Ravi
________________________________
From: Jagat Singh <ja...@gmail.com>
To: user@hadoop.apache.org
Sent: Tuesday, October 1, 2013 3:44 PM
Subject: Re: modify HDFS
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
Hi,
>
>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
You should read BUILDING.txt . I usually generate the eclipse files using
mvn eclipse:eclipse
Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
HTH
Ravi
________________________________
From: Jagat Singh <ja...@gmail.com>
To: user@hadoop.apache.org
Sent: Tuesday, October 1, 2013 3:44 PM
Subject: Re: modify HDFS
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
Hi,
>
>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Karim!
You should read BUILDING.txt . I usually generate the eclipse files using
mvn eclipse:eclipse
Then I can import all the projects into eclipse as eclipse projects. This is useful for code navigation and completion etc. however I still build using command line:
mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
HTH
Ravi
________________________________
From: Jagat Singh <ja...@gmail.com>
To: user@hadoop.apache.org
Sent: Tuesday, October 1, 2013 3:44 PM
Subject: Re: modify HDFS
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
Hi,
>
>I want to modify the source code of HDFS to my usage, but I can't get any handy sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but they are unresponsive). May you guide me?
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
Re: modify HDFS
Posted by Jagat Singh <ja...@gmail.com>.
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: modify HDFS
Posted by Jagat Singh <ja...@gmail.com>.
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: modify HDFS
Posted by Jagat Singh <ja...@gmail.com>.
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: modify HDFS
Posted by Jagat Singh <ja...@gmail.com>.
Hi,
What issue you having.
I Wrote it about here , might help you
Import it into eclipse as maven project
http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
Thanks
On 01/10/2013 11:56 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.