You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@bookkeeper.apache.org by Savitha Devi Inbasekaran <Sa...@smartek21.com> on 2013/01/29 08:00:22 UTC

Integrating namenode with bookkeeper

Hi Ivan,
I want to integrate namenode with bookkeeper. I followed the steps which you specified in the link http://mail-archives.apache.org/mod_mbox/zookeeper-bookkeeper-user/201203.mbox/browser .
While setting up Bookkeeper Journal Manager I got struck up. The trunk which I downloaded from github doesn't have hadoop-common-SNAPSHOT, hadoop-hdfs-SNAPSHOT and
hadoop-hdfs-bkjournal-0.24.0-SNAPSHOT.jar. Please help me.

Thanks,
Savitha

RE: Integrating namenode with bookkeeper

Posted by Savitha Devi Inbasekaran <Sa...@smartek21.com>.

Thanks Uma. I was able generate all the necessary jar files. But when I format the hdfs namenode I got the following error.

bin/hdfs: line 162: exec: : not found

Thanks,
Savitha
From: Uma Maheswara Rao G [mailto:maheswara@huawei.com]
Sent: Wednesday, January 30, 2013 6:24 PM
To: bookkeeper-user@zookeeper.apache.org
Cc: Ivan Kelly
Subject: RE: Integrating namenode with bookkeeper

I am not sure which commens you executed for building.

See the section in that document:

=========================

Building BookKeeper Journal Manager plugin jar

To generate the distribution packages for BK journal, do the following.

$ mvn clean package -Pdist

This will generate a jar with the BookKeeperJournalManager, all the dependencies needed by the journal manager, hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target/hadoop-hdfs-bkjournal-VERSION.jar
==================================================

After downloading Hadoop code with svn checkout, go to folder hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal
execute the folowing command:
mvn clean install -DskipTests

this will clean everything and generate the jars cleanly for you. I gave -DskipTests option to just skipping test cases running(that will take time if you run).

After this go to hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target folder. Here you can see the jar file.

Please note that: I hope you have maven installed and protocol buffers also installed. If not please install them first. They needed while building.

>From above trace, looks like you are executing some ant commands? BKJM build supports maven. please try with the above suggested commands.

________________________________
From: Savitha Devi Inbasekaran [Savithadi@smartek21.com]
Sent: Wednesday, January 30, 2013 5:06 PM
To: bookkeeper-user@zookeeper.apache.org<ma...@zookeeper.apache.org>
Cc: Ivan Kelly
Subject: RE: Integrating namenode with bookkeeper
Hi uma,
I followed the steps in the documentation to generate bkjournal jar files but I got struck again and following is the error.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (create-testdirs) on project hadoop-hdfs: Error executing ant tasks: /hadoop-common/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml (No such file or directory)

Thanks,
Savitha

From: Uma Maheswara Rao G [mailto:maheswara@huawei.com]
Sent: Tuesday, January 29, 2013 1:47 PM
To: bookkeeper-user@zookeeper.apache.org<ma...@zookeeper.apache.org>
Cc: Ivan Kelly
Subject: FW: Integrating namenode with bookkeeper

Hi Savitha,

It is already available with Hadoop-2.x releases

Please try the latest release Hadoop 2.0.2 release.

Take a look at the integration releated documentation here. (section BookKeeper_as_a_Shared_storage at the end)

http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html

This will give you the info that how you can generate the hadoop-hdfs-bkjournal jar files and place in lib folder of HDFS.

Here is the Hadoop release site: http://hadoop.apache.org/releases.html#9+October%2C+2012%3A+Release+2.0.2-alpha+available

Regards,

Uma

________________________________
From: Savitha Devi Inbasekaran [Savithadi@smartek21.com]
Sent: Tuesday, January 29, 2013 12:30 PM
To: Ivan Kelly
Cc: bookkeeper-user@zookeeper.apache.org<ma...@zookeeper.apache.org>
Subject: Integrating namenode with bookkeeper

Hi Ivan,
I want to integrate namenode with bookkeeper. I followed the steps which you specified in the link http://mail-archives.apache.org/mod_mbox/zookeeper-bookkeeper-user/201203.mbox/browser .
While setting up Bookkeeper Journal Manager I got struck up. The trunk which I downloaded from github doesn't have hadoop-common-SNAPSHOT, hadoop-hdfs-SNAPSHOT and
hadoop-hdfs-bkjournal-0.24.0-SNAPSHOT.jar. Please help me.

Thanks,
Savitha

RE: Integrating namenode with bookkeeper

Posted by Uma Maheswara Rao G <ma...@huawei.com>.


I am not sure which commens you executed for building.



See the section in that document:

=========================

Building BookKeeper Journal Manager plugin jar

To generate the distribution packages for BK journal, do the following.

$ mvn clean package -Pdist

This will generate a jar with the BookKeeperJournalManager, all the dependencies needed by the journal manager, hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target/hadoop-hdfs-bkjournal-VERSION.jar

==================================================

After downloading Hadoop code with svn checkout, go to folder hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal
execute the folowing command:
mvn clean install -DskipTests

this will clean everything and generate the jars cleanly for you. I gave -DskipTests option to just skipping test cases running(that will take time if you run).

After this go to hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target folder. Here you can see the jar file.

Please note that: I hope you have maven installed and protocol buffers also installed. If not please install them first. They needed while building.

>From above trace, looks like you are executing some ant commands? BKJM build supports maven. please try with the above suggested commands.

________________________________
From: Savitha Devi Inbasekaran [Savithadi@smartek21.com]
Sent: Wednesday, January 30, 2013 5:06 PM
To: bookkeeper-user@zookeeper.apache.org
Cc: Ivan Kelly
Subject: RE: Integrating namenode with bookkeeper

Hi uma,
I followed the steps in the documentation to generate bkjournal jar files but I got struck again and following is the error.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (create-testdirs) on project hadoop-hdfs: Error executing ant tasks: /hadoop-common/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml (No such file or directory)

Thanks,
Savitha

From: Uma Maheswara Rao G [mailto:maheswara@huawei.com]
Sent: Tuesday, January 29, 2013 1:47 PM
To: bookkeeper-user@zookeeper.apache.org
Cc: Ivan Kelly
Subject: FW: Integrating namenode with bookkeeper


Hi Savitha,



It is already available with Hadoop-2.x releases



Please try the latest release Hadoop 2.0.2 release.



Take a look at the integration releated documentation here. (section BookKeeper_as_a_Shared_storage at the end)

http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html

This will give you the info that how you can generate the hadoop-hdfs-bkjournal jar files and place in lib folder of HDFS.



Here is the Hadoop release site: http://hadoop.apache.org/releases.html#9+October%2C+2012%3A+Release+2.0.2-alpha+available



Regards,

Uma

________________________________
From: Savitha Devi Inbasekaran [Savithadi@smartek21.com]
Sent: Tuesday, January 29, 2013 12:30 PM
To: Ivan Kelly
Cc: bookkeeper-user@zookeeper.apache.org<ma...@zookeeper.apache.org>
Subject: Integrating namenode with bookkeeper

Hi Ivan,
I want to integrate namenode with bookkeeper. I followed the steps which you specified in the link http://mail-archives.apache.org/mod_mbox/zookeeper-bookkeeper-user/201203.mbox/browser .
While setting up Bookkeeper Journal Manager I got struck up. The trunk which I downloaded from github doesn’t have hadoop-common-SNAPSHOT, hadoop-hdfs-SNAPSHOT and
hadoop-hdfs-bkjournal-0.24.0-SNAPSHOT.jar. Please help me.

Thanks,
Savitha

RE: Integrating namenode with bookkeeper

Posted by Savitha Devi Inbasekaran <Sa...@smartek21.com>.

Hi uma,
I followed the steps in the documentation to generate bkjournal jar files but I got struck again and following is the error.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (create-testdirs) on project hadoop-hdfs: Error executing ant tasks: /hadoop-common/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml (No such file or directory)

Thanks,
Savitha

From: Uma Maheswara Rao G [mailto:maheswara@huawei.com]
Sent: Tuesday, January 29, 2013 1:47 PM
To: bookkeeper-user@zookeeper.apache.org
Cc: Ivan Kelly
Subject: FW: Integrating namenode with bookkeeper

Hi Savitha,

It is already available with Hadoop-2.x releases

Please try the latest release Hadoop 2.0.2 release.

Take a look at the integration releated documentation here. (section BookKeeper_as_a_Shared_storage at the end)

http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html

This will give you the info that how you can generate the hadoop-hdfs-bkjournal jar files and place in lib folder of HDFS.

Here is the Hadoop release site: http://hadoop.apache.org/releases.html#9+October%2C+2012%3A+Release+2.0.2-alpha+available

Regards,

Uma

________________________________
From: Savitha Devi Inbasekaran [Savithadi@smartek21.com]
Sent: Tuesday, January 29, 2013 12:30 PM
To: Ivan Kelly
Cc: bookkeeper-user@zookeeper.apache.org<ma...@zookeeper.apache.org>
Subject: Integrating namenode with bookkeeper

Hi Ivan,
I want to integrate namenode with bookkeeper. I followed the steps which you specified in the link http://mail-archives.apache.org/mod_mbox/zookeeper-bookkeeper-user/201203.mbox/browser .
While setting up Bookkeeper Journal Manager I got struck up. The trunk which I downloaded from github doesn't have hadoop-common-SNAPSHOT, hadoop-hdfs-SNAPSHOT and
hadoop-hdfs-bkjournal-0.24.0-SNAPSHOT.jar. Please help me.

Thanks,
Savitha

FW: Integrating namenode with bookkeeper

Posted by Uma Maheswara Rao G <ma...@huawei.com>.

Hi Savitha,



It is already available with Hadoop-2.x releases



Please try the latest release Hadoop 2.0.2 release.



Take a look at the integration releated documentation here. (section BookKeeper_as_a_Shared_storage at the end)

http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html

This will give you the info that how you can generate the hadoop-hdfs-bkjournal jar files and place in lib folder of HDFS.



Here is the Hadoop release site: http://hadoop.apache.org/releases.html#9+October%2C+2012%3A+Release+2.0.2-alpha+available



Regards,

Uma

________________________________
From: Savitha Devi Inbasekaran [Savithadi@smartek21.com]
Sent: Tuesday, January 29, 2013 12:30 PM
To: Ivan Kelly
Cc: bookkeeper-user@zookeeper.apache.org
Subject: Integrating namenode with bookkeeper


Hi Ivan,
I want to integrate namenode with bookkeeper. I followed the steps which you specified in the link http://mail-archives.apache.org/mod_mbox/zookeeper-bookkeeper-user/201203.mbox/browser .
While setting up Bookkeeper Journal Manager I got struck up. The trunk which I downloaded from github doesn’t have hadoop-common-SNAPSHOT, hadoop-hdfs-SNAPSHOT and
hadoop-hdfs-bkjournal-0.24.0-SNAPSHOT.jar. Please help me.

Thanks,
Savitha