Re: An Idea to make Hadoop Code setup easy

[Chris Nauroth <cn...@hortonworks.com>]

Mac cmake had a bug in the 2.x line that impacted the Hadoop build.  I
forget the details of the bug, but I'm currently building successfully with
cmake 3.0.2 on Mac.  Our BUILDING.txt file documents this requirement:

* CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac

If you're certain that cmake is installed and on the PATH, then I recommend
running cmake --version to check your version and upgrade if necessary.

If that's not the problem, then another possibly helpful debugging step
would be to run the cmake command manually.  Here is the relevant snippet
from the hadoop-common pom.xml.

                    <exec executable="cmake"
dir="${project.build.directory}/native" failonerror="true">
                      <arg line="${basedir}/src/
-DGENERATED_JAVAH=${project.build.directory}/native/javah
-DJVM_ARCH_DATA_MODEL=${sun.arch.data.model}
-DREQUIRE_BZIP2=${require.bzip2} -DREQUIRE_SNAPPY=${require.snappy}
-DCUSTOM_SNAPPY_PREFIX=${snappy.prefix} -DCUSTOM_SNAPPY_LIB=${snappy.lib}
-DCUSTOM_SNAPPY_INCLUDE=${snappy.include}
-DREQUIRE_OPENSSL=${require.openssl}
-DCUSTOM_OPENSSL_PREFIX=${openssl.prefix}
-DCUSTOM_OPENSSL_LIB=${openssl.lib}
-DCUSTOM_OPENSSL_INCLUDE=${openssl.include}
-DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
                    </exec>

Chris Nauroth
Hortonworks
http://hortonworks.com/


On Mon, Dec 22, 2014 at 5:11 PM, Raghavendra Vaidya <
raghavendra.vaidya@gmail.com> wrote:

> Cmake is installed. When I enter Cmake at console - it prints out the help
> for the command.
>
> On Monday, 22 December 2014, Rich Haase <rh...@pandora.com> wrote:
>
> > Do you have cmake installed on your machine?  Compiling the native
> > libraries is done with cmake, so if it¹s missing I would expect to see an
> > error like this.
> >
> > Rich Haase | Sr. Software Engineer | Pandora
> > m 303.887.1146 | rhaase@pandora.com <javascript:;>
> >
> >
> >
> >
> > On 12/21/14, 1:23 AM, "Raghavendra Vaidya" <raghavendra.vaidya@gmail.com
> > <javascript:;>>
> > wrote:
> >
> > >HEre is the exception I get when I execute maven install -Pnative ...
> > >please note that I have installed Protobuf and openssl using Brew on Mac
> > >OSX
> > >
> > >[ERROR] Failed to execute goal
> > >org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> > >hadoop-common: An Ant BuildException has occured: exec returned: 1
> > >
> > >[ERROR] around Ant part ...<exec
> >
> >dir="/Users/302000545/Documents/workspace/Hadoop/hadoop-common/hadoop-comm
> > >on-project/hadoop-common/target/native"
> > >executable="cmake" failonerror="true">... @ 4:161 in
> >
> >/Users/302000545/Documents/workspace/Hadoop/hadoop-common/hadoop-common-pr
> > >oject/hadoop-common/target/antrun/build-main.xml
> > >
> > >[ERROR] -> [Help 1]
> > >
> > >org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> execute
> > >goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on
> > >project
> > >hadoop-common: An Ant BuildException has occured: exec returned: 1
> > >
> > >around Ant part ...<exec
> >
> >dir="/Users/302000545/Documents/workspace/Hadoop/hadoop-common/hadoop-comm
> > >on-project/hadoop-common/target/native"
> > >executable="cmake" failonerror="true">... @ 4:161 in
> >
> >/Users/302000545/Documents/workspace/Hadoop/hadoop-common/hadoop-common-pr
> > >oject/hadoop-common/target/antrun/build-main.xml
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java
> > >:216)
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java
> > >:153)
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java
> > >:145)
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(Li
> > >fecycleModuleBuilder.java:116)
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(Li
> > >fecycleModuleBuilder.java:80)
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedB
> > >uilder.build(SingleThreadedBuilder.java:51)
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStar
> > >ter.java:120)
> > >
> > >at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347)
> > >
> > >at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154)
> > >
> > >at org.apache.maven.cli.MavenCli.execute(MavenCli.java:582)
> > >
> > >at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> > >
> > >at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> > >
> > >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> > >at
> >
> >sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> > >39)
> > >
> > >at
> >
> >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
> > >pl.java:25)
> > >
> > >at java.lang.reflect.Method.invoke(Method.java:597)
> > >
> > >at
> >
> >org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.
> > >java:289)
> > >
> > >at
> >
> >org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229
> > >)
> > >
> > >at
> >
> >org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launche
> > >r.java:415)
> > >
> > >at
> >
> >org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> > >
> > >Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
> > >BuildException has occured: exec returned: 1
> > >
> > >around Ant part ...<exec
> >
> >dir="/Users/302000545/Documents/workspace/Hadoop/hadoop-common/hadoop-comm
> > >on-project/hadoop-common/target/native"
> > >executable="cmake" failonerror="true">... @ 4:161 in
> >
> >/Users/302000545/Documents/workspace/Hadoop/hadoop-common/hadoop-common-pr
> > >oject/hadoop-common/target/antrun/build-main.xml
> > >
> > >at
> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> > >
> > >at
> >
> >org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuild
> > >PluginManager.java:132)
> > >
> > >at
> >
> >org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java
> > >:208)
> > >
> > >... 19 more
> > >
> > >Caused by:
> >
> >/Users/302000545/Documents/workspace/Hadoop/hadoop-common/hadoop-common-pr
> > >oject/hadoop-common/target/antrun/build-main.xml:4:
> > >exec returned: 1
> > >
> > >at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> > >
> > >at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> > >
> > >at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> > >
> > >at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> > >
> > >at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
> > >
> > >at
> >
> >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
> > >pl.java:25)
> > >
> > >at java.lang.reflect.Method.invoke(Method.java:597)
> > >
> > >at
> >
> >org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106
> > >)
> > >
> > >at org.apache.tools.ant.Task.perform(Task.java:348)
> > >
> > >at org.apache.tools.ant.Target.execute(Target.java:390)
> > >
> > >at org.apache.tools.ant.Target.performTasks(Target.java:411)
> > >
> > >at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> > >
> > >at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> > >
> > >at
> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> > >
> > >On Sun Dec 21 2014 at 8:27:47 AM Devopam Mittra <devopam@gmail.com
> > <javascript:;>> wrote:
> > >
> > >> Silly me, forgot to paste the link :
> > >> http://www.slideshare.net/devopam/hadoop-on-osx/
> > >> The basics remain the same, only tweaking required will be to
> > >>accommodate
> > >> new features if you are on an upgraded OS version.
> > >>
> > >>
> > >>
> > >> On Sun, Dec 21, 2014 at 8:25 AM, Devopam Mittra <devopam@gmail.com
> > <javascript:;>>
> > >>wrote:
> > >>
> > >> > Please check this out , and let me know what are the issues you are
> > >> facing.
> > >> >
> > >> > regards
> > >> > Dev
> > >> >
> > >> > On Sat, Dec 20, 2014 at 10:39 PM, Raghavendra Vaidya <
> > >> > raghavendra.vaidya@gmail.com <javascript:;>> wrote:
> > >> >
> > >> >> I have been struggling to setup hadoop code with native libraries
> on
> > >>mac
> > >> >> osx .... this gave me an idea to write a utility which can help
> setup
> > >> >> hadoop development environment either on intellij or eclipse
> > >>including
> > >> >> local libraries ....
> > >> >>
> > >> >> I would appreciate if some of the experienced developers respond to
> > >>this
> > >> >> thread and add notes on how to setup hadoop development
> environment I
> > >> can
> > >> >> start working on an automation utility
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Devopam Mittra
> > >> > Life and Relations are not binary
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Devopam Mittra
> > >> Life and Relations are not binary
> > >>
> >
> >
>
> --
> Regards
>
> Raghavendra Vaidya
> CIO, GE India Technology Center
>
> Sent from Gmail Mobile
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.