You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by David Parks <da...@yahoo.com> on 2013/04/17 06:26:56 UTC
Mapreduce jobs to download job input from across the internet
For a set of jobs to run I need to download about 100GB of data from the
internet (~1000 files of varying sizes from ~10 different domains).
Currently I do this in a simple linux script as it's easy to script FTP,
curl, and the like. But it's a mess to maintain a separate server for that
process. I'd rather it run in mapreduce. Just give it a bill of materials
and let it go about downloading it, retrying as necessary to deal with iffy
network conditions.
I wrote one such job to craw images we need to acquire, and it was the
royalist of royal pains. I wonder if there are any good approaches to this
kind of data acquisition task in Hadoop. It would certainly be nicer just to
schedule a data-acquisition job ahead of the processing jobs in Oozie rather
than try to maintain synchronization between the download processes and the
jobs.
Ideas?
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Could you copy the CMake log? Should be before the beginning of what you
had copied; we might see what the actual problem is in there.
Best,
Silvina
On 28 April 2014 15:43, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi Silvina,
>
> Thanks for your reply.
>
> cmake is installed, I try the following:
>
> apt-get install cmake
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> cmake is already the newest version.
> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>
> regards
>
>
> On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com>
> wrote:
>
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to install Hadoop 2.4.0 from source, I got the following
>> error, please help!!
>>
>> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>>
>> Regards
>>
>>
>> O/S Ubuntu: 12.04 (64-bit)
>> Java: java version "1.6.0_45"
>> protoc —version: libprotoc 2.5.0
>>
>>
>> Command: mvn package -Pdist,native -DskipTests -Dtar -X
>> Error message:
>>
>> [INFO] Total time: 18.096s
>> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
>> [INFO] Final Memory: 59M/1303M
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> [ERROR] around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> [ERROR] -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
>> BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
>> ... 19 more
>> Caused by:
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
>> exec returned: 1
>> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
>> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
>> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
>> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>> at org.apache.tools.ant.Task.perform(Task.java:348)
>> at org.apache.tools.ant.Target.execute(Target.java:390)
>> at org.apache.tools.ant.Target.performTasks(Target.java:411)
>> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
>> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
>> ... 21 more
>> [ERROR]
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :hadoop-common
>>
>
>
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Could you copy the CMake log? Should be before the beginning of what you
had copied; we might see what the actual problem is in there.
Best,
Silvina
On 28 April 2014 15:43, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi Silvina,
>
> Thanks for your reply.
>
> cmake is installed, I try the following:
>
> apt-get install cmake
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> cmake is already the newest version.
> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>
> regards
>
>
> On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com>
> wrote:
>
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to install Hadoop 2.4.0 from source, I got the following
>> error, please help!!
>>
>> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>>
>> Regards
>>
>>
>> O/S Ubuntu: 12.04 (64-bit)
>> Java: java version "1.6.0_45"
>> protoc —version: libprotoc 2.5.0
>>
>>
>> Command: mvn package -Pdist,native -DskipTests -Dtar -X
>> Error message:
>>
>> [INFO] Total time: 18.096s
>> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
>> [INFO] Final Memory: 59M/1303M
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> [ERROR] around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> [ERROR] -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
>> BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
>> ... 19 more
>> Caused by:
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
>> exec returned: 1
>> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
>> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
>> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
>> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>> at org.apache.tools.ant.Task.perform(Task.java:348)
>> at org.apache.tools.ant.Target.execute(Target.java:390)
>> at org.apache.tools.ant.Target.performTasks(Target.java:411)
>> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
>> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
>> ... 21 more
>> [ERROR]
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :hadoop-common
>>
>
>
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Could you copy the CMake log? Should be before the beginning of what you
had copied; we might see what the actual problem is in there.
Best,
Silvina
On 28 April 2014 15:43, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi Silvina,
>
> Thanks for your reply.
>
> cmake is installed, I try the following:
>
> apt-get install cmake
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> cmake is already the newest version.
> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>
> regards
>
>
> On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com>
> wrote:
>
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to install Hadoop 2.4.0 from source, I got the following
>> error, please help!!
>>
>> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>>
>> Regards
>>
>>
>> O/S Ubuntu: 12.04 (64-bit)
>> Java: java version "1.6.0_45"
>> protoc —version: libprotoc 2.5.0
>>
>>
>> Command: mvn package -Pdist,native -DskipTests -Dtar -X
>> Error message:
>>
>> [INFO] Total time: 18.096s
>> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
>> [INFO] Final Memory: 59M/1303M
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> [ERROR] around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> [ERROR] -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
>> BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
>> ... 19 more
>> Caused by:
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
>> exec returned: 1
>> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
>> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
>> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
>> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>> at org.apache.tools.ant.Task.perform(Task.java:348)
>> at org.apache.tools.ant.Target.execute(Target.java:390)
>> at org.apache.tools.ant.Target.performTasks(Target.java:411)
>> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
>> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
>> ... 21 more
>> [ERROR]
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :hadoop-common
>>
>
>
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Could you copy the CMake log? Should be before the beginning of what you
had copied; we might see what the actual problem is in there.
Best,
Silvina
On 28 April 2014 15:43, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi Silvina,
>
> Thanks for your reply.
>
> cmake is installed, I try the following:
>
> apt-get install cmake
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> cmake is already the newest version.
> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>
> regards
>
>
> On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com>
> wrote:
>
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to install Hadoop 2.4.0 from source, I got the following
>> error, please help!!
>>
>> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>>
>> Regards
>>
>>
>> O/S Ubuntu: 12.04 (64-bit)
>> Java: java version "1.6.0_45"
>> protoc —version: libprotoc 2.5.0
>>
>>
>> Command: mvn package -Pdist,native -DskipTests -Dtar -X
>> Error message:
>>
>> [INFO] Total time: 18.096s
>> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
>> [INFO] Final Memory: 59M/1303M
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> [ERROR] around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> [ERROR] -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
>> hadoop-common: An Ant BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
>> BuildException has occured: exec returned: 1
>> around Ant part ...<exec
>> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
>> executable="cmake" failonerror="true">... @ 4:138 in
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
>> ... 19 more
>> Caused by:
>> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
>> exec returned: 1
>> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
>> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
>> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
>> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>> at org.apache.tools.ant.Task.perform(Task.java:348)
>> at org.apache.tools.ant.Target.execute(Target.java:390)
>> at org.apache.tools.ant.Target.performTasks(Target.java:411)
>> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
>> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
>> at
>> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
>> ... 21 more
>> [ERROR]
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :hadoop-common
>>
>
>
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi Silvina,
Thanks for your reply.
cmake is installed, I try the following:
apt-get install cmake
Reading package lists... Done
Building dependency tree
Reading state information... Done
cmake is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
regards
On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com> wrote:
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi Silvina,
Thanks for your reply.
cmake is installed, I try the following:
apt-get install cmake
Reading package lists... Done
Building dependency tree
Reading state information... Done
cmake is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
regards
On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com> wrote:
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi Silvina,
Thanks for your reply.
cmake is installed, I try the following:
apt-get install cmake
Reading package lists... Done
Building dependency tree
Reading state information... Done
cmake is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
regards
On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com> wrote:
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi Silvina,
Thanks for your reply.
cmake is installed, I try the following:
apt-get install cmake
Reading package lists... Done
Building dependency tree
Reading state information... Done
cmake is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
regards
On 28 Apr, 2014, at 7:43 pm, Silvina Caíno Lores <si...@gmail.com> wrote:
> Are you sure that CMake is installed?
>
> Best,
> Silvina
>
>
> On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Are you sure that CMake is installed?
Best,
Silvina
On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following
> error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
> BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by:
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
> exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Are you sure that CMake is installed?
Best,
Silvina
On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following
> error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
> BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by:
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
> exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Are you sure that CMake is installed?
Best,
Silvina
On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following
> error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
> BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by:
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
> exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Re: Install Hadoop 2.4.0 from Source - Compile error
Posted by Silvina Caíno Lores <si...@gmail.com>.
Are you sure that CMake is installed?
Best,
Silvina
On 28 April 2014 13:05, ascot.moss@gmail.com <as...@gmail.com> wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following
> error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
> O/S Ubuntu: 12.04 (64-bit)
> Java: java version "1.6.0_45"
> protoc —version: libprotoc 2.5.0
>
>
> Command: mvn package -Pdist,native -DskipTests -Dtar -X
> Error message:
>
> [INFO] Total time: 18.096s
> [INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
> [INFO] Final Memory: 59M/1303M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> [ERROR] around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
> hadoop-common: An Ant BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant
> BuildException has occured: exec returned: 1
> around Ant part ...<exec
> dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native"
> executable="cmake" failonerror="true">... @ 4:138 in
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> ... 19 more
> Caused by:
> /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4:
> exec returned: 1
> at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
> at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
> at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
> at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :hadoop-common
>
Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi,
I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
Regards
O/S Ubuntu: 12.04 (64-bit)
Java: java version "1.6.0_45"
protoc —version: libprotoc 2.5.0
Command: mvn package -Pdist,native -DskipTests -Dtar -X
Error message:
[INFO] Total time: 18.096s
[INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
[INFO] Final Memory: 59M/1303M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-common
Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi,
I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
Regards
O/S Ubuntu: 12.04 (64-bit)
Java: java version "1.6.0_45"
protoc —version: libprotoc 2.5.0
Command: mvn package -Pdist,native -DskipTests -Dtar -X
Error message:
[INFO] Total time: 18.096s
[INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
[INFO] Final Memory: 59M/1303M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-common
Re: Mapreduce jobs to download job input from across the internet
Posted by Marcos Luis Ortiz Valmaseda <ma...@gmail.com>.
You can find it here:
http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-with-hadoop/
2013/4/17 Peyman Mohajerian <mo...@gmail.com>
> Apache Flume may help you for this use case. I read an article on
> Cloudera's site about using Flume to pull tweets and same idea may apply
> here.
>
>
> On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com>wrote:
>
>> For a set of jobs to run I need to download about 100GB of data from the
>> internet (~1000 files of varying sizes from ~10 different domains).****
>>
>> ** **
>>
>> Currently I do this in a simple linux script as it’s easy to script FTP,
>> curl, and the like. But it’s a mess to maintain a separate server for that
>> process. I’d rather it run in mapreduce. Just give it a bill of materials
>> and let it go about downloading it, retrying as necessary to deal with iffy
>> network conditions.****
>>
>> ** **
>>
>> I wrote one such job to craw images we need to acquire, and it was the
>> royalist of royal pains. I wonder if there are any good approaches to this
>> kind of data acquisition task in Hadoop. It would certainly be nicer just
>> to schedule a data-acquisition job ahead of the processing jobs in Oozie
>> rather than try to maintain synchronization between the download processes
>> and the jobs.****
>>
>> ** **
>>
>> Ideas?****
>>
>> ** **
>>
>
>
--
Marcos Ortiz Valmaseda,
*Data-Driven Product Manager* at PDVSA
*Blog*: http://dataddict.wordpress.com/
*LinkedIn: *http://www.linkedin.com/in/marcosluis2186
*Twitter*: @marcosluis2186 <http://twitter.com/marcosluis2186>
Re: Mapreduce jobs to download job input from across the internet
Posted by Marcos Luis Ortiz Valmaseda <ma...@gmail.com>.
You can find it here:
http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-with-hadoop/
2013/4/17 Peyman Mohajerian <mo...@gmail.com>
> Apache Flume may help you for this use case. I read an article on
> Cloudera's site about using Flume to pull tweets and same idea may apply
> here.
>
>
> On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com>wrote:
>
>> For a set of jobs to run I need to download about 100GB of data from the
>> internet (~1000 files of varying sizes from ~10 different domains).****
>>
>> ** **
>>
>> Currently I do this in a simple linux script as it’s easy to script FTP,
>> curl, and the like. But it’s a mess to maintain a separate server for that
>> process. I’d rather it run in mapreduce. Just give it a bill of materials
>> and let it go about downloading it, retrying as necessary to deal with iffy
>> network conditions.****
>>
>> ** **
>>
>> I wrote one such job to craw images we need to acquire, and it was the
>> royalist of royal pains. I wonder if there are any good approaches to this
>> kind of data acquisition task in Hadoop. It would certainly be nicer just
>> to schedule a data-acquisition job ahead of the processing jobs in Oozie
>> rather than try to maintain synchronization between the download processes
>> and the jobs.****
>>
>> ** **
>>
>> Ideas?****
>>
>> ** **
>>
>
>
--
Marcos Ortiz Valmaseda,
*Data-Driven Product Manager* at PDVSA
*Blog*: http://dataddict.wordpress.com/
*LinkedIn: *http://www.linkedin.com/in/marcosluis2186
*Twitter*: @marcosluis2186 <http://twitter.com/marcosluis2186>
Re: Mapreduce jobs to download job input from across the internet
Posted by Marcos Luis Ortiz Valmaseda <ma...@gmail.com>.
You can find it here:
http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-with-hadoop/
2013/4/17 Peyman Mohajerian <mo...@gmail.com>
> Apache Flume may help you for this use case. I read an article on
> Cloudera's site about using Flume to pull tweets and same idea may apply
> here.
>
>
> On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com>wrote:
>
>> For a set of jobs to run I need to download about 100GB of data from the
>> internet (~1000 files of varying sizes from ~10 different domains).****
>>
>> ** **
>>
>> Currently I do this in a simple linux script as it’s easy to script FTP,
>> curl, and the like. But it’s a mess to maintain a separate server for that
>> process. I’d rather it run in mapreduce. Just give it a bill of materials
>> and let it go about downloading it, retrying as necessary to deal with iffy
>> network conditions.****
>>
>> ** **
>>
>> I wrote one such job to craw images we need to acquire, and it was the
>> royalist of royal pains. I wonder if there are any good approaches to this
>> kind of data acquisition task in Hadoop. It would certainly be nicer just
>> to schedule a data-acquisition job ahead of the processing jobs in Oozie
>> rather than try to maintain synchronization between the download processes
>> and the jobs.****
>>
>> ** **
>>
>> Ideas?****
>>
>> ** **
>>
>
>
--
Marcos Ortiz Valmaseda,
*Data-Driven Product Manager* at PDVSA
*Blog*: http://dataddict.wordpress.com/
*LinkedIn: *http://www.linkedin.com/in/marcosluis2186
*Twitter*: @marcosluis2186 <http://twitter.com/marcosluis2186>
Re: Mapreduce jobs to download job input from across the internet
Posted by Marcos Luis Ortiz Valmaseda <ma...@gmail.com>.
You can find it here:
http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-with-hadoop/
2013/4/17 Peyman Mohajerian <mo...@gmail.com>
> Apache Flume may help you for this use case. I read an article on
> Cloudera's site about using Flume to pull tweets and same idea may apply
> here.
>
>
> On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com>wrote:
>
>> For a set of jobs to run I need to download about 100GB of data from the
>> internet (~1000 files of varying sizes from ~10 different domains).****
>>
>> ** **
>>
>> Currently I do this in a simple linux script as it’s easy to script FTP,
>> curl, and the like. But it’s a mess to maintain a separate server for that
>> process. I’d rather it run in mapreduce. Just give it a bill of materials
>> and let it go about downloading it, retrying as necessary to deal with iffy
>> network conditions.****
>>
>> ** **
>>
>> I wrote one such job to craw images we need to acquire, and it was the
>> royalist of royal pains. I wonder if there are any good approaches to this
>> kind of data acquisition task in Hadoop. It would certainly be nicer just
>> to schedule a data-acquisition job ahead of the processing jobs in Oozie
>> rather than try to maintain synchronization between the download processes
>> and the jobs.****
>>
>> ** **
>>
>> Ideas?****
>>
>> ** **
>>
>
>
--
Marcos Ortiz Valmaseda,
*Data-Driven Product Manager* at PDVSA
*Blog*: http://dataddict.wordpress.com/
*LinkedIn: *http://www.linkedin.com/in/marcosluis2186
*Twitter*: @marcosluis2186 <http://twitter.com/marcosluis2186>
Re: Mapreduce jobs to download job input from across the internet
Posted by Peyman Mohajerian <mo...@gmail.com>.
Apache Flume may help you for this use case. I read an article on
Cloudera's site about using Flume to pull tweets and same idea may apply
here.
On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com> wrote:
> For a set of jobs to run I need to download about 100GB of data from the
> internet (~1000 files of varying sizes from ~10 different domains).****
>
> ** **
>
> Currently I do this in a simple linux script as it’s easy to script FTP,
> curl, and the like. But it’s a mess to maintain a separate server for that
> process. I’d rather it run in mapreduce. Just give it a bill of materials
> and let it go about downloading it, retrying as necessary to deal with iffy
> network conditions.****
>
> ** **
>
> I wrote one such job to craw images we need to acquire, and it was the
> royalist of royal pains. I wonder if there are any good approaches to this
> kind of data acquisition task in Hadoop. It would certainly be nicer just
> to schedule a data-acquisition job ahead of the processing jobs in Oozie
> rather than try to maintain synchronization between the download processes
> and the jobs.****
>
> ** **
>
> Ideas?****
>
> ** **
>
Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi,
I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
Regards
O/S Ubuntu: 12.04 (64-bit)
Java: java version "1.6.0_45"
protoc —version: libprotoc 2.5.0
Command: mvn package -Pdist,native -DskipTests -Dtar -X
Error message:
[INFO] Total time: 18.096s
[INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
[INFO] Final Memory: 59M/1303M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-common
Re: Mapreduce jobs to download job input from across the internet
Posted by Peyman Mohajerian <mo...@gmail.com>.
Apache Flume may help you for this use case. I read an article on
Cloudera's site about using Flume to pull tweets and same idea may apply
here.
On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com> wrote:
> For a set of jobs to run I need to download about 100GB of data from the
> internet (~1000 files of varying sizes from ~10 different domains).****
>
> ** **
>
> Currently I do this in a simple linux script as it’s easy to script FTP,
> curl, and the like. But it’s a mess to maintain a separate server for that
> process. I’d rather it run in mapreduce. Just give it a bill of materials
> and let it go about downloading it, retrying as necessary to deal with iffy
> network conditions.****
>
> ** **
>
> I wrote one such job to craw images we need to acquire, and it was the
> royalist of royal pains. I wonder if there are any good approaches to this
> kind of data acquisition task in Hadoop. It would certainly be nicer just
> to schedule a data-acquisition job ahead of the processing jobs in Oozie
> rather than try to maintain synchronization between the download processes
> and the jobs.****
>
> ** **
>
> Ideas?****
>
> ** **
>
Install Hadoop 2.4.0 from Source - Compile error
Posted by "ascot.moss@gmail.com" <as...@gmail.com>.
Hi,
I am trying to install Hadoop 2.4.0 from source, I got the following error, please help!!
Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
Regards
O/S Ubuntu: 12.04 (64-bit)
Java: java version "1.6.0_45"
protoc —version: libprotoc 2.5.0
Command: mvn package -Pdist,native -DskipTests -Dtar -X
Error message:
[INFO] Total time: 18.096s
[INFO] Finished at: Mon Apr 28 18:56:00 HKT 2014
[INFO] Final Memory: 59M/1303M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="/usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/native" executable="cmake" failonerror="true">... @ 4:138 in /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: /usr/local/hadoop/hadoop-2.4.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:4: exec returned: 1
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-common
Re: Mapreduce jobs to download job input from across the internet
Posted by Peyman Mohajerian <mo...@gmail.com>.
Apache Flume may help you for this use case. I read an article on
Cloudera's site about using Flume to pull tweets and same idea may apply
here.
On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com> wrote:
> For a set of jobs to run I need to download about 100GB of data from the
> internet (~1000 files of varying sizes from ~10 different domains).****
>
> ** **
>
> Currently I do this in a simple linux script as it’s easy to script FTP,
> curl, and the like. But it’s a mess to maintain a separate server for that
> process. I’d rather it run in mapreduce. Just give it a bill of materials
> and let it go about downloading it, retrying as necessary to deal with iffy
> network conditions.****
>
> ** **
>
> I wrote one such job to craw images we need to acquire, and it was the
> royalist of royal pains. I wonder if there are any good approaches to this
> kind of data acquisition task in Hadoop. It would certainly be nicer just
> to schedule a data-acquisition job ahead of the processing jobs in Oozie
> rather than try to maintain synchronization between the download processes
> and the jobs.****
>
> ** **
>
> Ideas?****
>
> ** **
>
Re: Mapreduce jobs to download job input from across the internet
Posted by Peyman Mohajerian <mo...@gmail.com>.
Apache Flume may help you for this use case. I read an article on
Cloudera's site about using Flume to pull tweets and same idea may apply
here.
On Tue, Apr 16, 2013 at 9:26 PM, David Parks <da...@yahoo.com> wrote:
> For a set of jobs to run I need to download about 100GB of data from the
> internet (~1000 files of varying sizes from ~10 different domains).****
>
> ** **
>
> Currently I do this in a simple linux script as it’s easy to script FTP,
> curl, and the like. But it’s a mess to maintain a separate server for that
> process. I’d rather it run in mapreduce. Just give it a bill of materials
> and let it go about downloading it, retrying as necessary to deal with iffy
> network conditions.****
>
> ** **
>
> I wrote one such job to craw images we need to acquire, and it was the
> royalist of royal pains. I wonder if there are any good approaches to this
> kind of data acquisition task in Hadoop. It would certainly be nicer just
> to schedule a data-acquisition job ahead of the processing jobs in Oozie
> rather than try to maintain synchronization between the download processes
> and the jobs.****
>
> ** **
>
> Ideas?****
>
> ** **
>