You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Mohammad Tariq <do...@gmail.com> on 2012/06/04 11:29:16 UTC
Hbase+Flume
Hello list,
I had started using flume few months ago, but then due to some
change in the requirements I had to work on some other projects. Now I
want to resume my hadoop project that includes flume as well. Earlier
I was able to move data to the Hdfs and now I want to move data to
Hbase.I have downloaded "apache-flume-1.1.0-incubating" but there is o
plugin directory.Is Hbase plugin not included in it??or do I need to
download plugin separately and include it in Flume.Also, when I was
trying to build Flume using maven I was getting following error.Could
someone help me out.Many thanks.
mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ mvn compile -e
+ Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO] Flume
[INFO] Flume Core
[INFO] Flume Master Config Web Application
[INFO] Flume Node Web
[INFO] Flume Distribution Project
[INFO] A log4j appender for Flume
[INFO] Flume Hello World Plugin
[INFO] Flume HBase Plugin
[INFO] ------------------------------------------------------------------------
[INFO] Building Flume
[INFO] task-segment: [compile]
[INFO] ------------------------------------------------------------------------
[INFO] No goals needed for project - skipping
[INFO] ------------------------------------------------------------------------
[INFO] Building Flume Core
[INFO] task-segment: [compile]
[INFO] ------------------------------------------------------------------------
[INFO] [antlr3:antlr {execution: default}]
[INFO] ANTLR: Processing source directory
/home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3
ANTLR Parser Generator Version 3.2 Sep 23, 2009 14:05:07
Grammar /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/conf/FlumeDeploy.g
is up to date - build skipped
Grammar /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/shell/antlr/FlumeShell.g
is up to date - build skipped
[INFO] [antrun:run {execution: generate-version-file}]
[INFO] Executing tasks
main:
[copy] Copying 1 file to
/home/mohammad/Downloads/apache-flume-b01a760/flume-core/target/generated-sources/version/com/cloudera/flume
[INFO] Executed tasks
[INFO] [avro:idl-protocol {execution: default}]
[INFO] [thrift:compile {execution: default}]
[ERROR] thrift failed output:
[ERROR] thrift failed error: /bin/sh: null/bin/thrift: not found
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] thrift did not exit cleanly. Review output for more information.
[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.BuildFailureException: thrift did not exit cleanly.
Review output for more information.
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:715)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.plugin.MojoFailureException: thrift did
not exit cleanly. Review output for more information.
at org.apache.thrift.maven.AbstractThriftMojo.execute(AbstractThriftMojo.java:162)
at org.apache.thrift.maven.ThriftCompileMojo.execute(ThriftCompileMojo.java:22)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
... 17 more
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5 seconds
[INFO] Finished at: Mon Jun 04 14:33:01 IST 2012
[INFO] Final Memory: 27M/242M
[INFO] ------------------------------------------------------------------------
mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ sudo apt-get -t
lenny-backports install automake libboost-test-dev
[sudo] password for mohammad:
Reading package lists... Done
E: The value 'lenny-backports' is invalid for APT::Default-Release as
such a release is not available in the sources
Regards,
Mohammad Tariq
Re: Hbase+Flume
Posted by Mohammad Tariq <do...@gmail.com>.
Hi Alex,
Ya..I was actually trying out the steps you have specified..I''ll
try it out and let you know the progress.
Regards,
Mohammad Tariq
On Tue, Jun 5, 2012 at 1:11 PM, alo alt <wg...@googlemail.com> wrote:
> Hi,
>
> a few seconds ago I wrote a answer, Topic: flume installation steps
> HBase works in trunk 1.2.0
>
> cheers,
> Alex
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
> On Jun 5, 2012, at 9:34 AM, Mohammad Tariq wrote:
>
>> Hi Juhani,
>>
>> Thanks a lot for the response..Actually I tried building from the
>> source, but I was not able to do it properly..Could you share your
>> experience and what have you done to achieve that??I am trying the
>> guidelines given in "readme" file on github, but it did not help.Many
>> thanks.
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Tue, Jun 5, 2012 at 7:28 AM, Juhani Connolly
>> <ju...@cyberagent.co.jp> wrote:
>>> Hi Mohammed,
>>>
>>> If you want the HBase sink you're going to have work off the trunk where we
>>> have a working HBase sink. Of course there *may* be other problems, though
>>> our tests have generally turned out well.
>>>
>>>
>>> On 06/04/2012 06:29 PM, Mohammad Tariq wrote:
>>>>
>>>> Hello list,
>>>>
>>>> I had started using flume few months ago, but then due to some
>>>> change in the requirements I had to work on some other projects. Now I
>>>> want to resume my hadoop project that includes flume as well. Earlier
>>>> I was able to move data to the Hdfs and now I want to move data to
>>>> Hbase.I have downloaded "apache-flume-1.1.0-incubating" but there is o
>>>> plugin directory.Is Hbase plugin not included in it??or do I need to
>>>> download plugin separately and include it in Flume.Also, when I was
>>>> trying to build Flume using maven I was getting following error.Could
>>>> someone help me out.Many thanks.
>>>>
>>>> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ mvn compile -e
>>>> + Error stacktraces are turned on.
>>>> [INFO] Scanning for projects...
>>>> [INFO] Reactor build order:
>>>> [INFO] Flume
>>>> [INFO] Flume Core
>>>> [INFO] Flume Master Config Web Application
>>>> [INFO] Flume Node Web
>>>> [INFO] Flume Distribution Project
>>>> [INFO] A log4j appender for Flume
>>>> [INFO] Flume Hello World Plugin
>>>> [INFO] Flume HBase Plugin
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [INFO] Building Flume
>>>> [INFO] task-segment: [compile]
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [INFO] No goals needed for project - skipping
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [INFO] Building Flume Core
>>>> [INFO] task-segment: [compile]
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [INFO] [antlr3:antlr {execution: default}]
>>>> [INFO] ANTLR: Processing source directory
>>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3
>>>> ANTLR Parser Generator Version 3.2 Sep 23, 2009 14:05:07
>>>> Grammar
>>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/conf/FlumeDeploy.g
>>>> is up to date - build skipped
>>>> Grammar
>>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/shell/antlr/FlumeShell.g
>>>> is up to date - build skipped
>>>> [INFO] [antrun:run {execution: generate-version-file}]
>>>> [INFO] Executing tasks
>>>>
>>>> main:
>>>> [copy] Copying 1 file to
>>>>
>>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/target/generated-sources/version/com/cloudera/flume
>>>> [INFO] Executed tasks
>>>> [INFO] [avro:idl-protocol {execution: default}]
>>>> [INFO] [thrift:compile {execution: default}]
>>>> [ERROR] thrift failed output:
>>>> [ERROR] thrift failed error: /bin/sh: null/bin/thrift: not found
>>>>
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [ERROR] BUILD FAILURE
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [INFO] thrift did not exit cleanly. Review output for more information.
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [INFO] Trace
>>>> org.apache.maven.BuildFailureException: thrift did not exit cleanly.
>>>> Review output for more information.
>>>> at
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:715)
>>>> at
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
>>>> at
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
>>>> at
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
>>>> at
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
>>>> at
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
>>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
>>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
>>>> at
>>>> org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
>>>> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
>>>> at
>>>> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>>>> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>>>> Caused by: org.apache.maven.plugin.MojoFailureException: thrift did
>>>> not exit cleanly. Review output for more information.
>>>> at
>>>> org.apache.thrift.maven.AbstractThriftMojo.execute(AbstractThriftMojo.java:162)
>>>> at
>>>> org.apache.thrift.maven.ThriftCompileMojo.execute(ThriftCompileMojo.java:22)
>>>> at
>>>> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
>>>> at
>>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
>>>> ... 17 more
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> [INFO] Total time: 5 seconds
>>>> [INFO] Finished at: Mon Jun 04 14:33:01 IST 2012
>>>> [INFO] Final Memory: 27M/242M
>>>> [INFO]
>>>> ------------------------------------------------------------------------
>>>> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ sudo apt-get -t
>>>> lenny-backports install automake libboost-test-dev
>>>> [sudo] password for mohammad:
>>>> Reading package lists... Done
>>>> E: The value 'lenny-backports' is invalid for APT::Default-Release as
>>>> such a release is not available in the sources
>>>>
>>>>
>>>> Regards,
>>>> Mohammad Tariq
>>>>
>>>
>
Re: Hbase+Flume
Posted by alo alt <wg...@googlemail.com>.
Hi,
a few seconds ago I wrote a answer, Topic: flume installation steps
HBase works in trunk 1.2.0
cheers,
Alex
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
On Jun 5, 2012, at 9:34 AM, Mohammad Tariq wrote:
> Hi Juhani,
>
> Thanks a lot for the response..Actually I tried building from the
> source, but I was not able to do it properly..Could you share your
> experience and what have you done to achieve that??I am trying the
> guidelines given in "readme" file on github, but it did not help.Many
> thanks.
>
> Regards,
> Mohammad Tariq
>
>
> On Tue, Jun 5, 2012 at 7:28 AM, Juhani Connolly
> <ju...@cyberagent.co.jp> wrote:
>> Hi Mohammed,
>>
>> If you want the HBase sink you're going to have work off the trunk where we
>> have a working HBase sink. Of course there *may* be other problems, though
>> our tests have generally turned out well.
>>
>>
>> On 06/04/2012 06:29 PM, Mohammad Tariq wrote:
>>>
>>> Hello list,
>>>
>>> I had started using flume few months ago, but then due to some
>>> change in the requirements I had to work on some other projects. Now I
>>> want to resume my hadoop project that includes flume as well. Earlier
>>> I was able to move data to the Hdfs and now I want to move data to
>>> Hbase.I have downloaded "apache-flume-1.1.0-incubating" but there is o
>>> plugin directory.Is Hbase plugin not included in it??or do I need to
>>> download plugin separately and include it in Flume.Also, when I was
>>> trying to build Flume using maven I was getting following error.Could
>>> someone help me out.Many thanks.
>>>
>>> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ mvn compile -e
>>> + Error stacktraces are turned on.
>>> [INFO] Scanning for projects...
>>> [INFO] Reactor build order:
>>> [INFO] Flume
>>> [INFO] Flume Core
>>> [INFO] Flume Master Config Web Application
>>> [INFO] Flume Node Web
>>> [INFO] Flume Distribution Project
>>> [INFO] A log4j appender for Flume
>>> [INFO] Flume Hello World Plugin
>>> [INFO] Flume HBase Plugin
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Building Flume
>>> [INFO] task-segment: [compile]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] No goals needed for project - skipping
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Building Flume Core
>>> [INFO] task-segment: [compile]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] [antlr3:antlr {execution: default}]
>>> [INFO] ANTLR: Processing source directory
>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3
>>> ANTLR Parser Generator Version 3.2 Sep 23, 2009 14:05:07
>>> Grammar
>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/conf/FlumeDeploy.g
>>> is up to date - build skipped
>>> Grammar
>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/shell/antlr/FlumeShell.g
>>> is up to date - build skipped
>>> [INFO] [antrun:run {execution: generate-version-file}]
>>> [INFO] Executing tasks
>>>
>>> main:
>>> [copy] Copying 1 file to
>>>
>>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/target/generated-sources/version/com/cloudera/flume
>>> [INFO] Executed tasks
>>> [INFO] [avro:idl-protocol {execution: default}]
>>> [INFO] [thrift:compile {execution: default}]
>>> [ERROR] thrift failed output:
>>> [ERROR] thrift failed error: /bin/sh: null/bin/thrift: not found
>>>
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] BUILD FAILURE
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] thrift did not exit cleanly. Review output for more information.
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Trace
>>> org.apache.maven.BuildFailureException: thrift did not exit cleanly.
>>> Review output for more information.
>>> at
>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:715)
>>> at
>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
>>> at
>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
>>> at
>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
>>> at
>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
>>> at
>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
>>> at
>>> org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
>>> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
>>> at
>>> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>>> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>>> Caused by: org.apache.maven.plugin.MojoFailureException: thrift did
>>> not exit cleanly. Review output for more information.
>>> at
>>> org.apache.thrift.maven.AbstractThriftMojo.execute(AbstractThriftMojo.java:162)
>>> at
>>> org.apache.thrift.maven.ThriftCompileMojo.execute(ThriftCompileMojo.java:22)
>>> at
>>> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
>>> at
>>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
>>> ... 17 more
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Total time: 5 seconds
>>> [INFO] Finished at: Mon Jun 04 14:33:01 IST 2012
>>> [INFO] Final Memory: 27M/242M
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ sudo apt-get -t
>>> lenny-backports install automake libboost-test-dev
>>> [sudo] password for mohammad:
>>> Reading package lists... Done
>>> E: The value 'lenny-backports' is invalid for APT::Default-Release as
>>> such a release is not available in the sources
>>>
>>>
>>> Regards,
>>> Mohammad Tariq
>>>
>>
Re: Hbase+Flume
Posted by Mohammad Tariq <do...@gmail.com>.
Hi Juhani,
Thanks a lot for the response..Actually I tried building from the
source, but I was not able to do it properly..Could you share your
experience and what have you done to achieve that??I am trying the
guidelines given in "readme" file on github, but it did not help.Many
thanks.
Regards,
Mohammad Tariq
On Tue, Jun 5, 2012 at 7:28 AM, Juhani Connolly
<ju...@cyberagent.co.jp> wrote:
> Hi Mohammed,
>
> If you want the HBase sink you're going to have work off the trunk where we
> have a working HBase sink. Of course there *may* be other problems, though
> our tests have generally turned out well.
>
>
> On 06/04/2012 06:29 PM, Mohammad Tariq wrote:
>>
>> Hello list,
>>
>> I had started using flume few months ago, but then due to some
>> change in the requirements I had to work on some other projects. Now I
>> want to resume my hadoop project that includes flume as well. Earlier
>> I was able to move data to the Hdfs and now I want to move data to
>> Hbase.I have downloaded "apache-flume-1.1.0-incubating" but there is o
>> plugin directory.Is Hbase plugin not included in it??or do I need to
>> download plugin separately and include it in Flume.Also, when I was
>> trying to build Flume using maven I was getting following error.Could
>> someone help me out.Many thanks.
>>
>> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ mvn compile -e
>> + Error stacktraces are turned on.
>> [INFO] Scanning for projects...
>> [INFO] Reactor build order:
>> [INFO] Flume
>> [INFO] Flume Core
>> [INFO] Flume Master Config Web Application
>> [INFO] Flume Node Web
>> [INFO] Flume Distribution Project
>> [INFO] A log4j appender for Flume
>> [INFO] Flume Hello World Plugin
>> [INFO] Flume HBase Plugin
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Building Flume
>> [INFO] task-segment: [compile]
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] No goals needed for project - skipping
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Building Flume Core
>> [INFO] task-segment: [compile]
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] [antlr3:antlr {execution: default}]
>> [INFO] ANTLR: Processing source directory
>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3
>> ANTLR Parser Generator Version 3.2 Sep 23, 2009 14:05:07
>> Grammar
>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/conf/FlumeDeploy.g
>> is up to date - build skipped
>> Grammar
>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/shell/antlr/FlumeShell.g
>> is up to date - build skipped
>> [INFO] [antrun:run {execution: generate-version-file}]
>> [INFO] Executing tasks
>>
>> main:
>> [copy] Copying 1 file to
>>
>> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/target/generated-sources/version/com/cloudera/flume
>> [INFO] Executed tasks
>> [INFO] [avro:idl-protocol {execution: default}]
>> [INFO] [thrift:compile {execution: default}]
>> [ERROR] thrift failed output:
>> [ERROR] thrift failed error: /bin/sh: null/bin/thrift: not found
>>
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] thrift did not exit cleanly. Review output for more information.
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Trace
>> org.apache.maven.BuildFailureException: thrift did not exit cleanly.
>> Review output for more information.
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:715)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
>> at
>> org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
>> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
>> at
>> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>> Caused by: org.apache.maven.plugin.MojoFailureException: thrift did
>> not exit cleanly. Review output for more information.
>> at
>> org.apache.thrift.maven.AbstractThriftMojo.execute(AbstractThriftMojo.java:162)
>> at
>> org.apache.thrift.maven.ThriftCompileMojo.execute(ThriftCompileMojo.java:22)
>> at
>> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
>> ... 17 more
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 5 seconds
>> [INFO] Finished at: Mon Jun 04 14:33:01 IST 2012
>> [INFO] Final Memory: 27M/242M
>> [INFO]
>> ------------------------------------------------------------------------
>> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ sudo apt-get -t
>> lenny-backports install automake libboost-test-dev
>> [sudo] password for mohammad:
>> Reading package lists... Done
>> E: The value 'lenny-backports' is invalid for APT::Default-Release as
>> such a release is not available in the sources
>>
>>
>> Regards,
>> Mohammad Tariq
>>
>
Re: Hbase+Flume
Posted by Juhani Connolly <ju...@cyberagent.co.jp>.
Hi Mohammed,
If you want the HBase sink you're going to have work off the trunk
where we have a working HBase sink. Of course there *may* be other
problems, though our tests have generally turned out well.
On 06/04/2012 06:29 PM, Mohammad Tariq wrote:
> Hello list,
>
> I had started using flume few months ago, but then due to some
> change in the requirements I had to work on some other projects. Now I
> want to resume my hadoop project that includes flume as well. Earlier
> I was able to move data to the Hdfs and now I want to move data to
> Hbase.I have downloaded "apache-flume-1.1.0-incubating" but there is o
> plugin directory.Is Hbase plugin not included in it??or do I need to
> download plugin separately and include it in Flume.Also, when I was
> trying to build Flume using maven I was getting following error.Could
> someone help me out.Many thanks.
>
> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ mvn compile -e
> + Error stacktraces are turned on.
> [INFO] Scanning for projects...
> [INFO] Reactor build order:
> [INFO] Flume
> [INFO] Flume Core
> [INFO] Flume Master Config Web Application
> [INFO] Flume Node Web
> [INFO] Flume Distribution Project
> [INFO] A log4j appender for Flume
> [INFO] Flume Hello World Plugin
> [INFO] Flume HBase Plugin
> [INFO] ------------------------------------------------------------------------
> [INFO] Building Flume
> [INFO] task-segment: [compile]
> [INFO] ------------------------------------------------------------------------
> [INFO] No goals needed for project - skipping
> [INFO] ------------------------------------------------------------------------
> [INFO] Building Flume Core
> [INFO] task-segment: [compile]
> [INFO] ------------------------------------------------------------------------
> [INFO] [antlr3:antlr {execution: default}]
> [INFO] ANTLR: Processing source directory
> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3
> ANTLR Parser Generator Version 3.2 Sep 23, 2009 14:05:07
> Grammar /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/conf/FlumeDeploy.g
> is up to date - build skipped
> Grammar /home/mohammad/Downloads/apache-flume-b01a760/flume-core/src/main/antlr3/com/cloudera/flume/shell/antlr/FlumeShell.g
> is up to date - build skipped
> [INFO] [antrun:run {execution: generate-version-file}]
> [INFO] Executing tasks
>
> main:
> [copy] Copying 1 file to
> /home/mohammad/Downloads/apache-flume-b01a760/flume-core/target/generated-sources/version/com/cloudera/flume
> [INFO] Executed tasks
> [INFO] [avro:idl-protocol {execution: default}]
> [INFO] [thrift:compile {execution: default}]
> [ERROR] thrift failed output:
> [ERROR] thrift failed error: /bin/sh: null/bin/thrift: not found
>
> [INFO] ------------------------------------------------------------------------
> [ERROR] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] thrift did not exit cleanly. Review output for more information.
> [INFO] ------------------------------------------------------------------------
> [INFO] Trace
> org.apache.maven.BuildFailureException: thrift did not exit cleanly.
> Review output for more information.
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:715)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.apache.maven.plugin.MojoFailureException: thrift did
> not exit cleanly. Review output for more information.
> at org.apache.thrift.maven.AbstractThriftMojo.execute(AbstractThriftMojo.java:162)
> at org.apache.thrift.maven.ThriftCompileMojo.execute(ThriftCompileMojo.java:22)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> ... 17 more
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 5 seconds
> [INFO] Finished at: Mon Jun 04 14:33:01 IST 2012
> [INFO] Final Memory: 27M/242M
> [INFO] ------------------------------------------------------------------------
> mohammad@ubuntu:~/Downloads/apache-flume-b01a760$ sudo apt-get -t
> lenny-backports install automake libboost-test-dev
> [sudo] password for mohammad:
> Reading package lists... Done
> E: The value 'lenny-backports' is invalid for APT::Default-Release as
> such a release is not available in the sources
>
>
> Regards,
> Mohammad Tariq
>