You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by smilevasu6 <sr...@gmail.com> on 2014/11/17 07:50:58 UTC
Camel-Hive
Hi,
I am trying for camel-hive connection to fetch data from Hive table.
Please any one help for the same.
I am not able to fetch the information.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel-Hive
Posted by Charles Moulliard <ch...@gmail.com>.
This error is reports is reported by Hive framework. so please check your
stacktrace and have a look to Hive code.
On Wed, Nov 19, 2014 at 9:45 AM, smilevasu6 <sr...@gmail.com>
wrote:
> Hi,
>
> Error is like Invalid path '/home/cloudera/Desktop/Sri'': No files matching
> path file:/home/cloudera/Desktop/Sri
>
> i want read the from Hive table and copy into another Hive table using
> camel
>
> Reading part is done, i am able to read the data from Hive table using
> camel.
>
>
> Please advice
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189p5759315.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Charles Moulliard
Apache Committer / Architect @RedHat
Twitter : @cmoulliard | Blog : http://cmoulliard.github.io
Re: Camel-Hive
Posted by smilevasu6 <sr...@gmail.com>.
Hi,
Error is like Invalid path '/home/cloudera/Desktop/Sri'': No files matching
path file:/home/cloudera/Desktop/Sri
i want read the from Hive table and copy into another Hive table using camel
Reading part is done, i am able to read the data from Hive table using
camel.
Please advice
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189p5759315.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel-Hive
Posted by Charles Moulliard <ch...@gmail.com>.
Hi,
- As there is no camel hive component, the issue cannot be assigned to
something which does not exist
- As you are using the camel-sql (= Spring JDBC framework), can you please
describe the problem, error and how to reproduce
Regards,
On Wed, Nov 19, 2014 at 5:39 AM, smilevasu6 <sr...@gmail.com>
wrote:
> Hi,
>
> Again there is problem with camel hive.
>
> As i am able to connect to hive and able fetch the data from hive table.
>
> Now i want to load the data in another hive table.
>
> when i try to do the same some times i am getting invalid path and some
> times there is no errors but data is not moving to target hive table.
>
> Please help me
>
> Here is my router,
>
> from("timer://foo?period=60000").
>
> setBody(constant("select * from default.emp where
> salary > 5000"))
> .to("jdbc:fromdatasource?resetAutoCommit=false")
> .convertBodyTo(String.class)
> .to("file:/home/cloudera/Desktop/Sri")
> .end();
>
> from("file:/home/cloudera/Desktop/Sri").
> setBody(constant("LOAD DATA LOCAL INPATH
> '/home/cloudera/Desktop/Sri'
> OVERWRITE INTO TABLE default.emp_h"))
> .to("jdbc:todatasource?resetAutoCommit=false")
> .convertBodyTo(String.class)
> .end();
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189p5759284.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Charles Moulliard
Apache Committer / Architect @RedHat
Twitter : @cmoulliard | Blog : http://cmoulliard.github.io
Re: Camel-Hive
Posted by smilevasu6 <sr...@gmail.com>.
Hi,
Again there is problem with camel hive.
As i am able to connect to hive and able fetch the data from hive table.
Now i want to load the data in another hive table.
when i try to do the same some times i am getting invalid path and some
times there is no errors but data is not moving to target hive table.
Please help me
Here is my router,
from("timer://foo?period=60000").
setBody(constant("select * from default.emp where salary > 5000"))
.to("jdbc:fromdatasource?resetAutoCommit=false")
.convertBodyTo(String.class)
.to("file:/home/cloudera/Desktop/Sri")
.end();
from("file:/home/cloudera/Desktop/Sri").
setBody(constant("LOAD DATA LOCAL INPATH '/home/cloudera/Desktop/Sri'
OVERWRITE INTO TABLE default.emp_h"))
.to("jdbc:todatasource?resetAutoCommit=false")
.convertBodyTo(String.class)
.end();
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189p5759284.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel-Hive
Posted by smilevasu6 <sr...@gmail.com>.
Thank you so much Claus Ibsen, its working.
Now we can connect to hive using camel
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189p5759242.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel-Hive
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
>From the stacktrace it looks like its the jdbc component that fails.
Try set the option resetAutoCommit=false
http://camel.apache.org/jdbc
On Tue, Nov 18, 2014 at 8:25 AM, smilevasu6 <sr...@gmail.com> wrote:
> Hi Charles ,
>
> Thanks for such a quick response.
>
> I am able to connect to Hive from camel. I can see the Hive table data in my
> eclipse console.
>
> But the problem is, that data i want in a file.
>
> 22:58:17.081 [Camel (camel-1) thread #1 - timer://foo] DEBUG
> org.apache.camel.processor.Pipeline - Message exchange has failed: so
> breaking out of pipeline for exchange: Exchange[Message: [{bd_use_id=73.0,
> bd_use_number=CUSE0073}, {bd_use_id=2.0, bd_use_number=CASU2},
> {bd_use_id=379.0, bd_use_number=CUSE0379}, {bd_use_id=380.0,
> bd_use_number=CUSE0380}, {bd_use_id=381.0, bd_use_number=CUSE0381},
> {bd_use_id=263.0, bd_use_number=CUSE0263}, {bd_use_id=264.0,
> bd_use_number=CUSE0264}, {bd_use_id=314.0, bd_use_number=CUSE0314},
> {bd_use_id=326.0, bd_use_number=CUSE0326}, {bd_use_id=341.0,
> bd_use_number=CUSE0341}, {bd_use_id=342.0, bd_use_number=CUSE0342},
> {bd_use_id=348.0, bd_use_number=CUSE0348}, {bd_use_id=17.0,
> bd_use_number=CUSE0017}, {bd_use_id=354.0, bd_use_number=CUSE0354},
> {bd_use_id=357.0, bd_use_number=CUSE0357}, {bd_use_id=360.0,
> bd_use_number=CUSE0360}, {bd_use_id=361.0, bd_use_number=CUSE0361},
> {bd_use_id=362.0, bd_use_number=CUSE0362}, {bd_use_id=365.0,
> bd_use_number=CUSE0365}, {bd_use_id=372.0, bd_use_number=CUSE0372},
> {bd_use_id=382.0, bd_use_number=CUSE0382}, {bd_use_id=383.0,
> bd_use_number=CUSE0383}, {bd_use_id=384.0, bd_use_number=CUSE0384},
> {bd_use_id=18.0, ... [Body clipped after 1000 chars, total length is 17124]]
> Exception: java.sql.SQLException: Method not supported
> 22:58:17.094 [Camel (camel-1) thread #1 - timer://foo] WARN
> o.a.c.component.timer.TimerConsumer - Error processing exchange.
> Exchange[Message: [{bd_use_id=73.0, bd_use_number=CUSE0073}, {bd_use_id=2.0,
> bd_use_number=CASU2}, {bd_use_id=379.0, bd_use_number=CUSE0379},
> {bd_use_id=380.0, bd_use_number=CUSE0380}, {bd_use_id=381.0,
> bd_use_number=CUSE0381}, {bd_use_id=263.0, bd_use_number=CUSE0263},
> {bd_use_id=264.0, bd_use_number=CUSE0264}, {bd_use_id=314.0,
> bd_use_number=CUSE0314}, {bd_use_id=326.0, bd_use_number=CUSE0326},
> {bd_use_id=341.0, bd_use_number=CUSE0341}, {bd_use_id=342.0,
> bd_use_number=CUSE0342}, {bd_use_id=348.0, bd_use_number=CUSE0348},
> {bd_use_id=17.0, bd_use_number=CUSE0017}, {bd_use_id=354.0,
> bd_use_number=CUSE0354}, {bd_use_id=357.0, bd_use_number=CUSE0357},
> {bd_use_id=360.0, bd_use_number=CUSE0360}, {bd_use_id=361.0,
> bd_use_number=CUSE0361}, {bd_use_id=362.0, bd_use_number=CUSE0362},
> {bd_use_id=365.0, bd_use_number=CUSE0365}, {bd_use_id=372.0,
> bd_use_number=CUSE0372}, {bd_use_id=382.0, bd_use_number=CUSE0382},
> {bd_use_id=383.0, bd_use_number=CUSE0383}, {bd_use_id=384.0,
> bd_use_number=CUSE0384}, {bd_use_id=18.0, ... [Body clipped after 1000
> chars, total length is 17124]]. Caused by: [java.sql.SQLException - Method
> not supported]
> java.sql.SQLException: Method not supported
> at org.apache.hive.jdbc.HiveConnection.commit(HiveConnection.java:508)
> at
> org.apache.camel.component.jdbc.JdbcProducer.processingSqlBySettingAutoCommit(JdbcProducer.java:85)
> at
> org.apache.camel.component.jdbc.JdbcProducer.process(JdbcProducer.java:66)
> at
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:113)
> at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
> at
> org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:166)
> at
> org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:74)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
>
>
> I am getting hive data as bd_use_id=73.0, bd_use_number=CUSE0073, but when i
> try to copy the data in file, i am getting the above error, Method not
> supported.
>
> My Router is
>
> rom("timer://foo?period=60000").
> setBody(constant("select bd_use_id,bd_use_number from bd_users"))
> .to("jdbc:fromdatasource")
> .convertBodyTo(String.class)
> .to("file:/home/cloudera/Desktop/Sri")
> .end();
>
> Please help me. What to do?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189p5759233.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
Re: Camel-Hive
Posted by smilevasu6 <sr...@gmail.com>.
Hi Charles ,
Thanks for such a quick response.
I am able to connect to Hive from camel. I can see the Hive table data in my
eclipse console.
But the problem is, that data i want in a file.
22:58:17.081 [Camel (camel-1) thread #1 - timer://foo] DEBUG
org.apache.camel.processor.Pipeline - Message exchange has failed: so
breaking out of pipeline for exchange: Exchange[Message: [{bd_use_id=73.0,
bd_use_number=CUSE0073}, {bd_use_id=2.0, bd_use_number=CASU2},
{bd_use_id=379.0, bd_use_number=CUSE0379}, {bd_use_id=380.0,
bd_use_number=CUSE0380}, {bd_use_id=381.0, bd_use_number=CUSE0381},
{bd_use_id=263.0, bd_use_number=CUSE0263}, {bd_use_id=264.0,
bd_use_number=CUSE0264}, {bd_use_id=314.0, bd_use_number=CUSE0314},
{bd_use_id=326.0, bd_use_number=CUSE0326}, {bd_use_id=341.0,
bd_use_number=CUSE0341}, {bd_use_id=342.0, bd_use_number=CUSE0342},
{bd_use_id=348.0, bd_use_number=CUSE0348}, {bd_use_id=17.0,
bd_use_number=CUSE0017}, {bd_use_id=354.0, bd_use_number=CUSE0354},
{bd_use_id=357.0, bd_use_number=CUSE0357}, {bd_use_id=360.0,
bd_use_number=CUSE0360}, {bd_use_id=361.0, bd_use_number=CUSE0361},
{bd_use_id=362.0, bd_use_number=CUSE0362}, {bd_use_id=365.0,
bd_use_number=CUSE0365}, {bd_use_id=372.0, bd_use_number=CUSE0372},
{bd_use_id=382.0, bd_use_number=CUSE0382}, {bd_use_id=383.0,
bd_use_number=CUSE0383}, {bd_use_id=384.0, bd_use_number=CUSE0384},
{bd_use_id=18.0, ... [Body clipped after 1000 chars, total length is 17124]]
Exception: java.sql.SQLException: Method not supported
22:58:17.094 [Camel (camel-1) thread #1 - timer://foo] WARN
o.a.c.component.timer.TimerConsumer - Error processing exchange.
Exchange[Message: [{bd_use_id=73.0, bd_use_number=CUSE0073}, {bd_use_id=2.0,
bd_use_number=CASU2}, {bd_use_id=379.0, bd_use_number=CUSE0379},
{bd_use_id=380.0, bd_use_number=CUSE0380}, {bd_use_id=381.0,
bd_use_number=CUSE0381}, {bd_use_id=263.0, bd_use_number=CUSE0263},
{bd_use_id=264.0, bd_use_number=CUSE0264}, {bd_use_id=314.0,
bd_use_number=CUSE0314}, {bd_use_id=326.0, bd_use_number=CUSE0326},
{bd_use_id=341.0, bd_use_number=CUSE0341}, {bd_use_id=342.0,
bd_use_number=CUSE0342}, {bd_use_id=348.0, bd_use_number=CUSE0348},
{bd_use_id=17.0, bd_use_number=CUSE0017}, {bd_use_id=354.0,
bd_use_number=CUSE0354}, {bd_use_id=357.0, bd_use_number=CUSE0357},
{bd_use_id=360.0, bd_use_number=CUSE0360}, {bd_use_id=361.0,
bd_use_number=CUSE0361}, {bd_use_id=362.0, bd_use_number=CUSE0362},
{bd_use_id=365.0, bd_use_number=CUSE0365}, {bd_use_id=372.0,
bd_use_number=CUSE0372}, {bd_use_id=382.0, bd_use_number=CUSE0382},
{bd_use_id=383.0, bd_use_number=CUSE0383}, {bd_use_id=384.0,
bd_use_number=CUSE0384}, {bd_use_id=18.0, ... [Body clipped after 1000
chars, total length is 17124]]. Caused by: [java.sql.SQLException - Method
not supported]
java.sql.SQLException: Method not supported
at org.apache.hive.jdbc.HiveConnection.commit(HiveConnection.java:508)
at
org.apache.camel.component.jdbc.JdbcProducer.processingSqlBySettingAutoCommit(JdbcProducer.java:85)
at
org.apache.camel.component.jdbc.JdbcProducer.process(JdbcProducer.java:66)
at
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:113)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:166)
at
org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:74)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
I am getting hive data as bd_use_id=73.0, bd_use_number=CUSE0073, but when i
try to copy the data in file, i am getting the above error, Method not
supported.
My Router is
rom("timer://foo?period=60000").
setBody(constant("select bd_use_id,bd_use_number from bd_users"))
.to("jdbc:fromdatasource")
.convertBodyTo(String.class)
.to("file:/home/cloudera/Desktop/Sri")
.end();
Please help me. What to do?
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189p5759233.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel-Hive
Posted by Charles Moulliard <ch...@gmail.com>.
Hi Srinivas,
As there is no camel-hive component available for the moment, you should
encapsulate your HiveQL queries in an Bean Component (
http://camel.apache.org/bean.html).
Regards,
On Mon, Nov 17, 2014 at 7:50 AM, smilevasu6 <sr...@gmail.com>
wrote:
> Hi,
>
> I am trying for camel-hive connection to fetch data from Hive table.
>
> Please any one help for the same.
>
> I am not able to fetch the information.
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-Hive-tp5759189.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Charles Moulliard
Apache Committer / Architect @RedHat
Twitter : @cmoulliard | Blog : http://cmoulliard.github.io