You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@sqoop.apache.org by corbacho anthony <ac...@nflabs.com> on 2013/06/19 08:25:51 UTC

Sqoop and OSGI

Hello,

I am trying to use Sqoop in an osgi pluging but it wont work and I dont
know why.
I keep getting the following error:
"2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 | BaseSqoopTool  | ?  ?
|  -  -  | Got error creating database manager: java.io.IOException: No
manager for connect string: jdbc:mysql://...."

I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my mysql
driver loader (i can use in the code).

the part who throw an exception is:

String[] str = {
"import",
"--driver", "com.mysql.jdbc.Driver",
"--connect", "jdbc:mysql://___MY_DB___",
"--hadoop-home", __HOME__,
"--query", "Select [.....] AND $CONDITIONS",
"--split-by", "id",
"--target-dir", "hdfs://__SERVER__/__FOLDER__",
"--username", "foo",
"--password", "barou"};

return Sqoop.runTool(str);

Thanks,
Anthony

Re: Sqoop and OSGI

Posted by corbacho anthony <ac...@nflabs.com>.

Abe,

I see your point.
I tried with this config, but I get still the same error.
[com.cloudera.sqoop.manager.ConnMananger (not found)].

Anthony


On Fri, Jun 21, 2013 at 2:41 PM, Abraham Elmahrek <ab...@cloudera.com> wrote:

> Anthony,
>
> You've set that configuration incorrectly. It should be conf.set(
> "sqoop.connection.factories", "com.cloudera.sqoop.manager.ConnMananger");.
> Sqoop instantiates a Configuration object and adds resources to it.
>
> -Abe
>
>
> On Thu, Jun 20, 2013 at 6:44 PM, corbacho anthony <ac...@nflabs.com>wrote:
>
>> Hello Abe.
>>
>> The think is, I didn`t install sqoop, I use it via pom on my java code.
>> I added on my code this:
>>
>> Configuration conf = new HdfsConfiguration();
>> conf.set("sqoop.connection.factories",
>> "com.cloudera.sqoop.apache.ConnMananger");
>> [....]
>>
>> return Sqoop.runTool(str, conf);
>> But still the same error.
>>
>>
>> On Thu, Jun 20, 2013 at 12:50 PM, Abraham Elmahrek <ab...@cloudera.com>wrote:
>>
>>> Anthony,
>>>
>>> Weird, that should be com.cloudera.sqoop.manager.ConnMananger. I
>>> suspect there's a problem with your sqoop-site.xml (if its even being
>>> parsed at all) or classpath. The sqoop startup script sets
>>> SQOOP_HOME_DIR, SQOOP_CONF_DIR, HADOOP_CLASSPATH, and a few other
>>> environment variables. I suspect there's something required there.
>>>
>>> -Abe
>>>
>>>
>>> On Wed, Jun 19, 2013 at 6:53 PM, corbacho anthony <ac...@nflabs.com>wrote:
>>>
>>>> I tried an same issues.
>>>> The problem is Sqoop cannot
>>>> find com.cloudera.sqoop.apache.ConnMananger...
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 10:33 AM, corbacho anthony <
>>>> acorbacho@nflabs.com> wrote:
>>>>
>>>>> I used
>>>>> https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/sqoop/sqoop/
>>>>>
>>>>> I will try with sqoop 1.4.3.
>>>>> Thank you Jarec.
>>>>>
>>>>> Anthony
>>>>>
>>>>>
>>>>> On Thu, Jun 20, 2013 at 10:28 AM, Jarek Jarcec Cecho <
>>>>> jarcec@apache.org> wrote:
>>>>>
>>>>>> Hi Anthony,
>>>>>> what maven repository are you using?
>>>>>>
>>>>>> Cloudera maven repository [1] seems to be containing all released
>>>>>> versions (instructions how to use this repository are in installation guide
>>>>>> [3]). Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts available
>>>>>> in main maven repository [2].
>>>>>>
>>>>>> Jarcec
>>>>>>
>>>>>> Links:
>>>>>> 1:
>>>>>> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
>>>>>> 2: http://search.maven.org/#browse%7C585117616
>>>>>> 3:
>>>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html
>>>>>>
>>>>>> On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
>>>>>> > hello Jarec and Abe.
>>>>>> >
>>>>>> > Abe: the debugging mode show nothing :(
>>>>>> > Jarec: In cloudera repo I can only find Sqoop 1.4.0.
>>>>>> >
>>>>>> > There is the full stack (in case):
>>>>>> > 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory |
>>>>>> Could
>>>>>> > not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not
>>>>>> found)
>>>>>> > 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool
>>>>>> | Got
>>>>>> > error creating database manager: java.io.IOException: No manager for
>>>>>> > connect string: jdbc:mysql://xxx:0000/x
>>>>>> > at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
>>>>>> > at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>>>>>> > at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>>>>>> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
>>>>>> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>>>>> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>>>>> > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
>>>>>> > at
>>>>>> com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
>>>>>> > at
>>>>>> com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
>>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> > at
>>>>>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
>>>>>> > at
>>>>>> org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
>>>>>> > at
>>>>>> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
>>>>>> > at com.sun.proxy.$Proxy76.test(Unknown Source)
>>>>>> > at
>>>>>> >
>>>>>> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
>>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
>>>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
>>>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
>>>>>> > at
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>> > at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
>>>>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>>>>>> > at
>>>>>> >
>>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
>>>>>> > at
>>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
>>>>>> > at
>>>>>> >
>>>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
>>>>>> > at
>>>>>> >
>>>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
>>>>>> > at
>>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>>>> > at
>>>>>> >
>>>>>> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>>>>>> > at org.eclipse.jetty.server.Server.handle(Server.java:346)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>>>>> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
>>>>>> > at
>>>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
>>>>>> > at
>>>>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>>>>>> > at
>>>>>> >
>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>>>>>> > at java.lang.Thread.run(Thread.java:680)
>>>>>> >
>>>>>> > Thanks
>>>>>> > Anthony
>>>>>> >
>>>>>> >
>>>>>> > On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > > Anthony,
>>>>>> > >
>>>>>> > > You could also try enabling debug logging to see what managers
>>>>>> are being
>>>>>> > > loaded (if any). This can normally be done by adding "--verbose"
>>>>>> to your
>>>>>> > > list of arguments. I'm not sure how it will work with your
>>>>>> intended use
>>>>>> > > case.
>>>>>> > >
>>>>>> > > -Abe
>>>>>> > >
>>>>>> > >
>>>>>> > > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <
>>>>>> jarcec@apache.org>wrote:
>>>>>> > >
>>>>>> > >> Hi Anthony,
>>>>>> > >> I'm glad to see your interested in Sqoop project! I have to
>>>>>> mention that
>>>>>> > >> whereas the direct usage of the Sqoop classes might work, it's
>>>>>> not
>>>>>> > >> officially supported API and you might hit some issues along the
>>>>>> way.
>>>>>> > >>
>>>>>> > >> I would strongly suggest to upgrade to latest Sqoop version as
>>>>>> the 1.4.0
>>>>>> > >> is quite old. In most cases using the --driver parameter is not
>>>>>> needed as
>>>>>> > >> it will instruct Sqoop to use the Generic JDBC Driver instead of
>>>>>> > >> specialized MySQL connector, I would suggest to drop this
>>>>>> parameter
>>>>>> > >> entirely.
>>>>>> > >>
>>>>>> > >> Jarcec
>>>>>> > >>
>>>>>> > >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
>>>>>> > >> > Hello,
>>>>>> > >> >
>>>>>> > >> > I am trying to use Sqoop in an osgi pluging but it wont work
>>>>>> and I dont
>>>>>> > >> > know why.
>>>>>> > >> > I keep getting the following error:
>>>>>> > >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 |
>>>>>> BaseSqoopTool  |
>>>>>> > >> ?  ?
>>>>>> > >> > |  -  -  | Got error creating database manager:
>>>>>> java.io.IOException: No
>>>>>> > >> > manager for connect string: jdbc:mysql://...."
>>>>>> > >> >
>>>>>> > >> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have
>>>>>> my mysql
>>>>>> > >> > driver loader (i can use in the code).
>>>>>> > >> >
>>>>>> > >> > the part who throw an exception is:
>>>>>> > >> >
>>>>>> > >> > String[] str = {
>>>>>> > >> > "import",
>>>>>> > >> > "--driver", "com.mysql.jdbc.Driver",
>>>>>> > >> > "--connect", "jdbc:mysql://___MY_DB___",
>>>>>> > >> > "--hadoop-home", __HOME__,
>>>>>> > >> > "--query", "Select [.....] AND $CONDITIONS",
>>>>>> > >> > "--split-by", "id",
>>>>>> > >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
>>>>>> > >> > "--username", "foo",
>>>>>> > >> > "--password", "barou"};
>>>>>> > >> >
>>>>>> > >> > return Sqoop.runTool(str);
>>>>>> > >> >
>>>>>> > >> > Thanks,
>>>>>> > >> > Anthony
>>>>>> > >>
>>>>>> > >
>>>>>> > >
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Sqoop and OSGI

Posted by Abraham Elmahrek <ab...@cloudera.com>.

Anthony,

You've set that configuration incorrectly. It should be conf.set(
"sqoop.connection.factories", "com.cloudera.sqoop.manager.ConnMananger");.
Sqoop instantiates a Configuration object and adds resources to it.

-Abe


On Thu, Jun 20, 2013 at 6:44 PM, corbacho anthony <ac...@nflabs.com>wrote:

> Hello Abe.
>
> The think is, I didn`t install sqoop, I use it via pom on my java code.
> I added on my code this:
>
> Configuration conf = new HdfsConfiguration();
> conf.set("sqoop.connection.factories",
> "com.cloudera.sqoop.apache.ConnMananger");
> [....]
>
> return Sqoop.runTool(str, conf);
> But still the same error.
>
>
> On Thu, Jun 20, 2013 at 12:50 PM, Abraham Elmahrek <ab...@cloudera.com>wrote:
>
>> Anthony,
>>
>> Weird, that should be com.cloudera.sqoop.manager.ConnMananger. I suspect
>> there's a problem with your sqoop-site.xml (if its even being parsed at
>> all) or classpath. The sqoop startup script sets SQOOP_HOME_DIR,
>> SQOOP_CONF_DIR, HADOOP_CLASSPATH, and a few other environment variables. I
>> suspect there's something required there.
>>
>> -Abe
>>
>>
>> On Wed, Jun 19, 2013 at 6:53 PM, corbacho anthony <ac...@nflabs.com>wrote:
>>
>>> I tried an same issues.
>>> The problem is Sqoop cannot
>>> find com.cloudera.sqoop.apache.ConnMananger...
>>>
>>>
>>> On Thu, Jun 20, 2013 at 10:33 AM, corbacho anthony <acorbacho@nflabs.com
>>> > wrote:
>>>
>>>> I used
>>>> https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/sqoop/sqoop/
>>>>
>>>> I will try with sqoop 1.4.3.
>>>> Thank you Jarec.
>>>>
>>>> Anthony
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 10:28 AM, Jarek Jarcec Cecho <jarcec@apache.org
>>>> > wrote:
>>>>
>>>>> Hi Anthony,
>>>>> what maven repository are you using?
>>>>>
>>>>> Cloudera maven repository [1] seems to be containing all released
>>>>> versions (instructions how to use this repository are in installation guide
>>>>> [3]). Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts available
>>>>> in main maven repository [2].
>>>>>
>>>>> Jarcec
>>>>>
>>>>> Links:
>>>>> 1:
>>>>> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
>>>>> 2: http://search.maven.org/#browse%7C585117616
>>>>> 3:
>>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html
>>>>>
>>>>> On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
>>>>> > hello Jarec and Abe.
>>>>> >
>>>>> > Abe: the debugging mode show nothing :(
>>>>> > Jarec: In cloudera repo I can only find Sqoop 1.4.0.
>>>>> >
>>>>> > There is the full stack (in case):
>>>>> > 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory |
>>>>> Could
>>>>> > not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not
>>>>> found)
>>>>> > 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool |
>>>>> Got
>>>>> > error creating database manager: java.io.IOException: No manager for
>>>>> > connect string: jdbc:mysql://xxx:0000/x
>>>>> > at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
>>>>> > at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>>>>> > at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>>>>> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
>>>>> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>>>> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>>>> > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
>>>>> > at
>>>>> com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
>>>>> > at
>>>>> com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> > at
>>>>> >
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> > at
>>>>> >
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> > at
>>>>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
>>>>> > at
>>>>> org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
>>>>> > at
>>>>> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
>>>>> > at com.sun.proxy.$Proxy76.test(Unknown Source)
>>>>> > at
>>>>> >
>>>>> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
>>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> > at
>>>>> >
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> > at
>>>>> >
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
>>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
>>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
>>>>> > at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
>>>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>>>>> > at
>>>>> >
>>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
>>>>> > at
>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
>>>>> > at
>>>>> >
>>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
>>>>> > at
>>>>> >
>>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
>>>>> > at
>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>>> > at
>>>>> >
>>>>> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>>>>> > at org.eclipse.jetty.server.Server.handle(Server.java:346)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>>>> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
>>>>> > at
>>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
>>>>> > at
>>>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>>>>> > at
>>>>> >
>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>>>>> > at java.lang.Thread.run(Thread.java:680)
>>>>> >
>>>>> > Thanks
>>>>> > Anthony
>>>>> >
>>>>> >
>>>>> > On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com>
>>>>> wrote:
>>>>> >
>>>>> > > Anthony,
>>>>> > >
>>>>> > > You could also try enabling debug logging to see what managers are
>>>>> being
>>>>> > > loaded (if any). This can normally be done by adding "--verbose"
>>>>> to your
>>>>> > > list of arguments. I'm not sure how it will work with your
>>>>> intended use
>>>>> > > case.
>>>>> > >
>>>>> > > -Abe
>>>>> > >
>>>>> > >
>>>>> > > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <
>>>>> jarcec@apache.org>wrote:
>>>>> > >
>>>>> > >> Hi Anthony,
>>>>> > >> I'm glad to see your interested in Sqoop project! I have to
>>>>> mention that
>>>>> > >> whereas the direct usage of the Sqoop classes might work, it's not
>>>>> > >> officially supported API and you might hit some issues along the
>>>>> way.
>>>>> > >>
>>>>> > >> I would strongly suggest to upgrade to latest Sqoop version as
>>>>> the 1.4.0
>>>>> > >> is quite old. In most cases using the --driver parameter is not
>>>>> needed as
>>>>> > >> it will instruct Sqoop to use the Generic JDBC Driver instead of
>>>>> > >> specialized MySQL connector, I would suggest to drop this
>>>>> parameter
>>>>> > >> entirely.
>>>>> > >>
>>>>> > >> Jarcec
>>>>> > >>
>>>>> > >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
>>>>> > >> > Hello,
>>>>> > >> >
>>>>> > >> > I am trying to use Sqoop in an osgi pluging but it wont work
>>>>> and I dont
>>>>> > >> > know why.
>>>>> > >> > I keep getting the following error:
>>>>> > >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 |
>>>>> BaseSqoopTool  |
>>>>> > >> ?  ?
>>>>> > >> > |  -  -  | Got error creating database manager:
>>>>> java.io.IOException: No
>>>>> > >> > manager for connect string: jdbc:mysql://...."
>>>>> > >> >
>>>>> > >> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have
>>>>> my mysql
>>>>> > >> > driver loader (i can use in the code).
>>>>> > >> >
>>>>> > >> > the part who throw an exception is:
>>>>> > >> >
>>>>> > >> > String[] str = {
>>>>> > >> > "import",
>>>>> > >> > "--driver", "com.mysql.jdbc.Driver",
>>>>> > >> > "--connect", "jdbc:mysql://___MY_DB___",
>>>>> > >> > "--hadoop-home", __HOME__,
>>>>> > >> > "--query", "Select [.....] AND $CONDITIONS",
>>>>> > >> > "--split-by", "id",
>>>>> > >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
>>>>> > >> > "--username", "foo",
>>>>> > >> > "--password", "barou"};
>>>>> > >> >
>>>>> > >> > return Sqoop.runTool(str);
>>>>> > >> >
>>>>> > >> > Thanks,
>>>>> > >> > Anthony
>>>>> > >>
>>>>> > >
>>>>> > >
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Sqoop and OSGI

Posted by corbacho anthony <ac...@nflabs.com>.

Hello Abe.

The think is, I didn`t install sqoop, I use it via pom on my java code.
I added on my code this:

Configuration conf = new HdfsConfiguration();
conf.set("sqoop.connection.factories",
"com.cloudera.sqoop.apache.ConnMananger");
[....]

return Sqoop.runTool(str, conf);
But still the same error.


On Thu, Jun 20, 2013 at 12:50 PM, Abraham Elmahrek <ab...@cloudera.com> wrote:

> Anthony,
>
> Weird, that should be com.cloudera.sqoop.manager.ConnMananger. I suspect
> there's a problem with your sqoop-site.xml (if its even being parsed at
> all) or classpath. The sqoop startup script sets SQOOP_HOME_DIR,
> SQOOP_CONF_DIR, HADOOP_CLASSPATH, and a few other environment variables. I
> suspect there's something required there.
>
> -Abe
>
>
> On Wed, Jun 19, 2013 at 6:53 PM, corbacho anthony <ac...@nflabs.com>wrote:
>
>> I tried an same issues.
>> The problem is Sqoop cannot find com.cloudera.sqoop.apache.ConnMananger...
>>
>>
>> On Thu, Jun 20, 2013 at 10:33 AM, corbacho anthony <ac...@nflabs.com>wrote:
>>
>>> I used
>>> https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/sqoop/sqoop/
>>>
>>> I will try with sqoop 1.4.3.
>>> Thank you Jarec.
>>>
>>> Anthony
>>>
>>>
>>> On Thu, Jun 20, 2013 at 10:28 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
>>>
>>>> Hi Anthony,
>>>> what maven repository are you using?
>>>>
>>>> Cloudera maven repository [1] seems to be containing all released
>>>> versions (instructions how to use this repository are in installation guide
>>>> [3]). Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts available
>>>> in main maven repository [2].
>>>>
>>>> Jarcec
>>>>
>>>> Links:
>>>> 1:
>>>> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
>>>> 2: http://search.maven.org/#browse%7C585117616
>>>> 3:
>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html
>>>>
>>>> On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
>>>> > hello Jarec and Abe.
>>>> >
>>>> > Abe: the debugging mode show nothing :(
>>>> > Jarec: In cloudera repo I can only find Sqoop 1.4.0.
>>>> >
>>>> > There is the full stack (in case):
>>>> > 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory |
>>>> Could
>>>> > not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not
>>>> found)
>>>> > 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool |
>>>> Got
>>>> > error creating database manager: java.io.IOException: No manager for
>>>> > connect string: jdbc:mysql://xxx:0000/x
>>>> > at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
>>>> > at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>>>> > at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>>>> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
>>>> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>>> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>>> > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
>>>> > at
>>>> com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
>>>> > at
>>>> com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> > at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> > at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>> > at
>>>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
>>>> > at
>>>> org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
>>>> > at
>>>> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
>>>> > at com.sun.proxy.$Proxy76.test(Unknown Source)
>>>> > at
>>>> >
>>>> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> > at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> > at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>>> > at
>>>> >
>>>> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
>>>> > at
>>>> >
>>>> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
>>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
>>>> > at
>>>> >
>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
>>>> > at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> > at
>>>> >
>>>> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
>>>> > at
>>>> >
>>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
>>>> > at
>>>> >
>>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
>>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>>>> > at
>>>> >
>>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
>>>> > at
>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
>>>> > at
>>>> >
>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
>>>> > at
>>>> >
>>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
>>>> > at
>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>> > at
>>>> >
>>>> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>>>> > at org.eclipse.jetty.server.Server.handle(Server.java:346)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>>> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
>>>> > at
>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
>>>> > at
>>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>>>> > at
>>>> >
>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>>>> > at java.lang.Thread.run(Thread.java:680)
>>>> >
>>>> > Thanks
>>>> > Anthony
>>>> >
>>>> >
>>>> > On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com>
>>>> wrote:
>>>> >
>>>> > > Anthony,
>>>> > >
>>>> > > You could also try enabling debug logging to see what managers are
>>>> being
>>>> > > loaded (if any). This can normally be done by adding "--verbose" to
>>>> your
>>>> > > list of arguments. I'm not sure how it will work with your intended
>>>> use
>>>> > > case.
>>>> > >
>>>> > > -Abe
>>>> > >
>>>> > >
>>>> > > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <
>>>> jarcec@apache.org>wrote:
>>>> > >
>>>> > >> Hi Anthony,
>>>> > >> I'm glad to see your interested in Sqoop project! I have to
>>>> mention that
>>>> > >> whereas the direct usage of the Sqoop classes might work, it's not
>>>> > >> officially supported API and you might hit some issues along the
>>>> way.
>>>> > >>
>>>> > >> I would strongly suggest to upgrade to latest Sqoop version as the
>>>> 1.4.0
>>>> > >> is quite old. In most cases using the --driver parameter is not
>>>> needed as
>>>> > >> it will instruct Sqoop to use the Generic JDBC Driver instead of
>>>> > >> specialized MySQL connector, I would suggest to drop this parameter
>>>> > >> entirely.
>>>> > >>
>>>> > >> Jarcec
>>>> > >>
>>>> > >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
>>>> > >> > Hello,
>>>> > >> >
>>>> > >> > I am trying to use Sqoop in an osgi pluging but it wont work and
>>>> I dont
>>>> > >> > know why.
>>>> > >> > I keep getting the following error:
>>>> > >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 |
>>>> BaseSqoopTool  |
>>>> > >> ?  ?
>>>> > >> > |  -  -  | Got error creating database manager:
>>>> java.io.IOException: No
>>>> > >> > manager for connect string: jdbc:mysql://...."
>>>> > >> >
>>>> > >> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my
>>>> mysql
>>>> > >> > driver loader (i can use in the code).
>>>> > >> >
>>>> > >> > the part who throw an exception is:
>>>> > >> >
>>>> > >> > String[] str = {
>>>> > >> > "import",
>>>> > >> > "--driver", "com.mysql.jdbc.Driver",
>>>> > >> > "--connect", "jdbc:mysql://___MY_DB___",
>>>> > >> > "--hadoop-home", __HOME__,
>>>> > >> > "--query", "Select [.....] AND $CONDITIONS",
>>>> > >> > "--split-by", "id",
>>>> > >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
>>>> > >> > "--username", "foo",
>>>> > >> > "--password", "barou"};
>>>> > >> >
>>>> > >> > return Sqoop.runTool(str);
>>>> > >> >
>>>> > >> > Thanks,
>>>> > >> > Anthony
>>>> > >>
>>>> > >
>>>> > >
>>>>
>>>
>>>
>>
>

Re: Sqoop and OSGI

Posted by Abraham Elmahrek <ab...@cloudera.com>.

Anthony,

Weird, that should be com.cloudera.sqoop.manager.ConnMananger. I suspect
there's a problem with your sqoop-site.xml (if its even being parsed at
all) or classpath. The sqoop startup script sets SQOOP_HOME_DIR,
SQOOP_CONF_DIR, HADOOP_CLASSPATH, and a few other environment variables. I
suspect there's something required there.

-Abe


On Wed, Jun 19, 2013 at 6:53 PM, corbacho anthony <ac...@nflabs.com>wrote:

> I tried an same issues.
> The problem is Sqoop cannot find com.cloudera.sqoop.apache.ConnMananger...
>
>
> On Thu, Jun 20, 2013 at 10:33 AM, corbacho anthony <ac...@nflabs.com>wrote:
>
>> I used
>> https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/sqoop/sqoop/
>>
>> I will try with sqoop 1.4.3.
>> Thank you Jarec.
>>
>> Anthony
>>
>>
>> On Thu, Jun 20, 2013 at 10:28 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
>>
>>> Hi Anthony,
>>> what maven repository are you using?
>>>
>>> Cloudera maven repository [1] seems to be containing all released
>>> versions (instructions how to use this repository are in installation guide
>>> [3]). Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts available
>>> in main maven repository [2].
>>>
>>> Jarcec
>>>
>>> Links:
>>> 1:
>>> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
>>> 2: http://search.maven.org/#browse%7C585117616
>>> 3:
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html
>>>
>>> On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
>>> > hello Jarec and Abe.
>>> >
>>> > Abe: the debugging mode show nothing :(
>>> > Jarec: In cloudera repo I can only find Sqoop 1.4.0.
>>> >
>>> > There is the full stack (in case):
>>> > 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory |
>>> Could
>>> > not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not
>>> found)
>>> > 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool |
>>> Got
>>> > error creating database manager: java.io.IOException: No manager for
>>> > connect string: jdbc:mysql://xxx:0000/x
>>> > at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
>>> > at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>>> > at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>>> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
>>> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>> > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
>>> > at
>>> com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
>>> > at
>>> com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>> > at
>>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
>>> > at
>>> org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
>>> > at
>>> org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
>>> > at com.sun.proxy.$Proxy76.test(Unknown Source)
>>> > at
>>> >
>>> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>> > at
>>> >
>>> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
>>> > at
>>> >
>>> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
>>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
>>> > at
>>> >
>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
>>> > at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> > at
>>> >
>>> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
>>> > at
>>> >
>>> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
>>> > at
>>> >
>>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
>>> > at
>>> >
>>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
>>> > at
>>> >
>>> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
>>> > at
>>> >
>>> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
>>> > at
>>> >
>>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
>>> > at
>>> >
>>> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
>>> > at
>>> >
>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
>>> > at
>>> >
>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>>> > at
>>> >
>>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
>>> > at
>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
>>> > at
>>> >
>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
>>> > at
>>> >
>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
>>> > at
>>> >
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>>> > at
>>> >
>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
>>> > at
>>> >
>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
>>> > at
>>> >
>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
>>> > at
>>> >
>>> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
>>> > at
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
>>> > at
>>> >
>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
>>> > at
>>> >
>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
>>> > at
>>> >
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>> > at
>>> >
>>> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
>>> > at
>>> >
>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>>> > at org.eclipse.jetty.server.Server.handle(Server.java:346)
>>> > at
>>> >
>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
>>> > at
>>> >
>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
>>> > at
>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
>>> > at
>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
>>> > at
>>> >
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
>>> > at
>>> >
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>>> > at
>>> >
>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>>> > at java.lang.Thread.run(Thread.java:680)
>>> >
>>> > Thanks
>>> > Anthony
>>> >
>>> >
>>> > On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com>
>>> wrote:
>>> >
>>> > > Anthony,
>>> > >
>>> > > You could also try enabling debug logging to see what managers are
>>> being
>>> > > loaded (if any). This can normally be done by adding "--verbose" to
>>> your
>>> > > list of arguments. I'm not sure how it will work with your intended
>>> use
>>> > > case.
>>> > >
>>> > > -Abe
>>> > >
>>> > >
>>> > > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <
>>> jarcec@apache.org>wrote:
>>> > >
>>> > >> Hi Anthony,
>>> > >> I'm glad to see your interested in Sqoop project! I have to mention
>>> that
>>> > >> whereas the direct usage of the Sqoop classes might work, it's not
>>> > >> officially supported API and you might hit some issues along the
>>> way.
>>> > >>
>>> > >> I would strongly suggest to upgrade to latest Sqoop version as the
>>> 1.4.0
>>> > >> is quite old. In most cases using the --driver parameter is not
>>> needed as
>>> > >> it will instruct Sqoop to use the Generic JDBC Driver instead of
>>> > >> specialized MySQL connector, I would suggest to drop this parameter
>>> > >> entirely.
>>> > >>
>>> > >> Jarcec
>>> > >>
>>> > >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
>>> > >> > Hello,
>>> > >> >
>>> > >> > I am trying to use Sqoop in an osgi pluging but it wont work and
>>> I dont
>>> > >> > know why.
>>> > >> > I keep getting the following error:
>>> > >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 |
>>> BaseSqoopTool  |
>>> > >> ?  ?
>>> > >> > |  -  -  | Got error creating database manager:
>>> java.io.IOException: No
>>> > >> > manager for connect string: jdbc:mysql://...."
>>> > >> >
>>> > >> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my
>>> mysql
>>> > >> > driver loader (i can use in the code).
>>> > >> >
>>> > >> > the part who throw an exception is:
>>> > >> >
>>> > >> > String[] str = {
>>> > >> > "import",
>>> > >> > "--driver", "com.mysql.jdbc.Driver",
>>> > >> > "--connect", "jdbc:mysql://___MY_DB___",
>>> > >> > "--hadoop-home", __HOME__,
>>> > >> > "--query", "Select [.....] AND $CONDITIONS",
>>> > >> > "--split-by", "id",
>>> > >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
>>> > >> > "--username", "foo",
>>> > >> > "--password", "barou"};
>>> > >> >
>>> > >> > return Sqoop.runTool(str);
>>> > >> >
>>> > >> > Thanks,
>>> > >> > Anthony
>>> > >>
>>> > >
>>> > >
>>>
>>
>>
>

Re: Sqoop and OSGI

Posted by corbacho anthony <ac...@nflabs.com>.

I tried an same issues.
The problem is Sqoop cannot find com.cloudera.sqoop.apache.ConnMananger...


On Thu, Jun 20, 2013 at 10:33 AM, corbacho anthony <ac...@nflabs.com>wrote:

> I used
> https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/sqoop/sqoop/
>
> I will try with sqoop 1.4.3.
> Thank you Jarec.
>
> Anthony
>
>
> On Thu, Jun 20, 2013 at 10:28 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
>
>> Hi Anthony,
>> what maven repository are you using?
>>
>> Cloudera maven repository [1] seems to be containing all released
>> versions (instructions how to use this repository are in installation guide
>> [3]). Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts available
>> in main maven repository [2].
>>
>> Jarcec
>>
>> Links:
>> 1:
>> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
>> 2: http://search.maven.org/#browse%7C585117616
>> 3:
>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html
>>
>> On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
>> > hello Jarec and Abe.
>> >
>> > Abe: the debugging mode show nothing :(
>> > Jarec: In cloudera repo I can only find Sqoop 1.4.0.
>> >
>> > There is the full stack (in case):
>> > 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory | Could
>> > not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not
>> found)
>> > 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool | Got
>> > error creating database manager: java.io.IOException: No manager for
>> > connect string: jdbc:mysql://xxx:0000/x
>> > at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
>> > at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>> > at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
>> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>> > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
>> > at
>> com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
>> > at
>> com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
>> > at
>> org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
>> > at org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
>> > at com.sun.proxy.$Proxy76.test(Unknown Source)
>> > at
>> >
>> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> >
>> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
>> > at
>> >
>> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
>> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
>> > at
>> >
>> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
>> > at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> > at
>> >
>> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
>> > at
>> >
>> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
>> > at
>> >
>> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
>> > at
>> >
>> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
>> > at
>> >
>> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
>> > at
>> >
>> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
>> > at
>> >
>> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
>> > at
>> >
>> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
>> > at
>> >
>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
>> > at
>> >
>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>> > at
>> >
>> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
>> > at
>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
>> > at
>> >
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
>> > at
>> >
>> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>> > at
>> >
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
>> > at
>> >
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
>> > at
>> >
>> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
>> > at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
>> > at
>> >
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>> > at
>> >
>> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
>> > at
>> >
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>> > at org.eclipse.jetty.server.Server.handle(Server.java:346)
>> > at
>> >
>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
>> > at
>> >
>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
>> > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
>> > at
>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
>> > at
>> >
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
>> > at
>> >
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>> > at
>> >
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
>> > at java.lang.Thread.run(Thread.java:680)
>> >
>> > Thanks
>> > Anthony
>> >
>> >
>> > On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com>
>> wrote:
>> >
>> > > Anthony,
>> > >
>> > > You could also try enabling debug logging to see what managers are
>> being
>> > > loaded (if any). This can normally be done by adding "--verbose" to
>> your
>> > > list of arguments. I'm not sure how it will work with your intended
>> use
>> > > case.
>> > >
>> > > -Abe
>> > >
>> > >
>> > > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <
>> jarcec@apache.org>wrote:
>> > >
>> > >> Hi Anthony,
>> > >> I'm glad to see your interested in Sqoop project! I have to mention
>> that
>> > >> whereas the direct usage of the Sqoop classes might work, it's not
>> > >> officially supported API and you might hit some issues along the way.
>> > >>
>> > >> I would strongly suggest to upgrade to latest Sqoop version as the
>> 1.4.0
>> > >> is quite old. In most cases using the --driver parameter is not
>> needed as
>> > >> it will instruct Sqoop to use the Generic JDBC Driver instead of
>> > >> specialized MySQL connector, I would suggest to drop this parameter
>> > >> entirely.
>> > >>
>> > >> Jarcec
>> > >>
>> > >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
>> > >> > Hello,
>> > >> >
>> > >> > I am trying to use Sqoop in an osgi pluging but it wont work and I
>> dont
>> > >> > know why.
>> > >> > I keep getting the following error:
>> > >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 |
>> BaseSqoopTool  |
>> > >> ?  ?
>> > >> > |  -  -  | Got error creating database manager:
>> java.io.IOException: No
>> > >> > manager for connect string: jdbc:mysql://...."
>> > >> >
>> > >> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my
>> mysql
>> > >> > driver loader (i can use in the code).
>> > >> >
>> > >> > the part who throw an exception is:
>> > >> >
>> > >> > String[] str = {
>> > >> > "import",
>> > >> > "--driver", "com.mysql.jdbc.Driver",
>> > >> > "--connect", "jdbc:mysql://___MY_DB___",
>> > >> > "--hadoop-home", __HOME__,
>> > >> > "--query", "Select [.....] AND $CONDITIONS",
>> > >> > "--split-by", "id",
>> > >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
>> > >> > "--username", "foo",
>> > >> > "--password", "barou"};
>> > >> >
>> > >> > return Sqoop.runTool(str);
>> > >> >
>> > >> > Thanks,
>> > >> > Anthony
>> > >>
>> > >
>> > >
>>
>
>

Re: Sqoop and OSGI

Posted by corbacho anthony <ac...@nflabs.com>.

I used
https://repository.cloudera.com/artifactory/cloudera-repos/com/cloudera/sqoop/sqoop/

I will try with sqoop 1.4.3.
Thank you Jarec.

Anthony


On Thu, Jun 20, 2013 at 10:28 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:

> Hi Anthony,
> what maven repository are you using?
>
> Cloudera maven repository [1] seems to be containing all released versions
> (instructions how to use this repository are in installation guide [3]).
> Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts available in
> main maven repository [2].
>
> Jarcec
>
> Links:
> 1:
> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
> 2: http://search.maven.org/#browse%7C585117616
> 3:
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html
>
> On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
> > hello Jarec and Abe.
> >
> > Abe: the debugging mode show nothing :(
> > Jarec: In cloudera repo I can only find Sqoop 1.4.0.
> >
> > There is the full stack (in case):
> > 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory | Could
> > not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not
> found)
> > 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool | Got
> > error creating database manager: java.io.IOException: No manager for
> > connect string: jdbc:mysql://xxx:0000/x
> > at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
> > at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
> > at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
> > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
> > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
> > at
> com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
> > at com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
> > at
> org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
> > at org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
> > at com.sun.proxy.$Proxy76.test(Unknown Source)
> > at
> >
> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
> > at
> >
> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
> > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
> > at
> >
> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> > at
> >
> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
> > at
> >
> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
> > at
> >
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
> > at
> >
> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
> > at
> >
> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
> > at
> >
> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
> > at
> >
> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
> > at
> >
> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
> > at
> >
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
> > at
> >
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
> > at
> >
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
> > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
> > at
> >
> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
> > at
> >
> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
> > at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> > at
> >
> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
> > at org.eclipse.jetty.server.Server.handle(Server.java:346)
> > at
> >
> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
> > at
> >
> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
> > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
> > at
> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
> > at
> >
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
> > at
> >
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
> > at java.lang.Thread.run(Thread.java:680)
> >
> > Thanks
> > Anthony
> >
> >
> > On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com>
> wrote:
> >
> > > Anthony,
> > >
> > > You could also try enabling debug logging to see what managers are
> being
> > > loaded (if any). This can normally be done by adding "--verbose" to
> your
> > > list of arguments. I'm not sure how it will work with your intended use
> > > case.
> > >
> > > -Abe
> > >
> > >
> > > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <jarcec@apache.org
> >wrote:
> > >
> > >> Hi Anthony,
> > >> I'm glad to see your interested in Sqoop project! I have to mention
> that
> > >> whereas the direct usage of the Sqoop classes might work, it's not
> > >> officially supported API and you might hit some issues along the way.
> > >>
> > >> I would strongly suggest to upgrade to latest Sqoop version as the
> 1.4.0
> > >> is quite old. In most cases using the --driver parameter is not
> needed as
> > >> it will instruct Sqoop to use the Generic JDBC Driver instead of
> > >> specialized MySQL connector, I would suggest to drop this parameter
> > >> entirely.
> > >>
> > >> Jarcec
> > >>
> > >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
> > >> > Hello,
> > >> >
> > >> > I am trying to use Sqoop in an osgi pluging but it wont work and I
> dont
> > >> > know why.
> > >> > I keep getting the following error:
> > >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 | BaseSqoopTool
>  |
> > >> ?  ?
> > >> > |  -  -  | Got error creating database manager:
> java.io.IOException: No
> > >> > manager for connect string: jdbc:mysql://...."
> > >> >
> > >> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my
> mysql
> > >> > driver loader (i can use in the code).
> > >> >
> > >> > the part who throw an exception is:
> > >> >
> > >> > String[] str = {
> > >> > "import",
> > >> > "--driver", "com.mysql.jdbc.Driver",
> > >> > "--connect", "jdbc:mysql://___MY_DB___",
> > >> > "--hadoop-home", __HOME__,
> > >> > "--query", "Select [.....] AND $CONDITIONS",
> > >> > "--split-by", "id",
> > >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
> > >> > "--username", "foo",
> > >> > "--password", "barou"};
> > >> >
> > >> > return Sqoop.runTool(str);
> > >> >
> > >> > Thanks,
> > >> > Anthony
> > >>
> > >
> > >
>

Re: Sqoop and OSGI

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi Anthony,
what maven repository are you using?

Cloudera maven repository [1] seems to be containing all released versions (instructions how to use this repository are in installation guide [3]). Also thanks to SQOOP-1007 we do have vanila Sqoop artifacts available in main maven repository [2].

Jarcec

Links:
1: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sqoop/sqoop/
2: http://search.maven.org/#browse%7C585117616
3: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_31.html

On Thu, Jun 20, 2013 at 10:17:35AM +0900, corbacho anthony wrote:
> hello Jarec and Abe.
> 
> Abe: the debugging mode show nothing :(
> Jarec: In cloudera repo I can only find Sqoop 1.4.0.
> 
> There is the full stack (in case):
> 2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory | Could
> not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not found)
> 2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool | Got
> error creating database manager: java.io.IOException: No manager for
> connect string: jdbc:mysql://xxx:0000/x
> at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
> at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
> at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
> at com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
> at com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
> at org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
> at org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
> at com.sun.proxy.$Proxy76.test(Unknown Source)
> at
> com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
> at
> org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
> at
> org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
> at
> org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
> at
> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
> at
> org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
> at
> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
> at
> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
> at
> org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
> at
> org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
> at
> org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
> org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
> at org.eclipse.jetty.server.Server.handle(Server.java:346)
> at
> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
> at
> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
> at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
> at java.lang.Thread.run(Thread.java:680)
> 
> Thanks
> Anthony
> 
> 
> On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com> wrote:
> 
> > Anthony,
> >
> > You could also try enabling debug logging to see what managers are being
> > loaded (if any). This can normally be done by adding "--verbose" to your
> > list of arguments. I'm not sure how it will work with your intended use
> > case.
> >
> > -Abe
> >
> >
> > On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
> >
> >> Hi Anthony,
> >> I'm glad to see your interested in Sqoop project! I have to mention that
> >> whereas the direct usage of the Sqoop classes might work, it's not
> >> officially supported API and you might hit some issues along the way.
> >>
> >> I would strongly suggest to upgrade to latest Sqoop version as the 1.4.0
> >> is quite old. In most cases using the --driver parameter is not needed as
> >> it will instruct Sqoop to use the Generic JDBC Driver instead of
> >> specialized MySQL connector, I would suggest to drop this parameter
> >> entirely.
> >>
> >> Jarcec
> >>
> >> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
> >> > Hello,
> >> >
> >> > I am trying to use Sqoop in an osgi pluging but it wont work and I dont
> >> > know why.
> >> > I keep getting the following error:
> >> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 | BaseSqoopTool  |
> >> ?  ?
> >> > |  -  -  | Got error creating database manager: java.io.IOException: No
> >> > manager for connect string: jdbc:mysql://...."
> >> >
> >> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my mysql
> >> > driver loader (i can use in the code).
> >> >
> >> > the part who throw an exception is:
> >> >
> >> > String[] str = {
> >> > "import",
> >> > "--driver", "com.mysql.jdbc.Driver",
> >> > "--connect", "jdbc:mysql://___MY_DB___",
> >> > "--hadoop-home", __HOME__,
> >> > "--query", "Select [.....] AND $CONDITIONS",
> >> > "--split-by", "id",
> >> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
> >> > "--username", "foo",
> >> > "--password", "barou"};
> >> >
> >> > return Sqoop.runTool(str);
> >> >
> >> > Thanks,
> >> > Anthony
> >>
> >
> >

Re: Sqoop and OSGI

Posted by corbacho anthony <ac...@nflabs.com>.

hello Jarec and Abe.

Abe: the debugging mode show nothing :(
Jarec: In cloudera repo I can only find Sqoop 1.4.0.

There is the full stack (in case):
2013-06-20 10:10:29,022 | ERROR | qtp1742391956-59 | ConnFactory | Could
not load ManagerFactory com.cloudera.sqoop.apache.ConnMananger (not found)
2013-06-20 10:10:29,023 | ERROR | qtp1742391956-59 | BaseSqoopTool | Got
error creating database manager: java.io.IOException: No manager for
connect string: jdbc:mysql://xxx:0000/x
at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:119)
at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:441)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
at com.nflabs.com.peloton.imprt.db.SqoopImport.tested(SqoopImport.java:131)
at com.nflabs.peloton.imprt.impl.ImprtSvcImpl.test(ImprtSvcImpl.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.aries.proxy.impl.ProxyHandler$1.invoke(ProxyHandler.java:50)
at org.apache.aries.proxy.impl.DefaultWrapper.invoke(DefaultWrapper.java:31)
at org.apache.aries.proxy.impl.ProxyHandler.invoke(ProxyHandler.java:78)
at com.sun.proxy.$Proxy76.test(Unknown Source)
at
com.nflabs.peloton.rs.imprt.ImprtService.BrowseDirectory1(ImprtService.java:478)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
at
org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92)
at
org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
at
org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:106)
at
org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
at
org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at
org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:207)
at
org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:209)
at
org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:152)
at
org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:114)
at
org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:185)
at
org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:113)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
at
org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:164)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
at
org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:480)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:937)
at
org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:116)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:871)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:72)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
at org.eclipse.jetty.server.Server.handle(Server.java:346)
at
org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
at
org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:601)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:214)
at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
at java.lang.Thread.run(Thread.java:680)

Thanks
Anthony


On Thu, Jun 20, 2013 at 2:52 AM, Abraham Elmahrek <ab...@cloudera.com> wrote:

> Anthony,
>
> You could also try enabling debug logging to see what managers are being
> loaded (if any). This can normally be done by adding "--verbose" to your
> list of arguments. I'm not sure how it will work with your intended use
> case.
>
> -Abe
>
>
> On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
>
>> Hi Anthony,
>> I'm glad to see your interested in Sqoop project! I have to mention that
>> whereas the direct usage of the Sqoop classes might work, it's not
>> officially supported API and you might hit some issues along the way.
>>
>> I would strongly suggest to upgrade to latest Sqoop version as the 1.4.0
>> is quite old. In most cases using the --driver parameter is not needed as
>> it will instruct Sqoop to use the Generic JDBC Driver instead of
>> specialized MySQL connector, I would suggest to drop this parameter
>> entirely.
>>
>> Jarcec
>>
>> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
>> > Hello,
>> >
>> > I am trying to use Sqoop in an osgi pluging but it wont work and I dont
>> > know why.
>> > I keep getting the following error:
>> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 | BaseSqoopTool  |
>> ?  ?
>> > |  -  -  | Got error creating database manager: java.io.IOException: No
>> > manager for connect string: jdbc:mysql://...."
>> >
>> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my mysql
>> > driver loader (i can use in the code).
>> >
>> > the part who throw an exception is:
>> >
>> > String[] str = {
>> > "import",
>> > "--driver", "com.mysql.jdbc.Driver",
>> > "--connect", "jdbc:mysql://___MY_DB___",
>> > "--hadoop-home", __HOME__,
>> > "--query", "Select [.....] AND $CONDITIONS",
>> > "--split-by", "id",
>> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
>> > "--username", "foo",
>> > "--password", "barou"};
>> >
>> > return Sqoop.runTool(str);
>> >
>> > Thanks,
>> > Anthony
>>
>
>

Re: Sqoop and OSGI

Posted by Abraham Elmahrek <ab...@cloudera.com>.

Anthony,

You could also try enabling debug logging to see what managers are being
loaded (if any). This can normally be done by adding "--verbose" to your
list of arguments. I'm not sure how it will work with your intended use
case.

-Abe


On Wed, Jun 19, 2013 at 7:57 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:

> Hi Anthony,
> I'm glad to see your interested in Sqoop project! I have to mention that
> whereas the direct usage of the Sqoop classes might work, it's not
> officially supported API and you might hit some issues along the way.
>
> I would strongly suggest to upgrade to latest Sqoop version as the 1.4.0
> is quite old. In most cases using the --driver parameter is not needed as
> it will instruct Sqoop to use the Generic JDBC Driver instead of
> specialized MySQL connector, I would suggest to drop this parameter
> entirely.
>
> Jarcec
>
> On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
> > Hello,
> >
> > I am trying to use Sqoop in an osgi pluging but it wont work and I dont
> > know why.
> > I keep getting the following error:
> > "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 | BaseSqoopTool  | ?
>  ?
> > |  -  -  | Got error creating database manager: java.io.IOException: No
> > manager for connect string: jdbc:mysql://...."
> >
> > I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my mysql
> > driver loader (i can use in the code).
> >
> > the part who throw an exception is:
> >
> > String[] str = {
> > "import",
> > "--driver", "com.mysql.jdbc.Driver",
> > "--connect", "jdbc:mysql://___MY_DB___",
> > "--hadoop-home", __HOME__,
> > "--query", "Select [.....] AND $CONDITIONS",
> > "--split-by", "id",
> > "--target-dir", "hdfs://__SERVER__/__FOLDER__",
> > "--username", "foo",
> > "--password", "barou"};
> >
> > return Sqoop.runTool(str);
> >
> > Thanks,
> > Anthony
>

Re: Sqoop and OSGI

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi Anthony,
I'm glad to see your interested in Sqoop project! I have to mention that whereas the direct usage of the Sqoop classes might work, it's not officially supported API and you might hit some issues along the way.

I would strongly suggest to upgrade to latest Sqoop version as the 1.4.0 is quite old. In most cases using the --driver parameter is not needed as it will instruct Sqoop to use the Generic JDBC Driver instead of specialized MySQL connector, I would suggest to drop this parameter entirely.

Jarcec

On Wed, Jun 19, 2013 at 03:25:51PM +0900, corbacho anthony wrote:
> Hello,
> 
> I am trying to use Sqoop in an osgi pluging but it wont work and I dont
> know why.
> I keep getting the following error:
> "2013-06-19 13:53:27,556 | ERROR | qtp122715895-276 | BaseSqoopTool  | ?  ?
> |  -  -  | Got error creating database manager: java.io.IOException: No
> manager for connect string: jdbc:mysql://...."
> 
> I have include sqoop in my pom (1.4.0-cdh4b1) and I also have my mysql
> driver loader (i can use in the code).
> 
> the part who throw an exception is:
> 
> String[] str = {
> "import",
> "--driver", "com.mysql.jdbc.Driver",
> "--connect", "jdbc:mysql://___MY_DB___",
> "--hadoop-home", __HOME__,
> "--query", "Select [.....] AND $CONDITIONS",
> "--split-by", "id",
> "--target-dir", "hdfs://__SERVER__/__FOLDER__",
> "--username", "foo",
> "--password", "barou"};
> 
> return Sqoop.runTool(str);
> 
> Thanks,
> Anthony