You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by 孙如镇 <su...@gmail.com> on 2014/08/25 05:07:18 UTC
How to add new connector to sqoop2
Hi guys,
I am new to Sqoop, and I am developing a connector for Couchbase
database.Now I am wondering how to add my connector jar to Sqoop2 and what
are the steps to make my connector run.I am using Sqoop1.99.3.
Re: How to add new connector to sqoop2
Posted by 孙如镇 <su...@gmail.com>.
Hi Jarcec,
Thanks for yor tips. I have cost some time to read sqoop2's code, and
now I known how to integrate my connector into sqoop2.
Best regards,
Chen Sun
2014-08-25 13:14 GMT+08:00 Jarek Jarcec Cecho <ja...@apache.org>:
> Hi sir,
> did you seen our Connector developer guide for Sqoop 2 [1]? We are
> currently very extensively refactoring the connector interface via
> SQOOP-1367, so please note that you will have to do some changes for the
> next release.
>
> Jarcec
>
> Links:
> 1: http://sqoop.apache.org/docs/1.99.3/ConnectorDevelopment.html
>
> On Aug 25, 2014, at 5:08 AM, 孙如镇 <su...@gmail.com> wrote:
>
> > Hi guys,
> > I am new to Sqoop, and I am developing a connector for Couchbase
> > database.Now I am wondering how to add my connector jar to Sqoop2 and
> what
> > are the steps to make my connector run.I am using Sqoop1.99.3.
>
>
Re: How to add new connector to sqoop2
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi sir,
did you seen our Connector developer guide for Sqoop 2 [1]? We are currently very extensively refactoring the connector interface via SQOOP-1367, so please note that you will have to do some changes for the next release.
Jarcec
Links:
1: http://sqoop.apache.org/docs/1.99.3/ConnectorDevelopment.html
On Aug 25, 2014, at 5:08 AM, 孙如镇 <su...@gmail.com> wrote:
> Hi guys,
> I am new to Sqoop, and I am developing a connector for Couchbase
> database.Now I am wondering how to add my connector jar to Sqoop2 and what
> are the steps to make my connector run.I am using Sqoop1.99.3.
Fwd: How to add new connector to sqoop2
Posted by 孙如镇 <su...@gmail.com>.
Hi guys,
I am new to Sqoop, and I am developing a connector for Couchbase
database.Now I am wondering how to add my connector jar to Sqoop2 and what
are the steps to make my connector run.I am using Sqoop1.99.3.
Re: How to add new connector to sqoop2
Posted by pratik khadloya <ti...@gmail.com>.
Thanks Gwen. Is a user group presentation something we upload on the groups
as a shared document?
Thanks for the corrections regarding #1.
About #2, am not sure why sqoop needed those jars on hdfs for hadoop 2.4.1.
That did not happen when i tested with hadoop 0.20 CDH 4.4 it did not ask
me to pre ship to hdfs.
~Pratik
On Mon, Aug 25, 2014 at 9:31 AM, Gwen Shapira <gs...@cloudera.com> wrote:
> Hi Pratik,
>
> Very nice evernote :) Maybe you'll want to turn it into a user-group
> presentation at some point.
>
> Two comments:
> 1. The "awesome info" at the end is relevant for Sqoop1 but its the
> wrong syntax for sqoop2
> 2. Having to copy the jars to HDFS was not part of the plan. I think
> we need to troubleshoot why you had to do that. We expected Sqoop to
> distribute the jars it needs to all the nodes on its own.
>
> Gwen
>
> On Mon, Aug 25, 2014 at 7:49 AM, pratik khadloya <ti...@gmail.com>
> wrote:
> > I recently did that. Had noted all the steps in my evernote.
> > Sharing the link
> >
> https://www.evernote.com/shard/s141/sh/24ffd200-6d5e-4203-abc7-a4c80379e558/226076fdb2ed67232a55bfe761e00659
> > Let me know if you face any difficulties.
> >
> > ~Pratik
> >
> >
> > On Sun, Aug 24, 2014 at 8:07 PM, 孙如镇 <su...@gmail.com> wrote:
> >>
> >> Hi guys,
> >> I am new to Sqoop, and I am developing a connector for Couchbase
> >> database.Now I am wondering how to add my connector jar to Sqoop2 and
> what
> >> are the steps to make my connector run.I am using Sqoop1.99.3.
> >
> >
>
Re: How to add new connector to sqoop2
Posted by Gwen Shapira <gs...@cloudera.com>.
Hi Pratik,
Very nice evernote :) Maybe you'll want to turn it into a user-group
presentation at some point.
Two comments:
1. The "awesome info" at the end is relevant for Sqoop1 but its the
wrong syntax for sqoop2
2. Having to copy the jars to HDFS was not part of the plan. I think
we need to troubleshoot why you had to do that. We expected Sqoop to
distribute the jars it needs to all the nodes on its own.
Gwen
On Mon, Aug 25, 2014 at 7:49 AM, pratik khadloya <ti...@gmail.com> wrote:
> I recently did that. Had noted all the steps in my evernote.
> Sharing the link
> https://www.evernote.com/shard/s141/sh/24ffd200-6d5e-4203-abc7-a4c80379e558/226076fdb2ed67232a55bfe761e00659
> Let me know if you face any difficulties.
>
> ~Pratik
>
>
> On Sun, Aug 24, 2014 at 8:07 PM, 孙如镇 <su...@gmail.com> wrote:
>>
>> Hi guys,
>> I am new to Sqoop, and I am developing a connector for Couchbase
>> database.Now I am wondering how to add my connector jar to Sqoop2 and what
>> are the steps to make my connector run.I am using Sqoop1.99.3.
>
>
Re: How to add new connector to sqoop2
Posted by zn <su...@gmail.com>.
Hi Pratik,
Thanks for your reply. But what I mean is how to integrate my own connector in to sqoop2. And now I have known how to do this by reading sqoop’s source code.Thanks all the same.
在 2014年8月25日，下午10:49，pratik khadloya <ti...@gmail.com> 写道：
> I recently did that. Had noted all the steps in my evernote.
> Sharing the link https://www.evernote.com/shard/s141/sh/24ffd200-6d5e-4203-abc7-a4c80379e558/226076fdb2ed67232a55bfe761e00659
> Let me know if you face any difficulties.
>
> ~Pratik
>
>
> On Sun, Aug 24, 2014 at 8:07 PM, 孙如镇 <su...@gmail.com> wrote:
> Hi guys,
> I am new to Sqoop, and I am developing a connector for Couchbase database.Now I am wondering how to add my connector jar to Sqoop2 and what are the steps to make my connector run.I am using Sqoop1.99.3.
>
Re: How to add new connector to sqoop2
Posted by pratik khadloya <ti...@gmail.com>.
I recently did that. Had noted all the steps in my evernote.
Sharing the link
https://www.evernote.com/shard/s141/sh/24ffd200-6d5e-4203-abc7-a4c80379e558/226076fdb2ed67232a55bfe761e00659
Let me know if you face any difficulties.
~Pratik
On Sun, Aug 24, 2014 at 8:07 PM, 孙如镇 <su...@gmail.com> wrote:
> Hi guys,
> I am new to Sqoop, and I am developing a connector for Couchbase
> database.Now I am wondering how to add my connector jar to Sqoop2 and what
> are the steps to make my connector run.I am using Sqoop1.99.3.
>