You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Felix GV <fe...@mate1inc.com> on 2013/03/07 19:14:11 UTC
Sqoop 2 maturity?
Hello :)
I've played around with Sqoop 1 a fair while ago (possibly before Apache
incubation) around the time of CDH3u3. There were a few minor kinks at the
time but overall it seemed to be already pretty good and stable at the
time, so I'm assuming Sqoop 1.4.2 can only be a lot better.
I didn't end up needing Sqopp for my use cases after I experimented with it
at the time, but I'm now revisiting it and I just found out Sqoop 2 is in
the works.
I'm now using CDH 4.1.2 and I'm planning to upgrade to the latest (CDH 4.2)
soon, which includes the first release of Sqoop 2 (1.99.1).
I've read the architecture design articles and I think we could benefit
from some of the new capabilities. I've also scoured the last few months of
the (user) mailing list archive but haven't found much discussion regarding
Sqoop 2, besides the
announcement<http://mail-archives.apache.org/mod_mbox/sqoop-user/201212.mbox/%3C20121226063212.GB28520%40jarcec-thinkpad%3E>and
a post saying the Web
UI isn't included<http://mail-archives.apache.org/mod_mbox/sqoop-user/201211.mbox/%3C20121128161809.GE6589%40jarcec-thinkpad%3E>
.
I don't mind not having the Web UI yet, but I want to automate a couple of
Sqoop-based ETL scripts, and if there are any differences between the way
Sqoop 1 and 2 scripts are specified or used, then I'd like to start doing
them the version 2 way, so that my infrastructure is more future proof.
This, of course, is assuming the non-web UI capabilities of Sqoop 2 are
mature enough for general usage.
Can anyone comment on Sqoop 2's maturity?
Should I subscribe to the dev mailing list instead if I'm interested in
Sqoop 2, or is the user list the right place for general (non-dev) related
Sqoop 2 questions?
Thanks a lot guys (and gals?!) !!
--
Felix
Re: Sqoop 2 maturity?
Posted by Felix GV <fe...@mate1inc.com>.
Thanks for your quick reply Arvind,
I'll use Sqoop 1 for now then, because automatic Hive schema definition is
definitely a major boon to my use case.
As for which features I would need in order to move to Sqoop 2, I can't
really say for sure. But since I'll start using Sqoop 1, then I guess I'll
get a better idea as I go along...
Thanks again,
--
Felix
On Fri, Mar 8, 2013 at 1:58 PM, Arvind Prabhakar <ar...@apache.org> wrote:
> Hi Felix,
>
> Thanks for using Sqoop and providing your feedback. You are right that
> Sqoop 1.4.2 is the same Sqoop you tried out before, and is much more stable
> and functional than before. You are welcome to use and we will try our best
> to fix any issues with it that you may run into.
>
> Regarding Sqoop 2, it just made its debut and you are welcome to try it.
> But as its release version (1.99.1) indicates - this is not yet at the 2.0
> level and as such is missing features. One of these missing features at
> this time is Hive integration.
>
> Going forward, we will try to prioritize these missing features based on
> what the community's prioritization, so if you could send in a list of
> things you would like to be addressed, that will be great.
>
> Regards,
> Arvind Prabhakar
>
>
> On Fri, Mar 8, 2013 at 9:48 AM, Felix GV <fe...@mate1inc.com> wrote:
>
>> For example, looking at the options I have when creating a job in the
>> sqoop2 shell, it seems there is no Hive support yet. Is that correct?
>>
>> Thanks :) !
>>
>> --
>> Felix
>>
>>
>> On Thu, Mar 7, 2013 at 1:14 PM, Felix GV <fe...@mate1inc.com> wrote:
>>
>>> Hello :)
>>>
>>> I've played around with Sqoop 1 a fair while ago (possibly before Apache
>>> incubation) around the time of CDH3u3. There were a few minor kinks at the
>>> time but overall it seemed to be already pretty good and stable at the
>>> time, so I'm assuming Sqoop 1.4.2 can only be a lot better.
>>>
>>> I didn't end up needing Sqopp for my use cases after I experimented with
>>> it at the time, but I'm now revisiting it and I just found out Sqoop 2 is
>>> in the works.
>>>
>>> I'm now using CDH 4.1.2 and I'm planning to upgrade to the latest (CDH
>>> 4.2) soon, which includes the first release of Sqoop 2 (1.99.1).
>>>
>>> I've read the architecture design articles and I think we could benefit
>>> from some of the new capabilities. I've also scoured the last few months of
>>> the (user) mailing list archive but haven't found much discussion regarding
>>> Sqoop 2, besides the announcement<http://mail-archives.apache.org/mod_mbox/sqoop-user/201212.mbox/%3C20121226063212.GB28520%40jarcec-thinkpad%3E>and a post saying the Web
>>> UI isn't included<http://mail-archives.apache.org/mod_mbox/sqoop-user/201211.mbox/%3C20121128161809.GE6589%40jarcec-thinkpad%3E>
>>> .
>>>
>>> I don't mind not having the Web UI yet, but I want to automate a couple
>>> of Sqoop-based ETL scripts, and if there are any differences between the
>>> way Sqoop 1 and 2 scripts are specified or used, then I'd like to start
>>> doing them the version 2 way, so that my infrastructure is more future
>>> proof.
>>>
>>> This, of course, is assuming the non-web UI capabilities of Sqoop 2 are
>>> mature enough for general usage.
>>>
>>> Can anyone comment on Sqoop 2's maturity?
>>>
>>> Should I subscribe to the dev mailing list instead if I'm interested in
>>> Sqoop 2, or is the user list the right place for general (non-dev) related
>>> Sqoop 2 questions?
>>>
>>> Thanks a lot guys (and gals?!) !!
>>>
>>> --
>>> Felix
>>>
>>
>>
>
Re: Sqoop 2 maturity?
Posted by Arvind Prabhakar <ar...@apache.org>.
Hi Felix,
Thanks for using Sqoop and providing your feedback. You are right that
Sqoop 1.4.2 is the same Sqoop you tried out before, and is much more stable
and functional than before. You are welcome to use and we will try our best
to fix any issues with it that you may run into.
Regarding Sqoop 2, it just made its debut and you are welcome to try it.
But as its release version (1.99.1) indicates - this is not yet at the 2.0
level and as such is missing features. One of these missing features at
this time is Hive integration.
Going forward, we will try to prioritize these missing features based on
what the community's prioritization, so if you could send in a list of
things you would like to be addressed, that will be great.
Regards,
Arvind Prabhakar
On Fri, Mar 8, 2013 at 9:48 AM, Felix GV <fe...@mate1inc.com> wrote:
> For example, looking at the options I have when creating a job in the
> sqoop2 shell, it seems there is no Hive support yet. Is that correct?
>
> Thanks :) !
>
> --
> Felix
>
>
> On Thu, Mar 7, 2013 at 1:14 PM, Felix GV <fe...@mate1inc.com> wrote:
>
>> Hello :)
>>
>> I've played around with Sqoop 1 a fair while ago (possibly before Apache
>> incubation) around the time of CDH3u3. There were a few minor kinks at the
>> time but overall it seemed to be already pretty good and stable at the
>> time, so I'm assuming Sqoop 1.4.2 can only be a lot better.
>>
>> I didn't end up needing Sqopp for my use cases after I experimented with
>> it at the time, but I'm now revisiting it and I just found out Sqoop 2 is
>> in the works.
>>
>> I'm now using CDH 4.1.2 and I'm planning to upgrade to the latest (CDH
>> 4.2) soon, which includes the first release of Sqoop 2 (1.99.1).
>>
>> I've read the architecture design articles and I think we could benefit
>> from some of the new capabilities. I've also scoured the last few months of
>> the (user) mailing list archive but haven't found much discussion regarding
>> Sqoop 2, besides the announcement<http://mail-archives.apache.org/mod_mbox/sqoop-user/201212.mbox/%3C20121226063212.GB28520%40jarcec-thinkpad%3E>and a post saying the Web
>> UI isn't included<http://mail-archives.apache.org/mod_mbox/sqoop-user/201211.mbox/%3C20121128161809.GE6589%40jarcec-thinkpad%3E>
>> .
>>
>> I don't mind not having the Web UI yet, but I want to automate a couple
>> of Sqoop-based ETL scripts, and if there are any differences between the
>> way Sqoop 1 and 2 scripts are specified or used, then I'd like to start
>> doing them the version 2 way, so that my infrastructure is more future
>> proof.
>>
>> This, of course, is assuming the non-web UI capabilities of Sqoop 2 are
>> mature enough for general usage.
>>
>> Can anyone comment on Sqoop 2's maturity?
>>
>> Should I subscribe to the dev mailing list instead if I'm interested in
>> Sqoop 2, or is the user list the right place for general (non-dev) related
>> Sqoop 2 questions?
>>
>> Thanks a lot guys (and gals?!) !!
>>
>> --
>> Felix
>>
>
>
Re: Sqoop 2 maturity?
Posted by Felix GV <fe...@mate1inc.com>.
For example, looking at the options I have when creating a job in the
sqoop2 shell, it seems there is no Hive support yet. Is that correct?
Thanks :) !
--
Felix
On Thu, Mar 7, 2013 at 1:14 PM, Felix GV <fe...@mate1inc.com> wrote:
> Hello :)
>
> I've played around with Sqoop 1 a fair while ago (possibly before Apache
> incubation) around the time of CDH3u3. There were a few minor kinks at the
> time but overall it seemed to be already pretty good and stable at the
> time, so I'm assuming Sqoop 1.4.2 can only be a lot better.
>
> I didn't end up needing Sqopp for my use cases after I experimented with
> it at the time, but I'm now revisiting it and I just found out Sqoop 2 is
> in the works.
>
> I'm now using CDH 4.1.2 and I'm planning to upgrade to the latest (CDH
> 4.2) soon, which includes the first release of Sqoop 2 (1.99.1).
>
> I've read the architecture design articles and I think we could benefit
> from some of the new capabilities. I've also scoured the last few months of
> the (user) mailing list archive but haven't found much discussion regarding
> Sqoop 2, besides the announcement<http://mail-archives.apache.org/mod_mbox/sqoop-user/201212.mbox/%3C20121226063212.GB28520%40jarcec-thinkpad%3E>and a post saying the Web
> UI isn't included<http://mail-archives.apache.org/mod_mbox/sqoop-user/201211.mbox/%3C20121128161809.GE6589%40jarcec-thinkpad%3E>
> .
>
> I don't mind not having the Web UI yet, but I want to automate a couple of
> Sqoop-based ETL scripts, and if there are any differences between the way
> Sqoop 1 and 2 scripts are specified or used, then I'd like to start doing
> them the version 2 way, so that my infrastructure is more future proof.
>
> This, of course, is assuming the non-web UI capabilities of Sqoop 2 are
> mature enough for general usage.
>
> Can anyone comment on Sqoop 2's maturity?
>
> Should I subscribe to the dev mailing list instead if I'm interested in
> Sqoop 2, or is the user list the right place for general (non-dev) related
> Sqoop 2 questions?
>
> Thanks a lot guys (and gals?!) !!
>
> --
> Felix
>