Sqoop Incremental pull

[yogesh kumar <yo...@gmail.com>]

Hello all,

I have done sqoop import for a particluar table first time say table
Employee..

sqoop import -libjars .....
--query "select empno, name, date, loc from table Employee where
\$CONDITIONS ..  "
*--split-by empno*
--fields-terminated-by ','
.
.
.
.

I have created an external table on hive,

*Now I want to pull data on daily basis by using incremental pull.  can I
spsecy fy the diffrent column for --split-by*

like

 sqoop import -libjars .....
--query "select empno, name, date, loc from table Employee where
\$CONDITIONS ..  "
--check-column date
--incremental append
--last-value 2013-05-01
*--split-by date*
--split-by empno


Can I change the column for *split by in incremental sqoop*, if not then
how to do it.

Pls suggest

Thanks & Regards
Yogesh

Re: Sqoop Incremental pull

[Abraham Elmahrek <ab...@cloudera.com>]

Yogesh,

I haven't verified this, but how you split your data is less relevant in
the incremental mode. What's more important is what column you're checking
"--check-column", the incremental mode you choose "--incrmental", and the
relevant last value "--last-value". So this should work!

-Abe


On Mon, Dec 30, 2013 at 6:33 AM, yogesh kumar <yo...@gmail.com> wrote:

> Hello all,
>
> I have done sqoop import for a particluar table first time say table
> Employee..
>
> sqoop import -libjars .....
> --query "select empno, name, date, loc from table Employee where
> \$CONDITIONS ..  "
> *--split-by empno*
> --fields-terminated-by ','
> .
> .
> .
> .
>
> I have created an external table on hive,
>
> *Now I want to pull data on daily basis by using incremental pull.  can I
> spsecy fy the diffrent column for --split-by*
>
> like
>
>  sqoop import -libjars .....
> --query "select empno, name, date, loc from table Employee where
> \$CONDITIONS ..  "
> --check-column date
> --incremental append
> --last-value 2013-05-01
> *--split-by date*
> --split-by empno
>
>
> Can I change the column for *split by in incremental sqoop*, if not then
> how to do it.
>
> Pls suggest
>
> Thanks & Regards
> Yogesh
>
>