You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Cheolsoo Park <ch...@cloudera.com> on 2012/04/17 08:12:11 UTC
Review Request: SQOOP-445 Wrong query for getListColumnsQuery in
PostgresqlManager
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4752/
-----------------------------------------------------------
Review request for Sqoop.
Summary
-------
When querying for column available in a postgres table from postgres catalog, it returns also the dropped columns from the table in object. It can be easily fixed adding the extra constraint "AND col.ATTISDROPPED = 'f'" at the end of the query which sort out the dropped columns.
This addresses bug SQOOP-445.
https://issues.apache.org/jira/browse/SQOOP-445
Diffs
-----
/src/java/org/apache/sqoop/manager/PostgresqlManager.java 1326931
Diff: https://reviews.apache.org/r/4752/diff
Testing
-------
ant test, ant test -Dthirdparty=true, ant checkstyle.
In addition, I manually tested the constraint "col.ATTISDROPPED = 'f'" as follows:
sqooptest=# create table foo (i integer);
CREATE TABLE
sqooptest=# alter table foo drop column i;
ALTER TABLE
<without col.ATTISDROPPED = 'f'>
sqooptest=# SELECT col.ATTNAME FROM PG_CATALOG.PG_NAMESPACE sch, PG_CATALOG.PG_CLASS tab, PG_CATALOG.PG_ATTRIBUTE col WHERE sch.OID = tab.RELNAMESPACE AND tab.OID = col.ATTRELID AND sch.NSPNAME = (SELECT CURRENT_SCHEMA()) AND tab.RELNAME = 'foo' AND col.ATTNUM >= 1;
attname
------------------------------
........pg.dropped.1........
(1 row)
<with col.ATTISDROPPED = 'f'>
sqooptest=# SELECT col.ATTNAME FROM PG_CATALOG.PG_NAMESPACE sch, PG_CATALOG.PG_CLASS tab, PG_CATALOG.PG_ATTRIBUTE col WHERE sch.OID = tab.RELNAMESPACE AND tab.OID = col.ATTRELID AND sch.NSPNAME = (SELECT CURRENT_SCHEMA()) AND tab.RELNAME = 'foo' AND col.ATTNUM >= 1 AND col.ATTISDROPPED = 'f';
attname
---------
(0 rows)
Thanks,
Cheolsoo
Re: Review Request: SQOOP-445 Wrong query for getListColumnsQuery in
PostgresqlManager
Posted by Cheolsoo Park <ch...@cloudera.com>.
> On 2012-04-27 06:07:54, Bilung Lee wrote:
> > Look good to me.
> >
> > It would be even better if you could have a test case to
> > 1. create a table
> > 2. drop a column
> > 3. perform sqoop import
> > 4. would succeed with the fix and fail without
> >
Hi Bilung,
Thanks for reviewing the patch. Yes, I can add a test for this too. I will open a subtask to add a test.
- Cheolsoo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4752/#review7294
-----------------------------------------------------------
On 2012-04-17 06:12:11, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4752/
> -----------------------------------------------------------
>
> (Updated 2012-04-17 06:12:11)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> When querying for column available in a postgres table from postgres catalog, it returns also the dropped columns from the table in object. It can be easily fixed adding the extra constraint "AND col.ATTISDROPPED = 'f'" at the end of the query which sort out the dropped columns.
>
>
> This addresses bug SQOOP-445.
> https://issues.apache.org/jira/browse/SQOOP-445
>
>
> Diffs
> -----
>
> /src/java/org/apache/sqoop/manager/PostgresqlManager.java 1326931
>
> Diff: https://reviews.apache.org/r/4752/diff
>
>
> Testing
> -------
>
> ant test, ant test -Dthirdparty=true, ant checkstyle.
>
> In addition, I manually tested the constraint "col.ATTISDROPPED = 'f'" as follows:
>
>
> sqooptest=# create table foo (i integer);
> CREATE TABLE
>
> sqooptest=# alter table foo drop column i;
> ALTER TABLE
>
>
> <without col.ATTISDROPPED = 'f'>
>
> sqooptest=# SELECT col.ATTNAME FROM PG_CATALOG.PG_NAMESPACE sch, PG_CATALOG.PG_CLASS tab, PG_CATALOG.PG_ATTRIBUTE col WHERE sch.OID = tab.RELNAMESPACE AND tab.OID = col.ATTRELID AND sch.NSPNAME = (SELECT CURRENT_SCHEMA()) AND tab.RELNAME = 'foo' AND col.ATTNUM >= 1;
> attname
> ------------------------------
> ........pg.dropped.1........
> (1 row)
>
>
> <with col.ATTISDROPPED = 'f'>
>
> sqooptest=# SELECT col.ATTNAME FROM PG_CATALOG.PG_NAMESPACE sch, PG_CATALOG.PG_CLASS tab, PG_CATALOG.PG_ATTRIBUTE col WHERE sch.OID = tab.RELNAMESPACE AND tab.OID = col.ATTRELID AND sch.NSPNAME = (SELECT CURRENT_SCHEMA()) AND tab.RELNAME = 'foo' AND col.ATTNUM >= 1 AND col.ATTISDROPPED = 'f';
> attname
> ---------
> (0 rows)
>
>
> Thanks,
>
> Cheolsoo
>
>
Re: Review Request: SQOOP-445 Wrong query for getListColumnsQuery in
PostgresqlManager
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4752/#review7294
-----------------------------------------------------------
Ship it!
Look good to me.
It would be even better if you could have a test case to
1. create a table
2. drop a column
3. perform sqoop import
4. would succeed with the fix and fail without
- Bilung
On 2012-04-17 06:12:11, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4752/
> -----------------------------------------------------------
>
> (Updated 2012-04-17 06:12:11)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> When querying for column available in a postgres table from postgres catalog, it returns also the dropped columns from the table in object. It can be easily fixed adding the extra constraint "AND col.ATTISDROPPED = 'f'" at the end of the query which sort out the dropped columns.
>
>
> This addresses bug SQOOP-445.
> https://issues.apache.org/jira/browse/SQOOP-445
>
>
> Diffs
> -----
>
> /src/java/org/apache/sqoop/manager/PostgresqlManager.java 1326931
>
> Diff: https://reviews.apache.org/r/4752/diff
>
>
> Testing
> -------
>
> ant test, ant test -Dthirdparty=true, ant checkstyle.
>
> In addition, I manually tested the constraint "col.ATTISDROPPED = 'f'" as follows:
>
>
> sqooptest=# create table foo (i integer);
> CREATE TABLE
>
> sqooptest=# alter table foo drop column i;
> ALTER TABLE
>
>
> <without col.ATTISDROPPED = 'f'>
>
> sqooptest=# SELECT col.ATTNAME FROM PG_CATALOG.PG_NAMESPACE sch, PG_CATALOG.PG_CLASS tab, PG_CATALOG.PG_ATTRIBUTE col WHERE sch.OID = tab.RELNAMESPACE AND tab.OID = col.ATTRELID AND sch.NSPNAME = (SELECT CURRENT_SCHEMA()) AND tab.RELNAME = 'foo' AND col.ATTNUM >= 1;
> attname
> ------------------------------
> ........pg.dropped.1........
> (1 row)
>
>
> <with col.ATTISDROPPED = 'f'>
>
> sqooptest=# SELECT col.ATTNAME FROM PG_CATALOG.PG_NAMESPACE sch, PG_CATALOG.PG_CLASS tab, PG_CATALOG.PG_ATTRIBUTE col WHERE sch.OID = tab.RELNAMESPACE AND tab.OID = col.ATTRELID AND sch.NSPNAME = (SELECT CURRENT_SCHEMA()) AND tab.RELNAME = 'foo' AND col.ATTNUM >= 1 AND col.ATTISDROPPED = 'f';
> attname
> ---------
> (0 rows)
>
>
> Thanks,
>
> Cheolsoo
>
>