You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Colin Ma <ju...@intel.com> on 2016/02/15 03:53:02 UTC
Review Request 43570: SQOOP-2838: Sqoop2: Add new schema object for
the Clob
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43570/
-----------------------------------------------------------
Review request for Sqoop and Colin Ma.
Bugs: SQOOP-2838
https://issues.apache.org/jira/browse/SQOOP-2838
Repository: sqoop-sqoop2
Description
-------
When extract the data from database, the following code segment will be used to transform {{Clob}} into {{String}}:
{code}
Clob clob = resultSet.getClob(i + 1);
array[i] = clob.getSubString(1, (int)clob.length());
{code}
Add new schema object for the {{Clob}} to deal with this specific situation.
Diffs
-----
common-test/src/main/java/org/apache/sqoop/common/test/asserts/ProviderAsserts.java e9cea2d
common-test/src/main/java/org/apache/sqoop/common/test/db/DatabaseProvider.java f3efa92
common-test/src/main/java/org/apache/sqoop/common/test/db/types/DerbyTypeList.java fc02b83
common/src/main/java/org/apache/sqoop/json/util/SchemaSerialization.java ee385c0
common/src/main/java/org/apache/sqoop/schema/type/Clob.java PRE-CREATION
common/src/main/java/org/apache/sqoop/schema/type/ColumnType.java ac98ee8
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExtractor.java 41af177
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/util/SqlTypesUtils.java f8f9f0d
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 9baa743
Diff: https://reviews.apache.org/r/43570/diff/
Testing
-------
Thanks,
Colin Ma
Re: Review Request 43570: SQOOP-2838: Sqoop2: Add new schema object
for the Clob
Posted by Colin Ma <ju...@intel.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43570/
-----------------------------------------------------------
(Updated Feb. 17, 2016, 2:42 a.m.)
Review request for Sqoop and Colin Ma.
Bugs: SQOOP-2838
https://issues.apache.org/jira/browse/SQOOP-2838
Repository: sqoop-sqoop2
Description
-------
When extract the data from database, the following code segment will be used to transform {{Clob}} into {{String}}:
{code}
Clob clob = resultSet.getClob(i + 1);
array[i] = clob.getSubString(1, (int)clob.length());
{code}
Add new schema object for the {{Clob}} to deal with this specific situation.
Diffs (updated)
-----
common-test/src/main/java/org/apache/sqoop/common/test/asserts/ProviderAsserts.java e9cea2d
common-test/src/main/java/org/apache/sqoop/common/test/db/DatabaseProvider.java f3efa92
common-test/src/main/java/org/apache/sqoop/common/test/db/types/DerbyTypeList.java fc02b83
common/src/main/java/org/apache/sqoop/json/util/SchemaSerialization.java ee385c0
common/src/main/java/org/apache/sqoop/schema/type/Clob.java PRE-CREATION
common/src/main/java/org/apache/sqoop/schema/type/ColumnType.java ac98ee8
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExtractor.java 41af177
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/util/SqlTypesUtils.java f8f9f0d
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 9baa743
Diff: https://reviews.apache.org/r/43570/diff/
Testing
-------
Thanks,
Colin Ma
Re: Review Request 43570: SQOOP-2838: Sqoop2: Add new schema object
for the Clob
Posted by Colin Ma <ju...@intel.com>.
> On Feb. 16, 2016, 7:11 p.m., Abraham Fine wrote:
> > common/src/main/java/org/apache/sqoop/json/util/SchemaSerialization.java, line 127
> > <https://reviews.apache.org/r/43570/diff/1/?file=1241363#file1241363line127>
> >
> > do we need to check for a clob here?
Thanks for reminder.
The check here `switch (column.getType())` is used for extract additional information of the column definition, for example, Precision for Decimal, FRACTION for Time, etc.
Currently, Clob/Blob only extract the column name, no additional information is needed.
To be corresponding with other column extract, I think it's better to add Clob/Blob in `switch` clause without any check.
I'll update the patch for it.
- Colin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43570/#review119343
-----------------------------------------------------------
On Feb. 15, 2016, 2:53 a.m., Colin Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43570/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2016, 2:53 a.m.)
>
>
> Review request for Sqoop and Colin Ma.
>
>
> Bugs: SQOOP-2838
> https://issues.apache.org/jira/browse/SQOOP-2838
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> When extract the data from database, the following code segment will be used to transform {{Clob}} into {{String}}:
>
> {code}
> Clob clob = resultSet.getClob(i + 1);
> array[i] = clob.getSubString(1, (int)clob.length());
> {code}
>
> Add new schema object for the {{Clob}} to deal with this specific situation.
>
>
> Diffs
> -----
>
> common-test/src/main/java/org/apache/sqoop/common/test/asserts/ProviderAsserts.java e9cea2d
> common-test/src/main/java/org/apache/sqoop/common/test/db/DatabaseProvider.java f3efa92
> common-test/src/main/java/org/apache/sqoop/common/test/db/types/DerbyTypeList.java fc02b83
> common/src/main/java/org/apache/sqoop/json/util/SchemaSerialization.java ee385c0
> common/src/main/java/org/apache/sqoop/schema/type/Clob.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/schema/type/ColumnType.java ac98ee8
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExtractor.java 41af177
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/util/SqlTypesUtils.java f8f9f0d
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 9baa743
>
> Diff: https://reviews.apache.org/r/43570/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Colin Ma
>
>
Re: Review Request 43570: SQOOP-2838: Sqoop2: Add new schema object
for the Clob
Posted by Colin Ma <ju...@intel.com>.
On Feb. 16, 2016, 7:11 p.m., Colin Ma wrote:
> > do any changes need to be made to the avrointermediatedataformat?
I think Blob/Clob impact GenericJdbcExtractor only, and avrointermediatedataformat is used in HdfsExtractor. No changed should be made to it.
- Colin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43570/#review119343
-----------------------------------------------------------
On Feb. 15, 2016, 2:53 a.m., Colin Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43570/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2016, 2:53 a.m.)
>
>
> Review request for Sqoop and Colin Ma.
>
>
> Bugs: SQOOP-2838
> https://issues.apache.org/jira/browse/SQOOP-2838
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> When extract the data from database, the following code segment will be used to transform {{Clob}} into {{String}}:
>
> {code}
> Clob clob = resultSet.getClob(i + 1);
> array[i] = clob.getSubString(1, (int)clob.length());
> {code}
>
> Add new schema object for the {{Clob}} to deal with this specific situation.
>
>
> Diffs
> -----
>
> common-test/src/main/java/org/apache/sqoop/common/test/asserts/ProviderAsserts.java e9cea2d
> common-test/src/main/java/org/apache/sqoop/common/test/db/DatabaseProvider.java f3efa92
> common-test/src/main/java/org/apache/sqoop/common/test/db/types/DerbyTypeList.java fc02b83
> common/src/main/java/org/apache/sqoop/json/util/SchemaSerialization.java ee385c0
> common/src/main/java/org/apache/sqoop/schema/type/Clob.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/schema/type/ColumnType.java ac98ee8
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExtractor.java 41af177
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/util/SqlTypesUtils.java f8f9f0d
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 9baa743
>
> Diff: https://reviews.apache.org/r/43570/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Colin Ma
>
>
Re: Review Request 43570: SQOOP-2838: Sqoop2: Add new schema object
for the Clob
Posted by Abraham Fine <ab...@abrahamfine.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43570/#review119343
-----------------------------------------------------------
common/src/main/java/org/apache/sqoop/json/util/SchemaSerialization.java (line 127)
<https://reviews.apache.org/r/43570/#comment180655>
do we need to check for a clob here?
do any changes need to be made to the avrointermediatedataformat?
- Abraham Fine
On Feb. 15, 2016, 2:53 a.m., Colin Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43570/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2016, 2:53 a.m.)
>
>
> Review request for Sqoop and Colin Ma.
>
>
> Bugs: SQOOP-2838
> https://issues.apache.org/jira/browse/SQOOP-2838
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> When extract the data from database, the following code segment will be used to transform {{Clob}} into {{String}}:
>
> {code}
> Clob clob = resultSet.getClob(i + 1);
> array[i] = clob.getSubString(1, (int)clob.length());
> {code}
>
> Add new schema object for the {{Clob}} to deal with this specific situation.
>
>
> Diffs
> -----
>
> common-test/src/main/java/org/apache/sqoop/common/test/asserts/ProviderAsserts.java e9cea2d
> common-test/src/main/java/org/apache/sqoop/common/test/db/DatabaseProvider.java f3efa92
> common-test/src/main/java/org/apache/sqoop/common/test/db/types/DerbyTypeList.java fc02b83
> common/src/main/java/org/apache/sqoop/json/util/SchemaSerialization.java ee385c0
> common/src/main/java/org/apache/sqoop/schema/type/Clob.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/schema/type/ColumnType.java ac98ee8
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExtractor.java 41af177
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/util/SqlTypesUtils.java f8f9f0d
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 9baa743
>
> Diff: https://reviews.apache.org/r/43570/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Colin Ma
>
>