You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Abraham Elmahrek <ab...@cloudera.com> on 2014/09/16 01:05:33 UTC
Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client support
for supported directions
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-1454
https://issues.apache.org/jira/browse/SQOOP-1454
Repository: sqoop-SQOOP-1367
Description
-------
commit f7e49dcd70234398634be1ec9834d901b7e02992
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Mon Sep 15 14:32:40 2014 -0700
SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
:100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
:100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
:100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
:100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
:100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
Diffs
-----
common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
shell/src/main/resources/shell-resource.properties 73a19e8
Diff: https://reviews.apache.org/r/25672/diff/
Testing
-------
mvn test and manual verification:
+----+------------------------+----------------+------------------------------------------------------+----------------------+
| Id | Name | Version | Class | Supported Directions |
+----+------------------------+----------------+------------------------------------------------------+----------------------+
| 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
| 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
+----+------------------------+----------------+------------------------------------------------------+----------------------+
sqoop:000> show connector --all
2 connector(s) to show:
Connector with id 1:
Name: hdfs-connector
Class: org.apache.sqoop.connector.hdfs.HdfsConnector
Version: 2.0.0-SNAPSHOT
Supported Directions FROM/TO
Connection form 1:
Name: connection
Label: Connection configuration
Help: You must supply the information requested in order to create a connection object.
Input 1:
Name: connection.dummy
Label: Dummy parameter needed to get HDFS connector to register
Help: You can write anything here. Doesn't matter.
Type: STRING
Sensitive: false
Size: 255
FROM Job form 1:
Name: input
Label: Input configuration
Help: Specifies information required to get data from Hadoop ecosystem
Input 1:
Name: input.inputDirectory
Label: Input directory
Help: Directory that should be exported
Type: STRING
Sensitive: false
Size: 255
TO Job form 1:
Name: output
Label: Output configuration
Help: You must supply the information requested in order to get information where you want to store your data.
Input 1:
Name: output.outputFormat
Label: Output format
Help: Format in which data should be serialized
Type: ENUM
Sensitive: false
Possible values: TEXT_FILE,SEQUENCE_FILE
Input 2:
Name: output.compression
Label: Compression format
Help: Compression that should be used for the data
Type: ENUM
Sensitive: false
Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
Input 3:
Name: output.customCompression
Label: Custom compression format
Help: Full class name of the custom compression
Type: STRING
Sensitive: false
Size: 255
Input 4:
Name: output.outputDirectory
Label: Output directory
Help: Output directory for final data
Type: STRING
Sensitive: false
Size: 255
Connector with id 2:
Name: generic-jdbc-connector
Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
Version: 2.0.0-SNAPSHOT
Supported Directions FROM/TO
Connection form 1:
Name: connection
Label: Connection configuration
Help: You must supply the information requested in order to create a connection object.
Input 1:
Name: connection.jdbcDriver
Label: JDBC Driver Class
Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
Type: STRING
Sensitive: false
Size: 128
Input 2:
Name: connection.connectionString
Label: JDBC Connection String
Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
Type: STRING
Sensitive: false
Size: 128
Input 3:
Name: connection.username
Label: Username
Help: Enter the username to be used for connecting to the database.
Type: STRING
Sensitive: false
Size: 40
Input 4:
Name: connection.password
Label: Password
Help: Enter the password to be used for connecting to the database.
Type: STRING
Sensitive: true
Size: 40
Input 5:
Name: connection.jdbcProperties
Label: JDBC Connection Properties
Help: Enter any JDBC properties that should be supplied during the creation of connection.
Type: MAP
Sensitive: false
FROM Job form 1:
Name: fromTable
Label: From database configuration
Help: You must supply the information requested in order to create a job object.
Input 1:
Name: fromTable.schemaName
Label: Schema name
Help: Schema name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 2:
Name: fromTable.tableName
Label: Table name
Help: Table name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 3:
Name: fromTable.sql
Label: Table SQL statement
Help: SQL statement to process data in the remote database
Type: STRING
Sensitive: false
Size: 2000
Input 4:
Name: fromTable.columns
Label: Table column names
Help: Specific columns of a table name or a table SQL
Type: STRING
Sensitive: false
Size: 50
Input 5:
Name: fromTable.partitionColumn
Label: Partition column name
Help: A specific column for data partition
Type: STRING
Sensitive: false
Size: 50
Input 6:
Name: fromTable.partitionColumnNull
Label: Nulls in partition column
Help: Whether there are null values in partition column
Type: BOOLEAN
Sensitive: false
Input 7:
Name: fromTable.boundaryQuery
Label: Boundary query
Help: The boundary query for data partition
Type: STRING
Sensitive: false
Size: 50
TO Job form 1:
Name: toTable
Label: To database configuration
Help: You must supply the information requested in order to create a job object.
Input 1:
Name: toTable.schemaName
Label: Schema name
Help: Schema name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 2:
Name: toTable.tableName
Label: Table name
Help: Table name to process data in the remote database
Type: STRING
Sensitive: false
Size: 2000
Input 3:
Name: toTable.sql
Label: Table SQL statement
Help: SQL statement to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 4:
Name: toTable.columns
Label: Table column names
Help: Specific columns of a table name or a table SQL
Type: STRING
Sensitive: false
Size: 50
Input 5:
Name: toTable.stageTableName
Label: Stage table name
Help: Name of the stage table to use
Type: STRING
Sensitive: false
Size: 2000
Input 6:
Name: toTable.clearStageTable
Label: Clear stage table
Help: Indicate if the stage table should be cleared
Type: BOOLEAN
Sensitive: false
sqoop:000>
Thanks,
Abraham Elmahrek
Re: Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client
support for supported directions
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Sept. 17, 2014, 7:36 p.m., Gwen Shapira wrote:
> > LGTM (http://www.lgtm.in/i/Yk1eskAgm)
> > Few minor comments. And a question: Shouldn't we support "show connectors -direction FROM" and "show connectors -direction TO" in shell? This can be another Jira.
Why am I hypnotized by that crab.
https://issues.apache.org/jira/browse/SQOOP-1531
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/#review53727
-----------------------------------------------------------
On Sept. 15, 2014, 11:05 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25672/
> -----------------------------------------------------------
>
> (Updated Sept. 15, 2014, 11:05 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1454
> https://issues.apache.org/jira/browse/SQOOP-1454
>
>
> Repository: sqoop-SQOOP-1367
>
>
> Description
> -------
>
> commit f7e49dcd70234398634be1ec9834d901b7e02992
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Mon Sep 15 14:32:40 2014 -0700
>
> SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
>
> :100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
> :100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
> :100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
> :100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
> :100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
> common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
> core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
> shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
> shell/src/main/resources/shell-resource.properties 73a19e8
>
> Diff: https://reviews.apache.org/r/25672/diff/
>
>
> Testing
> -------
>
> mvn test and manual verification:
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | Id | Name | Version | Class | Supported Directions |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
> | 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
>
> sqoop:000> show connector --all
> 2 connector(s) to show:
> Connector with id 1:
> Name: hdfs-connector
> Class: org.apache.sqoop.connector.hdfs.HdfsConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.dummy
> Label: Dummy parameter needed to get HDFS connector to register
> Help: You can write anything here. Doesn't matter.
> Type: STRING
> Sensitive: false
> Size: 255
> FROM Job form 1:
> Name: input
> Label: Input configuration
> Help: Specifies information required to get data from Hadoop ecosystem
> Input 1:
> Name: input.inputDirectory
> Label: Input directory
> Help: Directory that should be exported
> Type: STRING
> Sensitive: false
> Size: 255
> TO Job form 1:
> Name: output
> Label: Output configuration
> Help: You must supply the information requested in order to get information where you want to store your data.
> Input 1:
> Name: output.outputFormat
> Label: Output format
> Help: Format in which data should be serialized
> Type: ENUM
> Sensitive: false
> Possible values: TEXT_FILE,SEQUENCE_FILE
> Input 2:
> Name: output.compression
> Label: Compression format
> Help: Compression that should be used for the data
> Type: ENUM
> Sensitive: false
> Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
> Input 3:
> Name: output.customCompression
> Label: Custom compression format
> Help: Full class name of the custom compression
> Type: STRING
> Sensitive: false
> Size: 255
> Input 4:
> Name: output.outputDirectory
> Label: Output directory
> Help: Output directory for final data
> Type: STRING
> Sensitive: false
> Size: 255
> Connector with id 2:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.jdbcDriver
> Label: JDBC Driver Class
> Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 2:
> Name: connection.connectionString
> Label: JDBC Connection String
> Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 3:
> Name: connection.username
> Label: Username
> Help: Enter the username to be used for connecting to the database.
> Type: STRING
> Sensitive: false
> Size: 40
> Input 4:
> Name: connection.password
> Label: Password
> Help: Enter the password to be used for connecting to the database.
> Type: STRING
> Sensitive: true
> Size: 40
> Input 5:
> Name: connection.jdbcProperties
> Label: JDBC Connection Properties
> Help: Enter any JDBC properties that should be supplied during the creation of connection.
> Type: MAP
> Sensitive: false
> FROM Job form 1:
> Name: fromTable
> Label: From database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: fromTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: fromTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 3:
> Name: fromTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 4:
> Name: fromTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: fromTable.partitionColumn
> Label: Partition column name
> Help: A specific column for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> Input 6:
> Name: fromTable.partitionColumnNull
> Label: Nulls in partition column
> Help: Whether there are null values in partition column
> Type: BOOLEAN
> Sensitive: false
> Input 7:
> Name: fromTable.boundaryQuery
> Label: Boundary query
> Help: The boundary query for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> TO Job form 1:
> Name: toTable
> Label: To database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: toTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: toTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 3:
> Name: toTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 4:
> Name: toTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: toTable.stageTableName
> Label: Stage table name
> Help: Name of the stage table to use
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 6:
> Name: toTable.clearStageTable
> Label: Clear stage table
> Help: Indicate if the stage table should be cleared
> Type: BOOLEAN
> Sensitive: false
> sqoop:000>
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client
support for supported directions
Posted by Gwen Shapira <gs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/#review53727
-----------------------------------------------------------
LGTM (http://www.lgtm.in/i/Yk1eskAgm)
Few minor comments. And a question: Shouldn't we support "show connectors -direction FROM" and "show connectors -direction TO" in shell? This can be another Jira.
common/src/main/java/org/apache/sqoop/model/MConnector.java
<https://reviews.apache.org/r/25672/#comment93519>
We are never going to have more than two directions (at least I can't see how this would make sense...) - Maybe a Pair or an object for supported directions will work better than a list?
common/src/test/java/org/apache/sqoop/model/TestMConnector.java
<https://reviews.apache.org/r/25672/#comment93517>
Maybe mention in Jira that this patch also modifies the unrelated testClone()? Or more this to another ticket?
- Gwen Shapira
On Sept. 15, 2014, 11:05 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25672/
> -----------------------------------------------------------
>
> (Updated Sept. 15, 2014, 11:05 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1454
> https://issues.apache.org/jira/browse/SQOOP-1454
>
>
> Repository: sqoop-SQOOP-1367
>
>
> Description
> -------
>
> commit f7e49dcd70234398634be1ec9834d901b7e02992
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Mon Sep 15 14:32:40 2014 -0700
>
> SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
>
> :100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
> :100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
> :100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
> :100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
> :100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
> common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
> core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
> shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
> shell/src/main/resources/shell-resource.properties 73a19e8
>
> Diff: https://reviews.apache.org/r/25672/diff/
>
>
> Testing
> -------
>
> mvn test and manual verification:
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | Id | Name | Version | Class | Supported Directions |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
> | 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
>
> sqoop:000> show connector --all
> 2 connector(s) to show:
> Connector with id 1:
> Name: hdfs-connector
> Class: org.apache.sqoop.connector.hdfs.HdfsConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.dummy
> Label: Dummy parameter needed to get HDFS connector to register
> Help: You can write anything here. Doesn't matter.
> Type: STRING
> Sensitive: false
> Size: 255
> FROM Job form 1:
> Name: input
> Label: Input configuration
> Help: Specifies information required to get data from Hadoop ecosystem
> Input 1:
> Name: input.inputDirectory
> Label: Input directory
> Help: Directory that should be exported
> Type: STRING
> Sensitive: false
> Size: 255
> TO Job form 1:
> Name: output
> Label: Output configuration
> Help: You must supply the information requested in order to get information where you want to store your data.
> Input 1:
> Name: output.outputFormat
> Label: Output format
> Help: Format in which data should be serialized
> Type: ENUM
> Sensitive: false
> Possible values: TEXT_FILE,SEQUENCE_FILE
> Input 2:
> Name: output.compression
> Label: Compression format
> Help: Compression that should be used for the data
> Type: ENUM
> Sensitive: false
> Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
> Input 3:
> Name: output.customCompression
> Label: Custom compression format
> Help: Full class name of the custom compression
> Type: STRING
> Sensitive: false
> Size: 255
> Input 4:
> Name: output.outputDirectory
> Label: Output directory
> Help: Output directory for final data
> Type: STRING
> Sensitive: false
> Size: 255
> Connector with id 2:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.jdbcDriver
> Label: JDBC Driver Class
> Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 2:
> Name: connection.connectionString
> Label: JDBC Connection String
> Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 3:
> Name: connection.username
> Label: Username
> Help: Enter the username to be used for connecting to the database.
> Type: STRING
> Sensitive: false
> Size: 40
> Input 4:
> Name: connection.password
> Label: Password
> Help: Enter the password to be used for connecting to the database.
> Type: STRING
> Sensitive: true
> Size: 40
> Input 5:
> Name: connection.jdbcProperties
> Label: JDBC Connection Properties
> Help: Enter any JDBC properties that should be supplied during the creation of connection.
> Type: MAP
> Sensitive: false
> FROM Job form 1:
> Name: fromTable
> Label: From database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: fromTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: fromTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 3:
> Name: fromTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 4:
> Name: fromTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: fromTable.partitionColumn
> Label: Partition column name
> Help: A specific column for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> Input 6:
> Name: fromTable.partitionColumnNull
> Label: Nulls in partition column
> Help: Whether there are null values in partition column
> Type: BOOLEAN
> Sensitive: false
> Input 7:
> Name: fromTable.boundaryQuery
> Label: Boundary query
> Help: The boundary query for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> TO Job form 1:
> Name: toTable
> Label: To database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: toTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: toTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 3:
> Name: toTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 4:
> Name: toTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: toTable.stageTableName
> Label: Stage table name
> Help: Name of the stage table to use
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 6:
> Name: toTable.clearStageTable
> Label: Clear stage table
> Help: Indicate if the stage table should be cleared
> Type: BOOLEAN
> Sensitive: false
> sqoop:000>
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client
support for supported directions
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Sept. 21, 2014, 8:07 a.m., Jarek Cecho wrote:
> > common/src/main/java/org/apache/sqoop/common/SupportedDirections.java, lines 39-40
> > <https://reviews.apache.org/r/25672/diff/2/?file=693121#file693121line39>
> >
> > The enum usage seems kind of uneasy here as it requires a lot of magic around it when using. Would it be better to simply use pair of booleans here?
> >
> > Then a constructor would be also simpler as you would just do new SupportedDirections(my-cool-codition-for-from, my-cool-condition-for-to).
Great idea!
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/#review54090
-----------------------------------------------------------
On Sept. 18, 2014, 1:42 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25672/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 1:42 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1454
> https://issues.apache.org/jira/browse/SQOOP-1454
>
>
> Repository: sqoop-SQOOP-1367
>
>
> Description
> -------
>
> commit f7e49dcd70234398634be1ec9834d901b7e02992
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Mon Sep 15 14:32:40 2014 -0700
>
> SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
>
> :100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
> :100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
> :100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
> :100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
> :100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/common/SupportedDirections.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
> common/src/test/java/org/apache/sqoop/common/TestSupportedDirections.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
> core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
> shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
> shell/src/main/resources/shell-resource.properties 73a19e8
>
> Diff: https://reviews.apache.org/r/25672/diff/
>
>
> Testing
> -------
>
> mvn test and manual verification:
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | Id | Name | Version | Class | Supported Directions |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
> | 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
>
> sqoop:000> show connector --all
> 2 connector(s) to show:
> Connector with id 1:
> Name: hdfs-connector
> Class: org.apache.sqoop.connector.hdfs.HdfsConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.dummy
> Label: Dummy parameter needed to get HDFS connector to register
> Help: You can write anything here. Doesn't matter.
> Type: STRING
> Sensitive: false
> Size: 255
> FROM Job form 1:
> Name: input
> Label: Input configuration
> Help: Specifies information required to get data from Hadoop ecosystem
> Input 1:
> Name: input.inputDirectory
> Label: Input directory
> Help: Directory that should be exported
> Type: STRING
> Sensitive: false
> Size: 255
> TO Job form 1:
> Name: output
> Label: Output configuration
> Help: You must supply the information requested in order to get information where you want to store your data.
> Input 1:
> Name: output.outputFormat
> Label: Output format
> Help: Format in which data should be serialized
> Type: ENUM
> Sensitive: false
> Possible values: TEXT_FILE,SEQUENCE_FILE
> Input 2:
> Name: output.compression
> Label: Compression format
> Help: Compression that should be used for the data
> Type: ENUM
> Sensitive: false
> Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
> Input 3:
> Name: output.customCompression
> Label: Custom compression format
> Help: Full class name of the custom compression
> Type: STRING
> Sensitive: false
> Size: 255
> Input 4:
> Name: output.outputDirectory
> Label: Output directory
> Help: Output directory for final data
> Type: STRING
> Sensitive: false
> Size: 255
> Connector with id 2:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.jdbcDriver
> Label: JDBC Driver Class
> Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 2:
> Name: connection.connectionString
> Label: JDBC Connection String
> Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 3:
> Name: connection.username
> Label: Username
> Help: Enter the username to be used for connecting to the database.
> Type: STRING
> Sensitive: false
> Size: 40
> Input 4:
> Name: connection.password
> Label: Password
> Help: Enter the password to be used for connecting to the database.
> Type: STRING
> Sensitive: true
> Size: 40
> Input 5:
> Name: connection.jdbcProperties
> Label: JDBC Connection Properties
> Help: Enter any JDBC properties that should be supplied during the creation of connection.
> Type: MAP
> Sensitive: false
> FROM Job form 1:
> Name: fromTable
> Label: From database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: fromTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: fromTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 3:
> Name: fromTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 4:
> Name: fromTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: fromTable.partitionColumn
> Label: Partition column name
> Help: A specific column for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> Input 6:
> Name: fromTable.partitionColumnNull
> Label: Nulls in partition column
> Help: Whether there are null values in partition column
> Type: BOOLEAN
> Sensitive: false
> Input 7:
> Name: fromTable.boundaryQuery
> Label: Boundary query
> Help: The boundary query for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> TO Job form 1:
> Name: toTable
> Label: To database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: toTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: toTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 3:
> Name: toTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 4:
> Name: toTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: toTable.stageTableName
> Label: Stage table name
> Help: Name of the stage table to use
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 6:
> Name: toTable.clearStageTable
> Label: Clear stage table
> Help: Indicate if the stage table should be cleared
> Type: BOOLEAN
> Sensitive: false
> sqoop:000>
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client
support for supported directions
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/#review54090
-----------------------------------------------------------
common/src/main/java/org/apache/sqoop/common/SupportedDirections.java
<https://reviews.apache.org/r/25672/#comment94047>
The enum usage seems kind of uneasy here as it requires a lot of magic around it when using. Would it be better to simply use pair of booleans here?
Then a constructor would be also simpler as you would just do new SupportedDirections(my-cool-codition-for-from, my-cool-condition-for-to).
common/src/main/java/org/apache/sqoop/model/MConnector.java
<https://reviews.apache.org/r/25672/#comment94048>
With my note above, this would get simplified to:
return new SupportedDirections(this.getJobForms(Direction.FROM) != null, this.getJobForms(Direction.TO) != null;)
common/src/main/java/org/apache/sqoop/model/MConnector.java
<https://reviews.apache.org/r/25672/#comment94049>
I do believe that there is a typo here, it should be Direction.TO, correct? :)
- Jarek Cecho
On Sept. 18, 2014, 1:42 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25672/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 1:42 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1454
> https://issues.apache.org/jira/browse/SQOOP-1454
>
>
> Repository: sqoop-SQOOP-1367
>
>
> Description
> -------
>
> commit f7e49dcd70234398634be1ec9834d901b7e02992
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Mon Sep 15 14:32:40 2014 -0700
>
> SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
>
> :100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
> :100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
> :100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
> :100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
> :100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/common/SupportedDirections.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
> common/src/test/java/org/apache/sqoop/common/TestSupportedDirections.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
> core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
> shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
> shell/src/main/resources/shell-resource.properties 73a19e8
>
> Diff: https://reviews.apache.org/r/25672/diff/
>
>
> Testing
> -------
>
> mvn test and manual verification:
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | Id | Name | Version | Class | Supported Directions |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
> | 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
>
> sqoop:000> show connector --all
> 2 connector(s) to show:
> Connector with id 1:
> Name: hdfs-connector
> Class: org.apache.sqoop.connector.hdfs.HdfsConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.dummy
> Label: Dummy parameter needed to get HDFS connector to register
> Help: You can write anything here. Doesn't matter.
> Type: STRING
> Sensitive: false
> Size: 255
> FROM Job form 1:
> Name: input
> Label: Input configuration
> Help: Specifies information required to get data from Hadoop ecosystem
> Input 1:
> Name: input.inputDirectory
> Label: Input directory
> Help: Directory that should be exported
> Type: STRING
> Sensitive: false
> Size: 255
> TO Job form 1:
> Name: output
> Label: Output configuration
> Help: You must supply the information requested in order to get information where you want to store your data.
> Input 1:
> Name: output.outputFormat
> Label: Output format
> Help: Format in which data should be serialized
> Type: ENUM
> Sensitive: false
> Possible values: TEXT_FILE,SEQUENCE_FILE
> Input 2:
> Name: output.compression
> Label: Compression format
> Help: Compression that should be used for the data
> Type: ENUM
> Sensitive: false
> Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
> Input 3:
> Name: output.customCompression
> Label: Custom compression format
> Help: Full class name of the custom compression
> Type: STRING
> Sensitive: false
> Size: 255
> Input 4:
> Name: output.outputDirectory
> Label: Output directory
> Help: Output directory for final data
> Type: STRING
> Sensitive: false
> Size: 255
> Connector with id 2:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.jdbcDriver
> Label: JDBC Driver Class
> Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 2:
> Name: connection.connectionString
> Label: JDBC Connection String
> Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 3:
> Name: connection.username
> Label: Username
> Help: Enter the username to be used for connecting to the database.
> Type: STRING
> Sensitive: false
> Size: 40
> Input 4:
> Name: connection.password
> Label: Password
> Help: Enter the password to be used for connecting to the database.
> Type: STRING
> Sensitive: true
> Size: 40
> Input 5:
> Name: connection.jdbcProperties
> Label: JDBC Connection Properties
> Help: Enter any JDBC properties that should be supplied during the creation of connection.
> Type: MAP
> Sensitive: false
> FROM Job form 1:
> Name: fromTable
> Label: From database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: fromTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: fromTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 3:
> Name: fromTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 4:
> Name: fromTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: fromTable.partitionColumn
> Label: Partition column name
> Help: A specific column for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> Input 6:
> Name: fromTable.partitionColumnNull
> Label: Nulls in partition column
> Help: Whether there are null values in partition column
> Type: BOOLEAN
> Sensitive: false
> Input 7:
> Name: fromTable.boundaryQuery
> Label: Boundary query
> Help: The boundary query for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> TO Job form 1:
> Name: toTable
> Label: To database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: toTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: toTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 3:
> Name: toTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 4:
> Name: toTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: toTable.stageTableName
> Label: Stage table name
> Help: Name of the stage table to use
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 6:
> Name: toTable.clearStageTable
> Label: Clear stage table
> Help: Indicate if the stage table should be cleared
> Type: BOOLEAN
> Sensitive: false
> sqoop:000>
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client
support for supported directions
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/#review54098
-----------------------------------------------------------
Ship it!
Ship It!
- Jarek Cecho
On Sept. 21, 2014, 5:48 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25672/
> -----------------------------------------------------------
>
> (Updated Sept. 21, 2014, 5:48 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1454
> https://issues.apache.org/jira/browse/SQOOP-1454
>
>
> Repository: sqoop-SQOOP-1367
>
>
> Description
> -------
>
> commit f7e49dcd70234398634be1ec9834d901b7e02992
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Mon Sep 15 14:32:40 2014 -0700
>
> SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
>
> :100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
> :100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
> :100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
> :100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
> :100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/common/SupportedDirections.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
> common/src/test/java/org/apache/sqoop/common/TestSupportedDirections.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
> core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
> shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
> shell/src/main/resources/shell-resource.properties 73a19e8
>
> Diff: https://reviews.apache.org/r/25672/diff/
>
>
> Testing
> -------
>
> mvn test and manual verification:
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | Id | Name | Version | Class | Supported Directions |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
> | 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
> | 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
> +----+------------------------+----------------+------------------------------------------------------+----------------------+
>
> sqoop:000> show connector --all
> 2 connector(s) to show:
> Connector with id 1:
> Name: hdfs-connector
> Class: org.apache.sqoop.connector.hdfs.HdfsConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.dummy
> Label: Dummy parameter needed to get HDFS connector to register
> Help: You can write anything here. Doesn't matter.
> Type: STRING
> Sensitive: false
> Size: 255
> FROM Job form 1:
> Name: input
> Label: Input configuration
> Help: Specifies information required to get data from Hadoop ecosystem
> Input 1:
> Name: input.inputDirectory
> Label: Input directory
> Help: Directory that should be exported
> Type: STRING
> Sensitive: false
> Size: 255
> TO Job form 1:
> Name: output
> Label: Output configuration
> Help: You must supply the information requested in order to get information where you want to store your data.
> Input 1:
> Name: output.outputFormat
> Label: Output format
> Help: Format in which data should be serialized
> Type: ENUM
> Sensitive: false
> Possible values: TEXT_FILE,SEQUENCE_FILE
> Input 2:
> Name: output.compression
> Label: Compression format
> Help: Compression that should be used for the data
> Type: ENUM
> Sensitive: false
> Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
> Input 3:
> Name: output.customCompression
> Label: Custom compression format
> Help: Full class name of the custom compression
> Type: STRING
> Sensitive: false
> Size: 255
> Input 4:
> Name: output.outputDirectory
> Label: Output directory
> Help: Output directory for final data
> Type: STRING
> Sensitive: false
> Size: 255
> Connector with id 2:
> Name: generic-jdbc-connector
> Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
> Version: 2.0.0-SNAPSHOT
> Supported Directions FROM/TO
> Connection form 1:
> Name: connection
> Label: Connection configuration
> Help: You must supply the information requested in order to create a connection object.
> Input 1:
> Name: connection.jdbcDriver
> Label: JDBC Driver Class
> Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 2:
> Name: connection.connectionString
> Label: JDBC Connection String
> Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
> Type: STRING
> Sensitive: false
> Size: 128
> Input 3:
> Name: connection.username
> Label: Username
> Help: Enter the username to be used for connecting to the database.
> Type: STRING
> Sensitive: false
> Size: 40
> Input 4:
> Name: connection.password
> Label: Password
> Help: Enter the password to be used for connecting to the database.
> Type: STRING
> Sensitive: true
> Size: 40
> Input 5:
> Name: connection.jdbcProperties
> Label: JDBC Connection Properties
> Help: Enter any JDBC properties that should be supplied during the creation of connection.
> Type: MAP
> Sensitive: false
> FROM Job form 1:
> Name: fromTable
> Label: From database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: fromTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: fromTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 3:
> Name: fromTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 4:
> Name: fromTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: fromTable.partitionColumn
> Label: Partition column name
> Help: A specific column for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> Input 6:
> Name: fromTable.partitionColumnNull
> Label: Nulls in partition column
> Help: Whether there are null values in partition column
> Type: BOOLEAN
> Sensitive: false
> Input 7:
> Name: fromTable.boundaryQuery
> Label: Boundary query
> Help: The boundary query for data partition
> Type: STRING
> Sensitive: false
> Size: 50
> TO Job form 1:
> Name: toTable
> Label: To database configuration
> Help: You must supply the information requested in order to create a job object.
> Input 1:
> Name: toTable.schemaName
> Label: Schema name
> Help: Schema name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 2:
> Name: toTable.tableName
> Label: Table name
> Help: Table name to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 3:
> Name: toTable.sql
> Label: Table SQL statement
> Help: SQL statement to process data in the remote database
> Type: STRING
> Sensitive: false
> Size: 50
> Input 4:
> Name: toTable.columns
> Label: Table column names
> Help: Specific columns of a table name or a table SQL
> Type: STRING
> Sensitive: false
> Size: 50
> Input 5:
> Name: toTable.stageTableName
> Label: Stage table name
> Help: Name of the stage table to use
> Type: STRING
> Sensitive: false
> Size: 2000
> Input 6:
> Name: toTable.clearStageTable
> Label: Clear stage table
> Help: Indicate if the stage table should be cleared
> Type: BOOLEAN
> Sensitive: false
> sqoop:000>
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client
support for supported directions
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/
-----------------------------------------------------------
(Updated Sept. 21, 2014, 5:48 p.m.)
Review request for Sqoop.
Bugs: SQOOP-1454
https://issues.apache.org/jira/browse/SQOOP-1454
Repository: sqoop-SQOOP-1367
Description
-------
commit f7e49dcd70234398634be1ec9834d901b7e02992
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Mon Sep 15 14:32:40 2014 -0700
SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
:100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
:100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
:100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
:100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
:100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
Diffs (updated)
-----
common/src/main/java/org/apache/sqoop/common/SupportedDirections.java PRE-CREATION
common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
common/src/test/java/org/apache/sqoop/common/TestSupportedDirections.java PRE-CREATION
common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
shell/src/main/resources/shell-resource.properties 73a19e8
Diff: https://reviews.apache.org/r/25672/diff/
Testing
-------
mvn test and manual verification:
+----+------------------------+----------------+------------------------------------------------------+----------------------+
| Id | Name | Version | Class | Supported Directions |
+----+------------------------+----------------+------------------------------------------------------+----------------------+
| 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
| 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
+----+------------------------+----------------+------------------------------------------------------+----------------------+
sqoop:000> show connector --all
2 connector(s) to show:
Connector with id 1:
Name: hdfs-connector
Class: org.apache.sqoop.connector.hdfs.HdfsConnector
Version: 2.0.0-SNAPSHOT
Supported Directions FROM/TO
Connection form 1:
Name: connection
Label: Connection configuration
Help: You must supply the information requested in order to create a connection object.
Input 1:
Name: connection.dummy
Label: Dummy parameter needed to get HDFS connector to register
Help: You can write anything here. Doesn't matter.
Type: STRING
Sensitive: false
Size: 255
FROM Job form 1:
Name: input
Label: Input configuration
Help: Specifies information required to get data from Hadoop ecosystem
Input 1:
Name: input.inputDirectory
Label: Input directory
Help: Directory that should be exported
Type: STRING
Sensitive: false
Size: 255
TO Job form 1:
Name: output
Label: Output configuration
Help: You must supply the information requested in order to get information where you want to store your data.
Input 1:
Name: output.outputFormat
Label: Output format
Help: Format in which data should be serialized
Type: ENUM
Sensitive: false
Possible values: TEXT_FILE,SEQUENCE_FILE
Input 2:
Name: output.compression
Label: Compression format
Help: Compression that should be used for the data
Type: ENUM
Sensitive: false
Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
Input 3:
Name: output.customCompression
Label: Custom compression format
Help: Full class name of the custom compression
Type: STRING
Sensitive: false
Size: 255
Input 4:
Name: output.outputDirectory
Label: Output directory
Help: Output directory for final data
Type: STRING
Sensitive: false
Size: 255
Connector with id 2:
Name: generic-jdbc-connector
Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
Version: 2.0.0-SNAPSHOT
Supported Directions FROM/TO
Connection form 1:
Name: connection
Label: Connection configuration
Help: You must supply the information requested in order to create a connection object.
Input 1:
Name: connection.jdbcDriver
Label: JDBC Driver Class
Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
Type: STRING
Sensitive: false
Size: 128
Input 2:
Name: connection.connectionString
Label: JDBC Connection String
Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
Type: STRING
Sensitive: false
Size: 128
Input 3:
Name: connection.username
Label: Username
Help: Enter the username to be used for connecting to the database.
Type: STRING
Sensitive: false
Size: 40
Input 4:
Name: connection.password
Label: Password
Help: Enter the password to be used for connecting to the database.
Type: STRING
Sensitive: true
Size: 40
Input 5:
Name: connection.jdbcProperties
Label: JDBC Connection Properties
Help: Enter any JDBC properties that should be supplied during the creation of connection.
Type: MAP
Sensitive: false
FROM Job form 1:
Name: fromTable
Label: From database configuration
Help: You must supply the information requested in order to create a job object.
Input 1:
Name: fromTable.schemaName
Label: Schema name
Help: Schema name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 2:
Name: fromTable.tableName
Label: Table name
Help: Table name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 3:
Name: fromTable.sql
Label: Table SQL statement
Help: SQL statement to process data in the remote database
Type: STRING
Sensitive: false
Size: 2000
Input 4:
Name: fromTable.columns
Label: Table column names
Help: Specific columns of a table name or a table SQL
Type: STRING
Sensitive: false
Size: 50
Input 5:
Name: fromTable.partitionColumn
Label: Partition column name
Help: A specific column for data partition
Type: STRING
Sensitive: false
Size: 50
Input 6:
Name: fromTable.partitionColumnNull
Label: Nulls in partition column
Help: Whether there are null values in partition column
Type: BOOLEAN
Sensitive: false
Input 7:
Name: fromTable.boundaryQuery
Label: Boundary query
Help: The boundary query for data partition
Type: STRING
Sensitive: false
Size: 50
TO Job form 1:
Name: toTable
Label: To database configuration
Help: You must supply the information requested in order to create a job object.
Input 1:
Name: toTable.schemaName
Label: Schema name
Help: Schema name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 2:
Name: toTable.tableName
Label: Table name
Help: Table name to process data in the remote database
Type: STRING
Sensitive: false
Size: 2000
Input 3:
Name: toTable.sql
Label: Table SQL statement
Help: SQL statement to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 4:
Name: toTable.columns
Label: Table column names
Help: Specific columns of a table name or a table SQL
Type: STRING
Sensitive: false
Size: 50
Input 5:
Name: toTable.stageTableName
Label: Stage table name
Help: Name of the stage table to use
Type: STRING
Sensitive: false
Size: 2000
Input 6:
Name: toTable.clearStageTable
Label: Clear stage table
Help: Indicate if the stage table should be cleared
Type: BOOLEAN
Sensitive: false
sqoop:000>
Thanks,
Abraham Elmahrek
Re: Review Request 25672: SQOOP-1454: Sqoop2: From/To: Add client
support for supported directions
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25672/
-----------------------------------------------------------
(Updated Sept. 18, 2014, 1:42 a.m.)
Review request for Sqoop.
Bugs: SQOOP-1454
https://issues.apache.org/jira/browse/SQOOP-1454
Repository: sqoop-SQOOP-1367
Description
-------
commit f7e49dcd70234398634be1ec9834d901b7e02992
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Mon Sep 15 14:32:40 2014 -0700
SQOOP-1454: Sqoop2: From/To: Add client support for supported directions
:100644 100644 335a0cc... 78cf0e4... M common/src/main/java/org/apache/sqoop/model/MConnector.java
:100644 100644 9672d9d... 596fae4... M common/src/test/java/org/apache/sqoop/model/TestMConnector.java
:100644 100644 8782209... 6c10b54... M core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java
:100644 100644 bbfbb3f... 0ba40ae... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 efabc46... f0dc3a6... M shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
:100644 100644 73a19e8... 247ceae... M shell/src/main/resources/shell-resource.properties
Diffs (updated)
-----
common/src/main/java/org/apache/sqoop/common/SupportedDirections.java PRE-CREATION
common/src/main/java/org/apache/sqoop/model/MConnector.java 335a0cc
common/src/test/java/org/apache/sqoop/common/TestSupportedDirections.java PRE-CREATION
common/src/test/java/org/apache/sqoop/model/TestMConnector.java 9672d9d
core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 8782209
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java bbfbb3f
shell/src/main/java/org/apache/sqoop/shell/core/Constants.java efabc46
shell/src/main/resources/shell-resource.properties 73a19e8
Diff: https://reviews.apache.org/r/25672/diff/
Testing
-------
mvn test and manual verification:
+----+------------------------+----------------+------------------------------------------------------+----------------------+
| Id | Name | Version | Class | Supported Directions |
+----+------------------------+----------------+------------------------------------------------------+----------------------+
| 1 | hdfs-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
| 2 | generic-jdbc-connector | 2.0.0-SNAPSHOT | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
+----+------------------------+----------------+------------------------------------------------------+----------------------+
sqoop:000> show connector --all
2 connector(s) to show:
Connector with id 1:
Name: hdfs-connector
Class: org.apache.sqoop.connector.hdfs.HdfsConnector
Version: 2.0.0-SNAPSHOT
Supported Directions FROM/TO
Connection form 1:
Name: connection
Label: Connection configuration
Help: You must supply the information requested in order to create a connection object.
Input 1:
Name: connection.dummy
Label: Dummy parameter needed to get HDFS connector to register
Help: You can write anything here. Doesn't matter.
Type: STRING
Sensitive: false
Size: 255
FROM Job form 1:
Name: input
Label: Input configuration
Help: Specifies information required to get data from Hadoop ecosystem
Input 1:
Name: input.inputDirectory
Label: Input directory
Help: Directory that should be exported
Type: STRING
Sensitive: false
Size: 255
TO Job form 1:
Name: output
Label: Output configuration
Help: You must supply the information requested in order to get information where you want to store your data.
Input 1:
Name: output.outputFormat
Label: Output format
Help: Format in which data should be serialized
Type: ENUM
Sensitive: false
Possible values: TEXT_FILE,SEQUENCE_FILE
Input 2:
Name: output.compression
Label: Compression format
Help: Compression that should be used for the data
Type: ENUM
Sensitive: false
Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM
Input 3:
Name: output.customCompression
Label: Custom compression format
Help: Full class name of the custom compression
Type: STRING
Sensitive: false
Size: 255
Input 4:
Name: output.outputDirectory
Label: Output directory
Help: Output directory for final data
Type: STRING
Sensitive: false
Size: 255
Connector with id 2:
Name: generic-jdbc-connector
Class: org.apache.sqoop.connector.jdbc.GenericJdbcConnector
Version: 2.0.0-SNAPSHOT
Supported Directions FROM/TO
Connection form 1:
Name: connection
Label: Connection configuration
Help: You must supply the information requested in order to create a connection object.
Input 1:
Name: connection.jdbcDriver
Label: JDBC Driver Class
Help: Enter the fully qualified class name of the JDBC driver that will be used for establishing this connection.
Type: STRING
Sensitive: false
Size: 128
Input 2:
Name: connection.connectionString
Label: JDBC Connection String
Help: Enter the value of JDBC connection string to be used by this connector for creating connections.
Type: STRING
Sensitive: false
Size: 128
Input 3:
Name: connection.username
Label: Username
Help: Enter the username to be used for connecting to the database.
Type: STRING
Sensitive: false
Size: 40
Input 4:
Name: connection.password
Label: Password
Help: Enter the password to be used for connecting to the database.
Type: STRING
Sensitive: true
Size: 40
Input 5:
Name: connection.jdbcProperties
Label: JDBC Connection Properties
Help: Enter any JDBC properties that should be supplied during the creation of connection.
Type: MAP
Sensitive: false
FROM Job form 1:
Name: fromTable
Label: From database configuration
Help: You must supply the information requested in order to create a job object.
Input 1:
Name: fromTable.schemaName
Label: Schema name
Help: Schema name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 2:
Name: fromTable.tableName
Label: Table name
Help: Table name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 3:
Name: fromTable.sql
Label: Table SQL statement
Help: SQL statement to process data in the remote database
Type: STRING
Sensitive: false
Size: 2000
Input 4:
Name: fromTable.columns
Label: Table column names
Help: Specific columns of a table name or a table SQL
Type: STRING
Sensitive: false
Size: 50
Input 5:
Name: fromTable.partitionColumn
Label: Partition column name
Help: A specific column for data partition
Type: STRING
Sensitive: false
Size: 50
Input 6:
Name: fromTable.partitionColumnNull
Label: Nulls in partition column
Help: Whether there are null values in partition column
Type: BOOLEAN
Sensitive: false
Input 7:
Name: fromTable.boundaryQuery
Label: Boundary query
Help: The boundary query for data partition
Type: STRING
Sensitive: false
Size: 50
TO Job form 1:
Name: toTable
Label: To database configuration
Help: You must supply the information requested in order to create a job object.
Input 1:
Name: toTable.schemaName
Label: Schema name
Help: Schema name to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 2:
Name: toTable.tableName
Label: Table name
Help: Table name to process data in the remote database
Type: STRING
Sensitive: false
Size: 2000
Input 3:
Name: toTable.sql
Label: Table SQL statement
Help: SQL statement to process data in the remote database
Type: STRING
Sensitive: false
Size: 50
Input 4:
Name: toTable.columns
Label: Table column names
Help: Specific columns of a table name or a table SQL
Type: STRING
Sensitive: false
Size: 50
Input 5:
Name: toTable.stageTableName
Label: Stage table name
Help: Name of the stage table to use
Type: STRING
Sensitive: false
Size: 2000
Input 6:
Name: toTable.clearStageTable
Label: Clear stage table
Help: Indicate if the stage table should be cleared
Type: BOOLEAN
Sensitive: false
sqoop:000>
Thanks,
Abraham Elmahrek