You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Abraham Elmahrek <ab...@cloudera.com> on 2013/08/01 02:40:29 UTC
Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for client
commands
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description
-------
commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Wed Jul 31 14:23:58 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
:100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
:100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
:100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
:100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
:100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
:100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
:100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
:100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
:100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
:100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
:100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
:100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
:100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
:100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
:100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
:100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
:100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
:100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
:100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
:100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
:100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
:100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
:100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
:100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
:100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
:100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
:100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
:000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
Diffs
-----
client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java 0538901
client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java 6f62813
client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java ac555e1
client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java 04b240c
client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java cc4d546
client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java 18d3a70
client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java 736be20
client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java e04292a
client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java 5962cd2
client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java ed6dc3c
client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java 9e4e320
client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java e843ede
client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java 41fc17a
client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java 94f92b3
client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java b053339
client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java 58b2c6e
client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java 97a240b
client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java 81c5612
client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java 110f67e
client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java 666eb7a
client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java 8e17f67
client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java bf26761
client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java 2e1c8d3
client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java b854a90
client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java 49ab461
client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java 8556e2b
client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java 425a53f
client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java 2fbf129
client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java PRE-CREATION
Diff: https://reviews.apache.org/r/13181/diff/
Testing
-------
Executed the following commands in a file:
reate connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hu\
e.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageTy\
pe 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Thanks,
Abraham Elmahrek
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review24508
-----------------------------------------------------------
Abe,
Thanks for the patch. This looks pretty good. I did a quick initial and partial review - so I could get some feedback since I'll be able to get back to this only on Monday. Please note that most of the comments follow a pattern - so you should be able to fix it in more classes than I have mentioned here.
client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
<https://reviews.apache.org/r/13181/#comment48487>
Shouldn't isInteractive arg with value false be passed in?
client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
<https://reviews.apache.org/r/13181/#comment48488>
isInteractive = false to be passed in?
client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
<https://reviews.apache.org/r/13181/#comment48489>
Ideally, for 2 methods with the same name, you should not have them behave differently - one is setting isInteractive = false and the other sets it to true.
client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
<https://reviews.apache.org/r/13181/#comment48490>
Same as above, they should behave in the same way
client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
<https://reviews.apache.org/r/13181/#comment48491>
Same as above
client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
<https://reviews.apache.org/r/13181/#comment48492>
This same method seems to be overridden in all subclasses - is it possible to move this up, and just pass in the options which should be handled?
- Hari Shreedharan
On Aug. 1, 2013, 12:40 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Aug. 1, 2013, 12:40 a.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Wed Jul 31 14:23:58 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
> :100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
> :100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
> :100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
> :100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
> :100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
> :100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
> :100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
> :100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
> :100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
> :100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
> :100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
> :100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
> :100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
> :100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
> :100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
> :100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
> :100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
> :100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
> :100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
> :100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
> :100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
> :100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
> :100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
> :100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
> :100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
> :100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
> :100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
> :000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java 0538901
> client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java 6f62813
> client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java ac555e1
> client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java 04b240c
> client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java cc4d546
> client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java 18d3a70
> client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java 736be20
> client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java e04292a
> client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java 5962cd2
> client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java ed6dc3c
> client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java 9e4e320
> client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java e843ede
> client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java 41fc17a
> client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java 94f92b3
> client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java b053339
> client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java 58b2c6e
> client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java 97a240b
> client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java 81c5612
> client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java 110f67e
> client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java 666eb7a
> client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java 8e17f67
> client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java bf26761
> client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java 2e1c8d3
> client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java b854a90
> client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java 49ab461
> client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java 8556e2b
> client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java 425a53f
> client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java 2fbf129
> client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> reate connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hu\
> e.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageTy\
> pe 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 11, 2013, 8:02 p.m., Jarek Cecho wrote:
> > Hi Abe,
> > thank you very much for working on the patch, appreciated! I've tested it in my environment and I do have couple of notes:
> >
> > I think that we need to ensure that one failed command will end entire script execution. I'm thinking about use case that would do the following:
> >
> > 1) create connection
> > 2) create job for -^
> > 3) run the job -^
> >
> > In such situation we need to ensure that we are not creating job for non-existing connection or running non-existing job.
>
> Abraham Elmahrek wrote:
> Thanks for the review Jarcec. Would it be appropriate to have executeFunction and its parent callers return a boolean value rather than an Object value? I think an Object value would be useful, but perhaps a more abstracted Object value that we could handle in a follow up jira. i.e. A class that contains a Status and a return object?
>
> Jarek Cecho wrote:
> The reason why the executeFunction() methods are returning Object instead of boolean is that it's expected by Groovy shell. Having said that I would prefer to return what is expected from us, rather then limit it to a boolean value only.
Jarcec, I've created SQOOP-1217 to follow up. For now we can return an Object, but since the result will normally be null, the script will stop executing after one command. To improve this, in this review, I'll return a non-null object to indicate execution should progress and a null value to indicate execution should halt. Then, in SQOOP-1217, we can follow through with a different kind of object that would encapsulate information on whether the batch script should continue, or stop, and the result object itself.
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26935
-----------------------------------------------------------
On Sept. 27, 2013, 6:30 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 27, 2013, 6:30 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 11, 2013, 8:02 p.m., Jarek Cecho wrote:
> > Hi Abe,
> > thank you very much for working on the patch, appreciated! I've tested it in my environment and I do have couple of notes:
> >
> > I think that we need to ensure that one failed command will end entire script execution. I'm thinking about use case that would do the following:
> >
> > 1) create connection
> > 2) create job for -^
> > 3) run the job -^
> >
> > In such situation we need to ensure that we are not creating job for non-existing connection or running non-existing job.
Thanks for the review Jarcec. Would it be appropriate to have executeFunction and its parent callers return a boolean value rather than an Object value? I think an Object value would be useful, but perhaps a more abstracted Object value that we could handle in a follow up jira. i.e. A class that contains a Status and a return object?
> On Oct. 11, 2013, 8:02 p.m., Jarek Cecho wrote:
> > shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java, lines 107-115
> > <https://reviews.apache.org/r/13181/diff/6/?file=359340#file359340line107>
> >
> > Can we make this and similar code a bit more verbose?
> >
> > I've tried following script:
> >
> > create connection --cid 1 --name NameX
> >
> > And I got following output:
> >
> > [root@bousa-trunk sqoop]# ./bin/sqoop.sh client create.sqoop
> > Sqoop home directory: /root/sqoop
> > sqoop:000> create connection --cid 1 --name NameX
> > Creating connection for connector with id 1
> > [root@bousa-trunk sqoop]#
> >
> > Creating the connection has failed as I did not specified any arguments, but there is no error message which is really confusing.
I see FormFiller#printValidationMessage exists to aid in printing error messages. Can I simply re-use this for the time being? I think a more appropriate response would be to describe what exactly is missing (cf. help), but it seems fine to focus on one thing in this jira? The error output would look like:
Error message: Driver can't be empty
Error message: JDBC URL can't be empty
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26935
-----------------------------------------------------------
On Sept. 27, 2013, 6:30 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 27, 2013, 6:30 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
> On Oct. 11, 2013, 8:02 p.m., Jarek Cecho wrote:
> > Hi Abe,
> > thank you very much for working on the patch, appreciated! I've tested it in my environment and I do have couple of notes:
> >
> > I think that we need to ensure that one failed command will end entire script execution. I'm thinking about use case that would do the following:
> >
> > 1) create connection
> > 2) create job for -^
> > 3) run the job -^
> >
> > In such situation we need to ensure that we are not creating job for non-existing connection or running non-existing job.
>
> Abraham Elmahrek wrote:
> Thanks for the review Jarcec. Would it be appropriate to have executeFunction and its parent callers return a boolean value rather than an Object value? I think an Object value would be useful, but perhaps a more abstracted Object value that we could handle in a follow up jira. i.e. A class that contains a Status and a return object?
The reason why the executeFunction() methods are returning Object instead of boolean is that it's expected by Groovy shell. Having said that I would prefer to return what is expected from us, rather then limit it to a boolean value only.
> On Oct. 11, 2013, 8:02 p.m., Jarek Cecho wrote:
> > shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java, lines 107-115
> > <https://reviews.apache.org/r/13181/diff/6/?file=359340#file359340line107>
> >
> > Can we make this and similar code a bit more verbose?
> >
> > I've tried following script:
> >
> > create connection --cid 1 --name NameX
> >
> > And I got following output:
> >
> > [root@bousa-trunk sqoop]# ./bin/sqoop.sh client create.sqoop
> > Sqoop home directory: /root/sqoop
> > sqoop:000> create connection --cid 1 --name NameX
> > Creating connection for connector with id 1
> > [root@bousa-trunk sqoop]#
> >
> > Creating the connection has failed as I did not specified any arguments, but there is no error message which is really confusing.
>
> Abraham Elmahrek wrote:
> I see FormFiller#printValidationMessage exists to aid in printing error messages. Can I simply re-use this for the time being? I think a more appropriate response would be to describe what exactly is missing (cf. help), but it seems fine to focus on one thing in this jira? The error output would look like:
> Error message: Driver can't be empty
> Error message: JDBC URL can't be empty
Yeah, that seems good, let's try to reuse that!
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26935
-----------------------------------------------------------
On Sept. 27, 2013, 6:30 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 27, 2013, 6:30 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
> On Oct. 11, 2013, 8:02 p.m., Jarek Cecho wrote:
> > Hi Abe,
> > thank you very much for working on the patch, appreciated! I've tested it in my environment and I do have couple of notes:
> >
> > I think that we need to ensure that one failed command will end entire script execution. I'm thinking about use case that would do the following:
> >
> > 1) create connection
> > 2) create job for -^
> > 3) run the job -^
> >
> > In such situation we need to ensure that we are not creating job for non-existing connection or running non-existing job.
>
> Abraham Elmahrek wrote:
> Thanks for the review Jarcec. Would it be appropriate to have executeFunction and its parent callers return a boolean value rather than an Object value? I think an Object value would be useful, but perhaps a more abstracted Object value that we could handle in a follow up jira. i.e. A class that contains a Status and a return object?
>
> Jarek Cecho wrote:
> The reason why the executeFunction() methods are returning Object instead of boolean is that it's expected by Groovy shell. Having said that I would prefer to return what is expected from us, rather then limit it to a boolean value only.
>
> Abraham Elmahrek wrote:
> Jarcec, I've created SQOOP-1217 to follow up. For now we can return an Object, but since the result will normally be null, the script will stop executing after one command. To improve this, in this review, I'll return a non-null object to indicate execution should progress and a null value to indicate execution should halt. Then, in SQOOP-1217, we can follow through with a different kind of object that would encapsulate information on whether the batch script should continue, or stop, and the result object itself.
Sounds good to me, thank you for following on this!
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26935
-----------------------------------------------------------
On Oct. 14, 2013, 7:05 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2013, 7:05 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/SqoopShell.java 900c0e57b76509dccb835051a9336bd5f1f8daa9
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType HDFS --framework-output-outputFormat TEXT_FILE --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
> Also executed the following commands in a file:
> create connection --cid 1
> create connection --cid 1
>
> Verified erroneous commands stop the execution of the script. Verified statuses being returned will be visible on the screen when executing batch script.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26935
-----------------------------------------------------------
Hi Abe,
thank you very much for working on the patch, appreciated! I've tested it in my environment and I do have couple of notes:
I think that we need to ensure that one failed command will end entire script execution. I'm thinking about use case that would do the following:
1) create connection
2) create job for -^
3) run the job -^
In such situation we need to ensure that we are not creating job for non-existing connection or running non-existing job.
shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
<https://reviews.apache.org/r/13181/#comment52475>
Can we make this and similar code a bit more verbose?
I've tried following script:
create connection --cid 1 --name NameX
And I got following output:
[root@bousa-trunk sqoop]# ./bin/sqoop.sh client create.sqoop
Sqoop home directory: /root/sqoop
sqoop:000> create connection --cid 1 --name NameX
Creating connection for connector with id 1
[root@bousa-trunk sqoop]#
Creating the connection has failed as I did not specified any arguments, but there is no error message which is really confusing.
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
<https://reviews.apache.org/r/13181/#comment52464>
It seems that some of the functions are just returning true and not doing any validations. What about making such function default here and let the children override it only if some validations will be required?
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
<https://reviews.apache.org/r/13181/#comment52462>
Nit: Unused import.
Jarcec
- Jarek Cecho
On Sept. 27, 2013, 6:30 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 27, 2013, 6:30 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review27088
-----------------------------------------------------------
Ship it!
Ship It!
- Jarek Cecho
On Oct. 15, 2013, 10:55 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Oct. 15, 2013, 10:55 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/SqoopShell.java 900c0e57b76509dccb835051a9336bd5f1f8daa9
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType HDFS --framework-output-outputFormat TEXT_FILE --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
> Also executed the following commands in a file:
> create connection --cid 1
> create connection --cid 1
>
> Verified erroneous commands stop the execution of the script. Verified statuses being returned will be visible on the screen when executing batch script.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
(Updated Oct. 15, 2013, 10:55 p.m.)
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description
-------
commit 080ced16578c1d95015ce3e99b4335beb465861a
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Tue Aug 13 14:14:06 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
:100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
:100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
:100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
:100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
:100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
:100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
:100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
:100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
:100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
:100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
:100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
:100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
:100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
:100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
:100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
:100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
:100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
:100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
:100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
:100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
:100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
:100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
:100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
:100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
:100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
:100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
:100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
:100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
:100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
:100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
:000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
:000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
:100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
:000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
:000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
Diffs (updated)
-----
shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
shell/src/main/java/org/apache/sqoop/shell/SqoopShell.java 900c0e57b76509dccb835051a9336bd5f1f8daa9
shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
Diff: https://reviews.apache.org/r/13181/diff/
Testing
-------
Executed the following commands in a file:
create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType HDFS --framework-output-outputFormat TEXT_FILE --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Also executed the following commands in a file:
create connection --cid 1
create connection --cid 1
Verified erroneous commands stop the execution of the script. Verified statuses being returned will be visible on the screen when executing batch script.
Thanks,
Abraham Elmahrek
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review27042
-----------------------------------------------------------
Thank you for incorporating the changes Abe, appreciated! I like the changes and I do have just couple of nits from user perspective. I've tried following script:
set server --host my.cool.and.super.secret.host
set option --name verbose --value true
show version --all
create connection --cid 1 --name NameX
With following output:
jarcec@Odie sqoop2 % dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop200/bin/sqoop.sh client command.createConnectionInvalid
Sqoop home directory: /home/jarcec/apache/repos/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop200
sqoop:000> set server --host my.cool.and.super.secret.host
Server is set successfully
===> FINE
FINE
sqoop:000> set option --name verbose --value true
Verbose option was changed to true
===> FINE
FINE
sqoop:000> show version --all
client version:
Sqoop 2.0.0-SNAPSHOT revision 712b26b969187f48553eeab0bf64c26e14190984
Compiled by jarcec on Tue Oct 15 13:38:11 PDT 2013
server version:
Sqoop 2.0.0-SNAPSHOT revision 712b26b969187f48553eeab0bf64c26e14190984
Compiled by jarcec on Tue Oct 15 13:38:11 PDT 2013
Protocol version:
[1]
===> FINE
FINE
sqoop:000> create connection --cid 1 --name NameX
Creating connection for connector with id 1
Error message: Driver can't be empty
Error message: JDBC URL can't be empty
1) It seems that we are printing the "FINE" string twice, would it be feasible to print it only once?
2) The error messages are not tied to any input, so it's hard to correlate them. Would it be feasible to do something like:
input.driver: ERROR: Driver can't be empty
Jarcec
- Jarek Cecho
On Oct. 14, 2013, 7:05 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2013, 7:05 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/SqoopShell.java 900c0e57b76509dccb835051a9336bd5f1f8daa9
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType HDFS --framework-output-outputFormat TEXT_FILE --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
> Also executed the following commands in a file:
> create connection --cid 1
> create connection --cid 1
>
> Verified erroneous commands stop the execution of the script. Verified statuses being returned will be visible on the screen when executing batch script.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review27043
-----------------------------------------------------------
Thank you for incorporating the changes Abe, appreciated! I like the changes and I do have just couple of nits from user perspective. I've tried following script:
set server --host my.cool.and.super.secret.host
set option --name verbose --value true
show version --all
create connection --cid 1 --name NameX
With following output:
jarcec@Odie sqoop2 % dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop200/bin/sqoop.sh client command.createConnectionInvalid
Sqoop home directory: /home/jarcec/apache/repos/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop200
sqoop:000> set server --host my.cool.and.super.secret.host
Server is set successfully
===> FINE
FINE
sqoop:000> set option --name verbose --value true
Verbose option was changed to true
===> FINE
FINE
sqoop:000> show version --all
client version:
Sqoop 2.0.0-SNAPSHOT revision 712b26b969187f48553eeab0bf64c26e14190984
Compiled by jarcec on Tue Oct 15 13:38:11 PDT 2013
server version:
Sqoop 2.0.0-SNAPSHOT revision 712b26b969187f48553eeab0bf64c26e14190984
Compiled by jarcec on Tue Oct 15 13:38:11 PDT 2013
Protocol version:
[1]
===> FINE
FINE
sqoop:000> create connection --cid 1 --name NameX
Creating connection for connector with id 1
Error message: Driver can't be empty
Error message: JDBC URL can't be empty
1) It seems that we are printing the "FINE" string twice, would it be feasible to print it only once?
2) The error messages are not tied to any input, so it's hard to correlate them. Would it be feasible to do something like:
input.driver: ERROR: Driver can't be empty
Jarcec
- Jarek Cecho
On Oct. 14, 2013, 7:05 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2013, 7:05 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/SqoopShell.java 900c0e57b76509dccb835051a9336bd5f1f8daa9
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType HDFS --framework-output-outputFormat TEXT_FILE --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
> Also executed the following commands in a file:
> create connection --cid 1
> create connection --cid 1
>
> Verified erroneous commands stop the execution of the script. Verified statuses being returned will be visible on the screen when executing batch script.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
(Updated Oct. 14, 2013, 7:05 p.m.)
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description
-------
commit 080ced16578c1d95015ce3e99b4335beb465861a
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Tue Aug 13 14:14:06 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
:100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
:100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
:100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
:100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
:100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
:100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
:100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
:100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
:100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
:100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
:100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
:100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
:100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
:100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
:100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
:100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
:100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
:100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
:100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
:100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
:100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
:100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
:100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
:100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
:100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
:100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
:100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
:100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
:100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
:100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
:000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
:000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
:100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
:000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
:000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
Diffs (updated)
-----
shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
shell/src/main/java/org/apache/sqoop/shell/SqoopShell.java 900c0e57b76509dccb835051a9336bd5f1f8daa9
shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
Diff: https://reviews.apache.org/r/13181/diff/
Testing (updated)
-------
Executed the following commands in a file:
create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType HDFS --framework-output-outputFormat TEXT_FILE --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Also executed the following commands in a file:
create connection --cid 1
create connection --cid 1
Verified erroneous commands stop the execution of the script. Verified statuses being returned will be visible on the screen when executing batch script.
Thanks,
Abraham Elmahrek
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
(Updated Sept. 27, 2013, 6:30 p.m.)
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description
-------
commit 080ced16578c1d95015ce3e99b4335beb465861a
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Tue Aug 13 14:14:06 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
:100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
:100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
:100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
:100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
:100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
:100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
:100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
:100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
:100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
:100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
:100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
:100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
:100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
:100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
:100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
:100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
:100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
:100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
:100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
:100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
:100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
:100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
:100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
:100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
:100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
:100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
:100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
:100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
:100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
:100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
:000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
:000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
:100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
:000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
:000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
Diffs (updated)
-----
shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
shell/src/main/java/org/apache/sqoop/shell/core/ShellError.java e5a99f1cdef7afc4df12e7eed36e1096e2ccd3ea
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
Diff: https://reviews.apache.org/r/13181/diff/
Testing
-------
Executed the following commands in a file:
create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Thanks,
Abraham Elmahrek
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Sept. 24, 2013, 10:27 p.m., Jarek Cecho wrote:
> > Hi Abe,
> > thank you very much for working on this patch, greatly appreciated! I've took a look and I do have couple of notes:
Thank you Jarcec! Awesome suggestions as always!
> On Sept. 24, 2013, 10:27 p.m., Jarek Cecho wrote:
> > shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java, lines 56-61
> > <https://reviews.apache.org/r/13181/diff/5/?file=356427#file356427line56>
> >
> > Considering that almost all execute*Function() will have to validate the arguments the same way, do you think that it would make sense to introduce a new abstract method validateCommandLine(CommandLine, interactive)?
Are you referring to code segments that look like?:
if (!line.hasOption(Constants.OPT_CID)) {
printlnResource(Constants.RES_ARGS_CID_MISSING);
return null;
}
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26356
-----------------------------------------------------------
On Sept. 23, 2013, 11:48 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 23, 2013, 11:48 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
> On Sept. 24, 2013, 10:27 p.m., Jarek Cecho wrote:
> > shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java, lines 56-61
> > <https://reviews.apache.org/r/13181/diff/5/?file=356427#file356427line56>
> >
> > Considering that almost all execute*Function() will have to validate the arguments the same way, do you think that it would make sense to introduce a new abstract method validateCommandLine(CommandLine, interactive)?
>
> Abraham Elmahrek wrote:
> Are you referring to code segments that look like?:
> if (!line.hasOption(Constants.OPT_CID)) {
> printlnResource(Constants.RES_ARGS_CID_MISSING);
> return null;
> }
Yes, for example I was referring to this fragment.
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26356
-----------------------------------------------------------
On Sept. 23, 2013, 11:48 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 23, 2013, 11:48 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26356
-----------------------------------------------------------
Hi Abe,
thank you very much for working on this patch, greatly appreciated! I've took a look and I do have couple of notes:
shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
<https://reviews.apache.org/r/13181/#comment51475>
Would it make sense to remove the constant SHELL_0007 when it's unused now?
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
<https://reviews.apache.org/r/13181/#comment51477>
It seems that majority of the downstream implementations ends up calling the same method with boolean isInteractive as parameter, so I'm wondering if it would make sense to make it simpler and have executeFunction(CommandLine, Boolean isInteractive) in the first place. What do you think?
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
<https://reviews.apache.org/r/13181/#comment51444>
Considering that almost all execute*Function() will have to validate the arguments the same way, do you think that it would make sense to introduce a new abstract method validateCommandLine(CommandLine, interactive)?
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
<https://reviews.apache.org/r/13181/#comment51478>
I would suggest to use the value rather than index in case of batch loading of Enum values.
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
<https://reviews.apache.org/r/13181/#comment51476>
I'm wondering if something like srglist.subList(start, arglist.size()).toArray() wouldn't be simpler here?
Jarcec
- Jarek Cecho
On Sept. 23, 2013, 11:48 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 23, 2013, 11:48 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
(Updated Sept. 23, 2013, 11:48 p.m.)
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description (updated)
-------
commit 080ced16578c1d95015ce3e99b4335beb465861a
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Tue Aug 13 14:14:06 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
:100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
:100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
:100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
:100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
:100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
:100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
:100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
:100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
:100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
:100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
:100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
:100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
:100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
:100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
:100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
:100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
:100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
:100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
:100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
:100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
:100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
:100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
:100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
:100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
:100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
:100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
:100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
:100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
:100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
:100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
:000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
:000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
:100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
:000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
:000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
Diffs (updated)
-----
shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
Diff: https://reviews.apache.org/r/13181/diff/
Testing
-------
Executed the following commands in a file:
create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Thanks,
Abraham Elmahrek
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Sept. 17, 2013, 3:26 p.m., vasanthkumar wrote:
> >
>
> vasanthkumar wrote:
> In Create connection, for sensitive information password can we have separate in next line? Instead of getting password from same command line.
I'm not sure I entirely understand. If there are multiple sensitive fields, then there would be multiple next lines? How would we know which goes where? Also, since this is batch execution mode, then it wouldn't make sense to stop the execution of the commands to ask for inputs.
> On Sept. 17, 2013, 3:26 p.m., vasanthkumar wrote:
> > shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java, line 380
> > <https://reviews.apache.org/r/13181/diff/4/?file=350493#file350493line380>
> >
> > bundle object is not required. Likewise for fillInputBoolean, Integer, enum, Map, enum.
> >
Good point. Thank you!
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26175
-----------------------------------------------------------
On Sept. 11, 2013, 12:35 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 11, 2013, 12:35 a.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Wed Jul 31 14:23:58 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
> :100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
> :100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
> :100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
> :100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
> :100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
> :100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
> :100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
> :100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
> :100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
> :100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
> :100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
> :100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
> :100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
> :100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
> :100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
> :100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
> :100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
> :100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
> :100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
> :100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
> :100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
> :100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
> :100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
> :100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
> :100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
> :100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
> :100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
> :000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by rj...@gmail.com.
> On Sept. 17, 2013, 3:26 p.m., vasanthkumar wrote:
> >
>
> vasanthkumar wrote:
> In Create connection, for sensitive information password can we have separate in next line? Instead of getting password from same command line.
>
> Abraham Elmahrek wrote:
> I'm not sure I entirely understand. If there are multiple sensitive fields, then there would be multiple next lines? How would we know which goes where? Also, since this is batch execution mode, then it wouldn't make sense to stop the execution of the commands to ask for inputs.
create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
Above command, password is visible to user. So like in version 1.4.x, use -P for getting password from console.
- vasanthkumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26175
-----------------------------------------------------------
On Sept. 23, 2013, 11:48 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 23, 2013, 11:48 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Sept. 17, 2013, 3:26 p.m., vasanthkumar wrote:
> >
>
> vasanthkumar wrote:
> In Create connection, for sensitive information password can we have separate in next line? Instead of getting password from same command line.
>
> Abraham Elmahrek wrote:
> I'm not sure I entirely understand. If there are multiple sensitive fields, then there would be multiple next lines? How would we know which goes where? Also, since this is batch execution mode, then it wouldn't make sense to stop the execution of the commands to ask for inputs.
>
> vasanthkumar wrote:
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
>
> Above command, password is visible to user. So like in version 1.4.x, use -P for getting password from console.
>
>
> Jarek Cecho wrote:
> That is very good catch Vasanth! Indeed you are right, the password will be clearly visible in plaintext on the command line which might be seen similar to parameter --password in Sqoop 1.x. Nevertheless I feel that there is a huge difference as Sqoop2 still have the interactive mode where the password is correctly hidden. The JIRA seems to be proposing fully batch mode that can be for example used from within scripts, so reading data from stdin do not appear to be an option in this case. Having said that there is a secure way how to enter sensitive information, do you think that current approach is sensible?
I see what you're saying now. I do agree with Jarcec on his points though. Ideally, connections would be reused so that sensitive information shouldn't less frequently.
One scenario I can imagine is where there are several users that each have their own credentials to a database (which is achievable through LDAP). Then, a sequence of sqoop commands to create a connection and a basic import job to a sandboxed area would be nice. This is treading on scripting territory though, which probably belongs in a shell script of some kind.
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26175
-----------------------------------------------------------
On Sept. 23, 2013, 11:48 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 23, 2013, 11:48 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by rj...@gmail.com.
> On Sept. 17, 2013, 3:26 p.m., vasanthkumar wrote:
> >
In Create connection, for sensitive information password can we have separate in next line? Instead of getting password from same command line.
- vasanthkumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26175
-----------------------------------------------------------
On Sept. 11, 2013, 12:35 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 11, 2013, 12:35 a.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Wed Jul 31 14:23:58 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
> :100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
> :100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
> :100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
> :100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
> :100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
> :100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
> :100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
> :100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
> :100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
> :100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
> :100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
> :100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
> :100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
> :100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
> :100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
> :100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
> :100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
> :100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
> :100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
> :100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
> :100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
> :100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
> :100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
> :100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
> :100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
> :100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
> :100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
> :000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
> On Sept. 17, 2013, 3:26 p.m., vasanthkumar wrote:
> >
>
> vasanthkumar wrote:
> In Create connection, for sensitive information password can we have separate in next line? Instead of getting password from same command line.
>
> Abraham Elmahrek wrote:
> I'm not sure I entirely understand. If there are multiple sensitive fields, then there would be multiple next lines? How would we know which goes where? Also, since this is batch execution mode, then it wouldn't make sense to stop the execution of the commands to ask for inputs.
>
> vasanthkumar wrote:
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
>
> Above command, password is visible to user. So like in version 1.4.x, use -P for getting password from console.
>
That is very good catch Vasanth! Indeed you are right, the password will be clearly visible in plaintext on the command line which might be seen similar to parameter --password in Sqoop 1.x. Nevertheless I feel that there is a huge difference as Sqoop2 still have the interactive mode where the password is correctly hidden. The JIRA seems to be proposing fully batch mode that can be for example used from within scripts, so reading data from stdin do not appear to be an option in this case. Having said that there is a secure way how to enter sensitive information, do you think that current approach is sensible?
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26175
-----------------------------------------------------------
On Sept. 23, 2013, 11:48 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 23, 2013, 11:48 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 080ced16578c1d95015ce3e99b4335beb465861a
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Tue Aug 13 14:14:06 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 980a908... a7e7e7d... M shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
> :100644 100644 856abaa... 2c2869c... M shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
> :100644 100644 3e23025... dd2eb2b... M shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
> :100644 100644 e62ce08... 9ad007b... M shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
> :100644 100644 5fbf0a3... 973fd53... M shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
> :100644 100644 6e4f04b... f0d4a6c... M shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
> :100644 100644 c123732... d79516d... M shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
> :100644 100644 d4095b7... df9a2cc... M shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
> :100644 100644 f119660... b5a54b9... M shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
> :100644 100644 a87e51f... 8cc4ea3... M shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
> :100644 100644 f782b16... 0c9f184... M shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
> :100644 100644 20c80dc... de5000b... M shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
> :100644 100644 1c43dce... f0153b8... M shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
> :100644 100644 abd9cea... 1aacd7c... M shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
> :100644 100644 b55d5d1... f0720a0... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
> :100644 100644 97a4ab2... a9aba71... M shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
> :100644 100644 28497db... c71696c... M shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
> :100644 100644 da4a4ff... 68bd3b3... M shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
> :100644 100644 5e3c3ff... d0b05e2... M shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
> :100644 100644 ec97e63... 11bff86... M shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
> :100644 100644 a592a98... a571269... M shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
> :100644 100644 764b754... 5d6f97f... M shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
> :100644 100644 675a796... 371c867... M shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
> :100644 100644 f03e08f... 914454f... M shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
> :100644 100644 02148de... b8771c4... M shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
> :100644 100644 184892a... ebd4548... M shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
> :100644 100644 be0de8c... c61e9b9... M shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
> :100644 100644 698bca7... 65a454b... M shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
> :100644 100644 6c0e3c2... ae559cc... M shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
> :100644 100644 9262ccd... 24f31ea... M shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
> :100644 100644 c062fe6... 31d9af9... M shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
> :100644 100644 da1e0c5... 6f83b91... M shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
> :000000 100644 0000000... 6b6e858... A shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
> :000000 100644 0000000... cc63610... A shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
> :100644 100644 9bc0b93... bf0615e... M shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
> :000000 100644 0000000... 516790a... A shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
> :000000 100644 0000000... aa118e1... A shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by rj...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review26175
-----------------------------------------------------------
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
<https://reviews.apache.org/r/13181/#comment51139>
bundle object is not required. Likewise for fillInputBoolean, Integer, enum, Map, enum.
- vasanthkumar
On Sept. 11, 2013, 12:35 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Sept. 11, 2013, 12:35 a.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Wed Jul 31 14:23:58 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
> :100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
> :100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
> :100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
> :100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
> :100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
> :100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
> :100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
> :100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
> :100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
> :100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
> :100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
> :100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
> :100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
> :100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
> :100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
> :100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
> :100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
> :100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
> :100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
> :100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
> :100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
> :100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
> :100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
> :100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
> :100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
> :100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
> :100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
> :000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
>
>
> Diffs
> -----
>
> shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
(Updated Sept. 11, 2013, 12:35 a.m.)
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description
-------
commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Wed Jul 31 14:23:58 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
:100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
:100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
:100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
:100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
:100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
:100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
:100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
:100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
:100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
:100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
:100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
:100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
:100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
:100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
:100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
:100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
:100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
:100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
:100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
:100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
:100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
:100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
:100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
:100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
:100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
:100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
:100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
:000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
Diffs (updated)
-----
shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java 980a9086f433fd3a2228122b6a253a9a1c562315
shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java e62ce08ca1ee9168a77b619d21db93b04809b7ec
shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b55d5d1126b1bd33f4517173257188504557b3f2
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java da4a4ff324c32deceadd45d336646eb5bc5a7882
shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
shell/src/main/java/org/apache/sqoop/shell/StartCommand.java f03e08fe2178c2b044eef1bfbfa82c9235d8914d
shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java 184892a33b1b26391c7a7eaa8c79cd27f0178c39
shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
shell/src/main/java/org/apache/sqoop/shell/StopCommand.java 698bca778d3aa09fa24372a6367986391f9e61d9
shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java 9262ccd085730c7101179157db9828ac709c267d
shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
Diff: https://reviews.apache.org/r/13181/diff/
Testing (updated)
-------
Executed the following commands in a file:
create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Thanks,
Abraham Elmahrek
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/#review25849
-----------------------------------------------------------
Hi Abe,
thank you very much for working on this. Would you mind rebasing the patch on current HEAD of sqoop2 branch? We've recently split the client module into separate shell and client module via SQOOP-921, so the patch no longer applies cleanly. Couple of high level notes:
server/src/main/java/org/apache/sqoop/server/v1/FrameworkServlet.java
<https://reviews.apache.org/r/13181/#comment50408>
Nit: This change do not seem to be necessary.
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
<https://reviews.apache.org/r/13181/#comment50409>
This file is missing license header.
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
<https://reviews.apache.org/r/13181/#comment50410>
This file is missing license header.
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
<https://reviews.apache.org/r/13181/#comment50412>
We're adding here .hasArg() in all cases (for all input types), however the usage of boolean inputs in fillInputBoolean() method suggest that we are not expecting any arguments as we are just verifying if the parameter was used or not.
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
<https://reviews.apache.org/r/13181/#comment50411>
This file is missing license header.
Jarcec
- Jarek Cecho
On Aug. 13, 2013, 9:30 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13181/
> -----------------------------------------------------------
>
> (Updated Aug. 13, 2013, 9:30 p.m.)
>
>
> Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
>
>
> Bugs: SQOOP-773
> https://issues.apache.org/jira/browse/SQOOP-773
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Wed Jul 31 14:23:58 2013 -0700
>
> SQOOP-773 Sqoop2: Batch execution support for client commands
>
> Separated options into two groups: fixed and dynamic options.
> Fixed options (IE: connector ID) come first and are used to select
> what options should be used in dynamic options. Dynamic options
> are automatically created based on forms selected from fixed options.
> The keys for these options take on the form "<prefix>-<form name>-<input-name>".
>
> :100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
> :100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
> :100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
> :100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
> :100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
> :100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
> :100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
> :100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
> :100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
> :100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
> :100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
> :100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
> :100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
> :100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
> :100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
> :100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
> :100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
> :100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
> :100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
> :100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
> :100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
> :100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
> :100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
> :100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
> :100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
> :100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
> :100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
> :100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
> :000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
>
>
> Diffs
> -----
>
> core/src/test/java/org/apache/sqoop/repository/TestJdbcRepository.java 40a6ecad351682cf6a582dfa46f31b156195949a
> server/src/main/java/org/apache/sqoop/server/v1/FrameworkServlet.java 9c9973146eb5756eef3801aca436e8b2eda4b274
> shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
> shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
> shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
> shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
> shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
> shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
> shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
> shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
> shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
> shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
> shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
> shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b7204ffeb5fb17ef4f2a88919e9e3e3c48cd67f8
> shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
> shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
> shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java fc6f41622125c85efa911b1cf6f7cc68fded8320
> shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
> shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
> shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
> shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
> shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
> shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
> shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
> shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
> shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
> shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
> shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
> shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
> shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13181/diff/
>
>
> Testing
> -------
>
> Executed the following commands in a file:
>
> reate connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hu\
> e.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
> show connection
> create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageTy\
> pe 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
> show job
> start job --jid 1
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
(Updated Aug. 13, 2013, 9:30 p.m.)
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Changes
-------
Rebase
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description
-------
commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Wed Jul 31 14:23:58 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
:100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
:100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
:100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
:100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
:100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
:100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
:100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
:100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
:100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
:100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
:100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
:100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
:100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
:100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
:100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
:100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
:100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
:100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
:100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
:100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
:100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
:100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
:100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
:100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
:100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
:100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
:100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
:000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
Diffs (updated)
-----
core/src/test/java/org/apache/sqoop/repository/TestJdbcRepository.java 40a6ecad351682cf6a582dfa46f31b156195949a
server/src/main/java/org/apache/sqoop/server/v1/FrameworkServlet.java 9c9973146eb5756eef3801aca436e8b2eda4b274
shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java 856abaae5a7745b38b1db42cd9283732d1bc0b64
shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java 3e2302514d9350affbc9ab199fb4e91355201290
shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java 5fbf0a3d8ec13c8e3628525b7d6432d56ddbb0c3
shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 6e4f04bf392f0c062e211fd6709f362ec6e584c3
shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java c1237320b3ceec48569c34f4b2a3fa78b4d2c26e
shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java d4095b736d3dcc97487c20cb71c7c460139c2afd
shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java f1196601857b90442b1c9394902b71d3bd140915
shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java a87e51f32226ad6e65b8970d159861e013c6fe9d
shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java f782b1654203c450b4d22b762dc0f406b560aa29
shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java 20c80dca160ddbfa9edb6c5ea5c19fdc3d0bb7c9
shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java 1c43dceea2581cea70d0324a63292a0909a402e1
shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java abd9ceac5a8338dcb1273b0f4732b6bd79962d65
shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java b7204ffeb5fb17ef4f2a88919e9e3e3c48cd67f8
shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java 97a4ab2a2377a2aff564f8ffc18884aa11d8a5e3
shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java 28497db27295c4ca89ef4b56986818803296cb10
shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java fc6f41622125c85efa911b1cf6f7cc68fded8320
shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java 5e3c3ff9469ee75d6b9d7635de66b033e93c37cd
shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java ec97e637937129ff0e2151612f98eceb284ec126
shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java a592a9861f01d333ef72f1f96bdf038ab334ea9e
shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java 764b754180500a86501450c90b119813de6cf1f7
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java 675a796d7919a3afadea2e14a88f7823d5dced47
shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java 02148de497c15caa363c17a54202690a9026a68b
shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java be0de8c29289a904cfc00361da7fbf9a6c8e7d1c
shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java 6c0e3c284b997561fc351a57e35ce2d6c77f6121
shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java c062fe6781aebac15c4d59b5690772695d8a10a9
shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java da1e0c570fb43cea439549c0ab5268f67d55996d
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java 9bc0b93649cb2ae6c84228b52c04b60a36c13b18
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java PRE-CREATION
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java PRE-CREATION
Diff: https://reviews.apache.org/r/13181/diff/
Testing
-------
Executed the following commands in a file:
reate connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hu\
e.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageTy\
pe 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Thanks,
Abraham Elmahrek
Re: Review Request 13181: SQOOP-773 Sqoop2: Batch execution support for
client commands
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13181/
-----------------------------------------------------------
(Updated Aug. 5, 2013, 9:17 p.m.)
Review request for Sqoop, Hari Shreedharan and Jarek Cecho.
Bugs: SQOOP-773
https://issues.apache.org/jira/browse/SQOOP-773
Repository: sqoop-sqoop2
Description
-------
commit d894ac7b09a18f3fed1fb58bed7554a873fa8630
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Wed Jul 31 14:23:58 2013 -0700
SQOOP-773 Sqoop2: Batch execution support for client commands
Separated options into two groups: fixed and dynamic options.
Fixed options (IE: connector ID) come first and are used to select
what options should be used in dynamic options. Dynamic options
are automatically created based on forms selected from fixed options.
The keys for these options take on the form "<prefix>-<form name>-<input-name>".
:100644 100644 0538901... 7f5df34... M client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java
:100644 100644 6f62813... 32f8c3f... M client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java
:100644 100644 ac555e1... c842c4d... M client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java
:100644 100644 04b240c... f23e479... M client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java
:100644 100644 cc4d546... 7b40645... M client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java
:100644 100644 18d3a70... 5220d61... M client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java
:100644 100644 736be20... e82a47b... M client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java
:100644 100644 e04292a... f6cd6e3... M client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java
:100644 100644 5962cd2... 6ea9f0c... M client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java
:100644 100644 ed6dc3c... 094438b... M client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java
:100644 100644 9e4e320... fb75fa8... M client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java
:100644 100644 e843ede... ba89724... M client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java
:100644 100644 41fc17a... a241419... M client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java
:100644 100644 94f92b3... c11b4f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java
:100644 100644 b053339... 97e4b9d... M client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java
:100644 100644 58b2c6e... 362d981... M client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java
:100644 100644 97a240b... 9222f50... M client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java
:100644 100644 81c5612... 39cf3f3... M client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java
:100644 100644 110f67e... ef4c907... M client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java
:100644 100644 666eb7a... 6933965... M client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java
:100644 100644 8e17f67... ba65a3c... M client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java
:100644 100644 bf26761... be06116... M client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java
:100644 100644 2e1c8d3... 7b4f7db... M client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java
:100644 100644 b854a90... a4808a2... M client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java
:100644 100644 49ab461... 3a4905f... M client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java
:100644 100644 8556e2b... 15cc722... M client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java
:100644 100644 425a53f... 4edb306... M client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java
:100644 100644 2fbf129... 711865d... M client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java
:000000 100644 0000000... 38e63ba... A client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java
Diffs (updated)
-----
client/src/main/java/org/apache/sqoop/client/shell/CloneConnectionFunction.java 0538901
client/src/main/java/org/apache/sqoop/client/shell/CloneJobFunction.java 6f62813
client/src/main/java/org/apache/sqoop/client/shell/CreateCommand.java ac555e1
client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java 04b240c
client/src/main/java/org/apache/sqoop/client/shell/CreateJobFunction.java cc4d546
client/src/main/java/org/apache/sqoop/client/shell/DeleteConnectionFunction.java 18d3a70
client/src/main/java/org/apache/sqoop/client/shell/DeleteJobFunction.java 736be20
client/src/main/java/org/apache/sqoop/client/shell/DisableConnectionFunction.java e04292a
client/src/main/java/org/apache/sqoop/client/shell/DisableJobFunction.java 5962cd2
client/src/main/java/org/apache/sqoop/client/shell/EnableConnectionFunction.java ed6dc3c
client/src/main/java/org/apache/sqoop/client/shell/EnableJobFunction.java 9e4e320
client/src/main/java/org/apache/sqoop/client/shell/SetOptionFunction.java e843ede
client/src/main/java/org/apache/sqoop/client/shell/SetServerFunction.java 41fc17a
client/src/main/java/org/apache/sqoop/client/shell/ShowConnectionFunction.java 94f92b3
client/src/main/java/org/apache/sqoop/client/shell/ShowConnectorFunction.java b053339
client/src/main/java/org/apache/sqoop/client/shell/ShowFrameworkFunction.java 58b2c6e
client/src/main/java/org/apache/sqoop/client/shell/ShowJobFunction.java 97a240b
client/src/main/java/org/apache/sqoop/client/shell/ShowOptionFunction.java 81c5612
client/src/main/java/org/apache/sqoop/client/shell/ShowServerFunction.java 110f67e
client/src/main/java/org/apache/sqoop/client/shell/ShowSubmissionFunction.java 666eb7a
client/src/main/java/org/apache/sqoop/client/shell/ShowVersionFunction.java 8e17f67
client/src/main/java/org/apache/sqoop/client/shell/SqoopFunction.java bf26761
client/src/main/java/org/apache/sqoop/client/shell/StartJobFunction.java 2e1c8d3
client/src/main/java/org/apache/sqoop/client/shell/StatusJobFunction.java b854a90
client/src/main/java/org/apache/sqoop/client/shell/StopJobFunction.java 49ab461
client/src/main/java/org/apache/sqoop/client/shell/UpdateConnectionFunction.java 8556e2b
client/src/main/java/org/apache/sqoop/client/shell/UpdateJobFunction.java 425a53f
client/src/main/java/org/apache/sqoop/client/utils/ConnectionDynamicFormOptions.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/utils/DynamicFormOptions.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java 2fbf129
client/src/main/java/org/apache/sqoop/client/utils/FormOptions.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/utils/JobDynamicFormOptions.java PRE-CREATION
server/src/main/java/org/apache/sqoop/server/v1/ConnectionServlet.java e8af4d7
server/src/main/java/org/apache/sqoop/server/v1/FrameworkServlet.java 9c99731
server/src/main/java/org/apache/sqoop/server/v1/JobServlet.java 34a0ffb
server/src/main/java/org/apache/sqoop/server/v1/SubmissionServlet.java 7252e11
Diff: https://reviews.apache.org/r/13181/diff/
Testing
-------
Executed the following commands in a file:
reate connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hu\
e.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageTy\
pe 0 --framework-output-outputFormat 0 --framework-output-outputDirectory /tmp/output
show job
start job --jid 1
Thanks,
Abraham Elmahrek